`min` is presumptuous (on empty iterables) about problem-solving

[niner]

I would like to point out that nothing of consequence happened. No design was chosen. No commits were merged to rakudo's master branch. All that happened was some discussion, some learning about design docs, some brainstorming and experimentation with code.

That's actually what the whole problem-solving process is about. Someone states a problem and creative minds explore solutions and hopefully we come up with a design that takes all different aspects into account. It may even be your intuitively picked favorite solution with the added bonus of a whole group having high confidence that it's also the right solution.

[Leont]

This sounds to me like the kind of case were returning Any/Nil makes most sense TBH.

[raiph]

min returns Inf on empty Iterables, even though there is no reason to assume Numeric context.

Where did Numeric context come into things?

Quoting from Raku's design documents:

any value of any type may be compared with +Inf or -Inf values, in which case the infinite value stands for "larger/smaller than any possible value of the type."

"foo" min +Inf              # "foo"
"foo" min -Inf              # -Inf
"foo" max +Inf              # +Inf
"foo" max -Inf              # "foo"

All orderable object types must support +Inf and -Inf values as special forms of the undefined value.

[duncand]

Responding to initial comments, I had the impression that +Inf and -Inf were NOT numeric specifically, but rather were generic for all data types that were orderable, so they would canonically sort after or before every other value in the type system.

[salortiz]

These "numerically" in the documentation must be removed as min and max use, by default, cmp (structural comparison) semantics that is well defined for Any type in core.

cmp always returns an Order:D, but emit a warn when a Any:U is used.

So certainly min and max could issue a 'warn' when none of its arguments are defined. Empty iterables are only the simplest case.

… some sentinel values that would numerically / stringwise would always be less / more than what they're being compared to

In the context of cmp (and by extension min and max) those sentinel values already exists:

say ("", "foo", -Inf, Inf, 100, -100).min; # -Inf
say ("", "foo", -Inf, Inf, 100, -100).max; # Inf

On the other hand, the actual values returned by min and max when they have no defined values to work with, "compose" well with the Range constructed by minmax:

say ().minmax cmp Empty; # Same
say ().minmax.list ~~ Empty; # True

If other values are chosen, they must retain that "closure" property.

Finally, although it is not formally documented, the user can always change the semantic of min and max, providing the comparison operator (function of arity 2) that he needs. For example <=> or leg to force Numeric or String context:

say (8, 200, 10000, NaN).min(&infix:<leg>); # 10000

[lizmat]

Actually, trying to implement this, and seeing nine's response on channel, I'm starting to think that adding special infix candidates (like done in this PR), but instead use Inf for Most, and -Inf for Least would actually make the most sense.

Still mulling over this, though :-)

[2colours]

Disclaimer: the same problem affects max as well, which in turn returns -Inf.

I think, without any adaptation that might lead far, the best option would be to just use Nil.

1. it's conceptually compatible with all possible types
2. it doesn't implicitly turn into legitimate values of the intended type - confer Empty which could sneak in as 0 or '', directly violating min semantics
3. it is a falsy, furthermore undefined value so it's easy to pinpoint without awkward unreasonable boilerplate like if $foo == Inf would be for a supposed string, pair etc.

It's not a perfect choice because it can also serve as a reset value of a container which then can be a defined value - however, I think it's overall more feasible than to try to define a "natural minimum/maximum" for everything that might appear in the domain of min, or to just downright ban it on empty iterables (which results in the same boilerplate that seems like the best solution with Inf as well).

[lizmat]

How about it returning a NumStr ? NumStr.new(-Inf,"") and NumStr(Inf,chr(0x10FFFF))

FWIW, I think the use of Inf and -Inf actually predates the existence of allomorphs.

[niner]

That would only cover one more type of input and I'm not sure it would even cover that well (after all a string consisting of multiple 0x10FFFF characters is a valid input). I think the problem is that a "natural" minimum/maximum only really exists for numbers. Or rather that for numbers this "natural" minimum/maximum can come in quite handy and helps avoiding additional tests. So we want that in numeric context but it hurts in all other contexts. Sadly an empty input list doesn't tell us anything about the context the result will be used in :(

[lizmat]

The alternative would be to create some sentinel values, similar to IterationEnd, that would numerically / stringwise would always be less / more than what they're being compared to (by adding the right candidates to the comparison infix ops). And have min and max return those values.

[lizmat]

Proof of concept, for 6.e:

constant MinInf = Any.new;
multi sub infix:<gt>(MinInf,$ --> False) { }
say MinInf gt "";  # False

[2colours]

I would advise against overly protecting the status quo, as if it had some inherent value. Mind you, we are talking about something that is merely an (apparently controversial) API-level design decision, not something essential or hardwired. And I open problem solving issues with this in mind; generally presenting arguments why I think the behavior is problematic and coming up with a possible resolution.

@raiph

Where did Numeric context come into things?

Inf is a Num even according to the quoted design documents. I think this says it all. The minimum element of a list of Strs, even if empty, should never return a Num.

Quoting from Raku's design documents:

First, "Raku's design documents" are outdated, abandoned and not really followed anymore. From what I know, the only authoritative specification is Roast. Actually, this behavior is following the design documents and present in Roast - the very code snippet you included. I opened this issue as a problem solving issue exactly because I propose a change in the specification.

Second, I looked up that part of the "old design documents" - please make sure to include links because it can be quite bothersome to search them; the named part comes here. I don't think you should have left the first sentence of that paragraph off:

Not all types can support the concept of infinity. Therefore any value of any type may be compared with +Inf or -Inf values, in which case the infinite value stands for "larger/smaller than any possible value of the type."

In my opinion, the design document doesn't even sound convincing - especially if we consider what I opened the issue for. My main concern isn't what 'foo' min -Inf should return - my concern is that, despite apparently acknowledging that not all types support the concept of a natural maximum or minimum, yet an Inf can leak from, well, not comparing anything at all.

By the way, I don't know if this is specced or not but apparently 'foo' is treated as the second biggest number after Inf - 'foo' min $n returns $n for any other number. Is this interpretation bizarre? I'd think so but for me, saying that the smallest string of an empty list is Inf is just as bizarre.

All orderable object types must support +Inf and -Inf values as special forms of the undefined value.

I could actually agree with this but I don't think this is what we have currently. The types are "trying to play nice" with the one and only Numeric (floating point) Inf but they don't actually have +Inf and -Inf values they could use natively.

Consider this piece of code:

my $input = prompt 'Insert your words here:';
my Str $first = $input.words.min

This will actually fail with a type error if the input only contains whitespace. I don't think this is what "support" means.

@salortiz

These "numerically" in the documentation must be removed as min and max use, by default, cmp (structural comparison) semantics that is well defined for Any type in core.

Definitely. Ironically, I would argue the confusion could have arised from this "default to Inf" behavior.

So certainly min and max could issue a 'warn' when none of its arguments are defined. Empty iterables are only the simplest case.

This is insightful, I wouldn't have expected that, either. But I'm not using undefined values in Iterables, usually. And I'd assume most people also don't, most of the time. Therefore I think that could also be an interesting discussion but for now, I would focus on this particular case that I myself have used twice in this month.

If other values are chosen, they must retain that "closure" property.

I think this is a fair argument but I think it's a bit overboard to declare a "must". I for one would trade this "closure" property for type consistency, as demonstrated in the snippet above, at any moment. Apparently, type consistency wasn't a "must", although I am pretty sure that would be more useful for more people who ever come across these corner cases.
Anyway, I think supporting that "closure" property shouldn't be too hard, e.g. by defining Nil as a range endpoint, or simply specialcasing an empty range for minmax (this solution would probably be more fragile out of the two).

By the way, it's also a little interesting for a consequence that (Nil, Any, Empty, Str).minmax.list ~~ Empty, isn't it? Not sure what result I would expect but it did eliminate the elements big time, that's for sure. :)

Also, I think if we scratched the surface a little harder, we could bump into more not-so-strangely-consistent things: (1, 'foo') has a min of 1, a max of 'foo' but it fails to provide minmax because such Ranges cannot be constructed in Raku (for now, at least).

[niner]

On Wednesday, 30 November 2022 03:24:48 CET Márton Polgár wrote: I would advise against protecting the status quo, as if it had some inherent value.

But it does? After all we promised to stay backwards compatible to existing code when we made the 6.c release. Granted, we have broken it quite a few times already, but always with reluctance and good reason.

> Quoting from Raku's design documents: First, "Raku's design documents" are outdated, abandoned and not really followed anymore.

While the design documents aren't kept up to date anymore, they still have worth. They often explain the reasoning behind the specification (i.e. Roast). In this case they explain why `Inf cmp "cat"` returns `More` and point at a way out of this conundrum. Maybe everything actually is fine just as it is and all we need is better documentation.

my $input = prompt 'Insert your words here:'; my Str $first = $input.words.min ``` This will actually fail with a type error if the input only contains whitespace. I don't think this is what "support" means.

That would be a bug then, as it doesn't follow the "All orderable object types must support +Inf and -Inf values as special forms of the undefined value" rule.

[2colours]

But it does? After all we promised to stay backwards compatible to existing
code when we made the 6.c release. Granted, we have broken it quite a few
times already, but always with reluctance and good reason.

Honestly, I don't think I can relate to this statement. I can only say that it definitely doesn't look like this promise have been kept, or even consistently followed in the core. The renaming and all that it involved/involves is the dead horse we can always beat but I could refer to the "build-depends", "test-depends" deprecation in META.json that ugexe didn't even notice back in the day, or the issue I opened about in fact revoking a semi-implemented deprecation. So I feel by now, only the casual reluctance to change things remained from that policy...

While the design documents aren't kept up to date anymore, they still have worth.

I wholeheartedly agree with this, and they indeed explain a lot of things about hyper metaoperators that I opened a humongous issue about. It could have helped greatly, still could. For me, that's still an open issue, though.

In this case they explain why Inf cmp "cat" returns More and point at a way out of this conundrum.

I have already said why I think that is not the case here. The document states that not all types support the concept of a minimum and a maximum - and reaches the conclusion that all types need to integrate +Inf and -Inf? I don't think this is a good explanation. It is not only vague but to me, it sounds contradictory, even - therefore unless it's authoritative (which it isn't), I don't find much value in quoting it. Unlike many other cases, of course.

That would be a bug then, as it doesn't follow the "All orderable object types
must support +Inf and -Inf values as special forms of the undefined value"
rule.

As I brought it up on the raku-dev IRC as well, I think that ship has actually sailed "much harder" than changing the return value of min and max for empty Iterables, at least. If these are floating point, Num values, what sort of special-casing would it take in the type system to allow them as undefined values for another type? Currently, not even Num would pass as an undefined value for Str or List, and at the same time Inf is defined which might even make sense if we look at it as a Num value. I honestly can't even imagine the magic involved to make a Num value pretend it's undefined for a different type, and possibly also pretending that it's defined for Num and possibly other Numeric types as well... how many breaking changes would that "fix" involve?

[Leont]

As I brought it up on the raku-dev IRC as well, I think that ship has actually sailed "much harder" than changing the return value of min and max for empty Iterables, at least. If these are floating point, Num values, what sort of special-casing would it take in the type system to allow them as undefined values for another type? Currently, not even Num would pass as an undefined value for Str or List, and at the same time Inf is defined which might even make sense if we look at it as a Num value. I honestly can't even imagine the magic involved to make a Num value pretend it's undefined for a different type, and possibly also pretending that it's defined for Num and possibly other Numeric types as well... how many breaking changes would that "fix" involve?

Yeah, all of that. This just isn't a workable concept.

[salortiz]

The beauty of the matter is that all this is implemented directly in Raku and better than discussing it is presenting a concrete proposal. Lizmat is already working on a draft of what could be a solution for v6e, let's wait and see how it progresses.

[lizmat]

Adding:

multi sub infix:<eq>(Inf, Inf --> Bool::True) { }
multi sub infix:<eq>(Inf, \b --> Bool::False) { }
multi sub infix:<eq>(\a, Inf --> Bool::False) { }

to the core makes this test fail in roast:

ok( $x eq 'Inf', 'string equal');

So this will probably at least need to be moved to 6.e only.

[salortiz]

Why would you need to modify the semantics of eq?
This is basically cmp, min, max and minmax territory, no?

[lizmat]

because if you expect the result of a (...).min to be a string, you would be using eq for testing

[salortiz]

I don't expect the result of min to change until 6.e.

[lizmat]

Oh, definitely... I was just trying out stuff :-)

[2colours]

Actually, trying to implement this, and seeing nine's response on channel, I'm starting to think that adding special infix candidates (like done in this PR), but instead use Inf for Most, and -Inf for Least would actually make the most sense.

I either don't understand what you mean, or I can't see how that would address the very issue of empty Iterables returning Num-typed Inf values, regardless the actual context. I've had an issue that had completely the opposite resolution what I opened it for, please, not again.

[lizmat]

@2colours We try to be data driven here. The fact that a previous issue that you opened didn't get the resolution you wanted, will never affect the resolution of this issue. So saying:

had completely the opposite resolution what I opened it for, please, not again.

is what I would call, rather immature (put rather more radically than I normally would)

[2colours]

@lizmat I don't know if the banally rational logic of "I won't open issues if I can anticipate the opposite will be done to what I want to achieve" is immature or not, However, I could have elaborated more on what I don't want to be repeated.

I presented a whole reasoning why something that did work made sense and was useful and basically all of it was neglected in the nature of "something that was supposed to be useless incidentally turned useful, fix the regression". I put effort into reasoning for an improvement and got slapped by the dogma.

This is what I mostly want to avoid. Ending up on the "losing side" of a constructive discourse is fine, however, if we fall back to "how about somehow save Inf in comparisons" while ignoring all the incoherence that arises around it, I think that would be more like getting slapped by the dogma, again.

Also, please keep in mind that problems and solutions are two different things. I opened the issue for one particular problem: empty Iterables not knowing what kind of data they stand for, and presumptiously deciding on certain Num values, Inf and -Inf to be used as minimum and maximum. I had a proposal regarding that (returning Nil instead), which Leon seemed to support. Then we rallied off to the broader topic of Infs as universal sentinel values, and there has been some brainstorming about how to "support Inf values for all ordered types", as the "old design documents" declared. I don't see how this addresses the issue I raised, and it doesn't seem like anyone has a simple idea how to support the type safety of the my $first = $input.words.min snippet and all of this kind.

So I'd like to ask you all to either explicitly state that you consider the original issue a non-issue about a feature - I'd be happy with an attached reasoning in this case - or please propose solutions that actually address the issue.

[lizmat]

I'm closing this issue now. Please re-open / comment if you disagree.

[2colours]

This is still an issue but it cannot be re-opened...

[2colours]

Responding to initial comments, I had the impression that +Inf and -Inf were NOT numeric specifically, but rather were generic for all data types that were orderable, so they would canonically sort after or before every other value in the type system.

This is directly contradicted by calling Inf.WHAT. It is a Num.

In fact, from what I know, it is specifically the infinity of some specific floating point number standard.

I don't know if this is actually specified in Roast but I don't even think having a proper, typed floating-point infinity is a problem, and in the light of this, it sounds needlessly painful to repurpose a floating-point infinity value to a universal maximum or minimum.

[raiph]

This is directly contradicted by calling Inf.WHAT. It is a Num.

Yes it's a Num. No that doesn't make it a direct contradiction.

[2colours]

Yes it's a Num. No that doesn't make it a direct contradiction.

I cannot do anything with this statement. There is literally nothing we can talk about if we can't agree on that. The whole discussion goes to the trash bin.