"Failure is Nil" and the anomalies this leads to about problem-solving

[TimToady]

Nil is a kind of singularity. Black holes have no hair, and trying to fit them neatly back into the fabric of the universe results in contradictions. Similarly, attempting to fit Nil back into the normal type system is bound to cause problems, which is why we have Raku's Nil rather than Perl's undef. Perl programmers often fall into the trap of trying to store undef as a normal value, and get themselves into trouble down the road. Black holes can also have an accretion disk (and maybe a "firewall"), which records some of the history of what is falling into the black hole. This is what a Failure does; it records some of the history of a singularity. The stuff still falling in is where black holes can keep their "hair", at least temporarily. The design of Nil with respect to Failure was primarily a cultural decision, not a technical decision. We do not want people thinking of Nil as a value, or as a type from which other user-defined types can be derived, other than user-defined Failures. We want them to think of Nil as the least-marked form of failure, so that they never treat Nil as some kind of nuanced success marker, or try to create a parallel system of nothingness that pretends to be storable as a value. We could conceivably redefine Nil and Failure as both forms of Singularity (the property which subverts the return type system of the universe by the permanent absence of a return value), but the universe discourages people from creating their own singularities, and I suspect we should too. Hope this helps... Larry

…

On Wed, Aug 10, 2022 at 7:24 AM Vadim Belman ***@***.***> wrote: What got in the way? I don't actually remember any prior complains about this subject. So, in my memory you're the first. But my memory doesn't span beyond 2018. Anyway, just to leave a quick note here, without actually considering all the consequences of the move. Nil and Failure could, possibly, be "divorced" if given a property in common. That property could be a special role to mark typecheck-transparent types. This way I could probably even like it, unless it would cause havoc in the existing code. Unfortunately, that might not be easy to implement even as a 6.e+ feature. I foresee a number of issues there. — Reply to this email directly, view it on GitHub <#342 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABHSYWEGDBKJHUAFEXRIWTVYO3R7ANCNFSM55YXFYIA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[MasterDuke17]

I see now that you were talking about my Int $a? I doubt the behavior of Nil will change. And I have to say it works the way I expect it to. Maybe the solution would be introducing a new type, like Nothing. I'm not sure.
…

I’ve sometimes felt that ‘Empty’ should be used a bit more.

[vrurg]

What got in the way?

I don't actually remember any prior complains about this subject. So, in my memory you're the first. But my memory doesn't span beyond 2018.

Anyway, just to leave a quick note here, without actually considering all the consequences of the move. Nil and Failure could, possibly, be "divorced" if given a property in common. That property could be a special role to mark typecheck-transparent types. This way I could probably even like it, unless it would cause havoc in the existing code.

Unfortunately, that might not be easy to implement even as a 6.e+ feature. I foresee a number of issues there.

[salortiz]

Nil is both a Type and a value, thinking of it as "absence of value" lacks a fundamental part: It is "absence of defined value".

Nil, besides always being undefined, is special in two important and orthogonal ways:

• The value Nil, when assigned to Scalar containers ask them to take its declared default value.
  A simple my $can-be-nil is default(Nil); suffice for Nil to stuck at assignment.
• Values of Type Nil and its subclasses are exempt from return type constraints.

The second is the reason that Failure is a subclass of Nil. A Type hierarchy that allows us to handle exceptions in much more terse ways, without CATCH and try blocks.

IMO the fact that any Routine, regardless of its declared return Type, can fail instead of die is an elegant property of raku´s type system, not a hole.

[lizmat]

Also note that Empty is definitely DEFINITE but not defined:

say Empty.DEFINITE;  # True
say Empty.defined;   # False

[vrurg]

To be precise, empty Slip is undefined: say ().Slip.defined;

[lizmat]

I would really like feedback from @TimToady @thoughtstream @jnthn on this.

I know Jonathan is on vacation and offline for the next weeks, so I'm not expecting anything from them for a while.

[CIAvash]

FWIW you can bind Nil:

my $nil := Nil;
say $nil.^name;
=output Nil

But if type and type safety is important, then I guess you should do at least one of two things:

• Specify the type on your variable, so Nil sets its value to the type, when there is no value
• Instead of Nil, return the type object

If I'm missing something, ignore my comment.

[lizmat]

@CIAvash Why wouldn't you be able to bind Nil?

If you wouldn't be able to do that, something like my $a := @b.first(/ ... /) would die if no match could be found.

Re being able to do one of two things: isn't that already the case?

my Int $a := Nil;
# Type check failed in binding; expected Int:D but got Nil (Nil)
my Int $a = Nil;
say $a;  # (Int)

Or am I missing something?

[CIAvash]

@CIAvash Why wouldn't you be able to bind Nil?

You can, I mentioned it in case @2colours wasn't aware of it.

Re being able to do one of two things: isn't that already the case?

my Int $a := Nil;
# Type check failed in binding; expected Int:D but got Nil (Nil)
my Int $a = Nil;
say $a;  # (Int)

Or am I missing something?

You are correct. I'm saying that if your function is returning Nil and you want type safety, then your variable should have a type:

my Int $a = some_function_which_returns_nil;
# Instead of
my $a = some_function_which_returns_nil;

And the other one, your function can return Int type object instead of Nil, if you don't want to specify the type of the variable:

my $a = some_function_which_returns_defined_integer_or_Int;

And about the example given about [].pop. I'm not sure what it's showing about the issue:

my $subnil-assigned = [].pop;
say $subnil-assigned.^name
=output Failure

say $subnil-assigned;
=output Cannot pop from an empty Array

Again, it's possible I'm misinterpreting things, or not understanding the issue.

[2colours]

Yes, I "knew" (or at least didn't assume the opposite) that Nil is available as a value - after all, when you aren't using containers, the whole reasoning about "assignment to containers" won't apply.

I would say it's a good workaround that you hardcode (Type) when you want function return types, still just a workaround. To provide further context: this came up in relations with a supposed ?. operator which should be able to aggregate missing values. Now that we have typed missing values, it would be good to achieve that in such a (theoretical? eventual?) situation as well. This is another case where Nil as a function return value came up and since it signifies a missing value, it would be both convenient and logical that it doesn't leak untyped but wraps into the right type, like with containers.

And about the example given about [].pop. I'm not sure what it's showing about the issue:

It's just a hilarious illustration that semantically Failure is not Nil, disproving the values of the smartmatch swiftly. They have none of the absence semantics which is so essential to Nil. It's really just funnier that the exact behavior ends up reading like "Nil is not Nil but something not Nil is indeed Nil".

[CIAvash]

Am I correct to assume that the main goal of this issue is that you don't want functions to be able to return Nil, but instead return the default value/type of the return value/type of the function?

I would say it's a good workaround that you hardcode (Type) when you want function return types, still just a workaround.

I wouldn't say it's a workaround, type objects in Raku are like option types from other programming languages; if there is a defined value, then it's Some(value)(Rust) or Just value(Haskell) and if there is no value, then it's a type object which is similar to None(Rust) or Nothing(Haskell).

[vrurg]

The Liskov Substitution Principle is a theory. But in practice when something is subclasses then the reason for a subclass to exist is to change the something and make it behave differently. In my view, what is missing from the short definition of the principle is the scope. I would specify it as what the existing environment expects from a type object must be provided by its subclasses. In a short form.

What it means for Nil/Failure is that the environment expects for them to be transparent for type checks. And that's, basically, it. All other expectations are formulated specifically for each type. Therefore there are no discrepancies or holes.

The case of assigning is also solved by the scope definition.

Nil/Failure in their current form allow to solve some tasks which are not solvable otherwise. This and the fact that they are too widely used by the existing codebase make any changes in this area basically impossible.

[2colours]

I wouldn't say it's a workaround, type objects in Raku are like option types from other programming languages;

I'm not even talking about that (even though I'm also not really happy about the way it's solved in Raku). All I'm saying is: if a function that has a return type returns Nil, it should either be the proper typed absence value returned, or a warning (possibly an error) should be issued for returning a not type safe value from the function. It would be elegant, perhaps even useful, to be able to return Nil and get the type wrapping for free.

[2colours]

What it means for Nil/Failure is that the environment expects for them to be transparent for type checks. And that's, basically, it. All other expectations are formulated specifically for each type. Therefore there are no discrepancies or holes.

I think this is a very arbitrary and implementation-centered approach. Read the documentations: https://docs.raku.org/type/Nil "Absence of a value or a benign failure", "The value Nil may be used to fill a spot where a value would normally go, and in so doing, explicitly indicate that no value is present."
This is a good description, and this what concerns the user. Of course you will find common things in the implementation - it would be a big problem if related things have completely unrelated implementations - but why should it concern the supposed user if they are transparent for type checks or not?

Anyway, what value does it add for a plain Nil and a Failure that they are transparent for type checks? This is what matters, after all, not the sole fact that they do. For a Failure, the added value is that you can return them as a sort of non-throwing exception; for a Nil, it's arguably that it can substitute anything missing, regardless of the type system. These are still unrelated motives.

Nil/Failure in their current form allow to solve some tasks which are not solvable otherwise. This and the fact that they are too widely used by the existing codebase make any changes in this area basically impossible.

This is why I asked the questions. What would be the cost to break them up, and if this cost is hard to estimate or just not worth it - what was the perceived benefit of basing non-throwing errors (that are very much present and act as such) on the absence value? Not going to lie: the behavior seems insane enough that it would require strong justification and a reasoning that one can just recite in case such a question arises.

[vrurg]

the behavior seems insane enough that it would require strong justification

I totally disagree here. It goes against your expectations – perhaps, why not? But it fitted mines almost perfectly from the start. So, that's very questionable statement of yours! :)

related things have completely unrelated implementations

Not true, in my view. As I said, there are certain expectations from Nil, and there are expectations from Failure. The common subset of properties is what is the essence of Nillish types. That's it.

What would be the cost to break them up

I'm somewhat busy to get deeper into theory and point of views. Besides, I think Jonathan may cover this part way better that I would. But I can tell you about the cost. It's going to be about overall performance of Rakudo runtime. Where it currently does a single typecheck two would be necessary. Where the proposed defaulting needs to take place more opcodes would be injected to replace the returned value. It's hard to say if we talk about 1%, or 10%, or whatever else. But the ubiquitousness of typechecks across the core gives me rather unfortunate estimations. :)

[2colours]

I totally disagree here. It goes against your expectations – perhaps, why not? But it fitted mines almost perfectly from the start. So, that's very questionable statement of yours! :)

Ask anyone about the snippet I embedded in the issue-opener. I wonder how many of them would say that it looks sane to them. You are free to add why one works this way and why one the other way - I still don't expect many of them would say it's perfectly okay for Failures to be Nil in this context...

[CIAvash]

I see now that you were talking about `my Int $a`? I doubt the behavior of `Nil` will change. And I have to say it works the way I expect it to. Maybe the solution would be introducing a new type, like `Nothing`. I'm not sure.

…

-------- Original Message --------

On Aug 7, 2022, 1:54 AM, Márton Polgár wrote: > I wouldn't say it's a workaround, type objects in Raku are like option types from other programming languages; I'm not even talking about that (even though I'm also not really happy about the way it's solved in Raku). All I'm saying is: if a function that has a return type returns Nil, it should either be the proper typed absence value returned, or a warning (possibly an error) should be issued for returning a not type safe value from the function. It would be elegant, perhaps even useful, to be able to return Nil and get the type wrapping for free. — Reply to this email directly, [view it on GitHub](#342 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AAIDK7WWUEUPHJVNPQQ5GA3VX3JZLANCNFSM55YXFYIA). You are receiving this because you were mentioned.Message ID: ***@***.***>

[vrurg]

Ask anyone about the snippet I embedded in the issue-opener.

I'm not sure it's not survivorship bias we're observing. Most of those who see no problem understanding the current behavior would just shrug their shoulders and proceed to their own business.

[2colours]

I'm not sure it's not survivorship bias we're observing. Most of those who see no problem understanding the current behavior would just shrug their shoulders and proceed to their own business.

However, I'm pretty sure we are observing survivor bias on your side because somebody who has possibly spent years with the internals of a language without getting sick of it and moving on, pretty much must say that it makes sense for them.

So yes, actually, I think "anyone" should rather include beginners and potential users in this case because those people are quite surely very well-represented who get it. They form the core members, after all.

[librasteve]

This has made me learn a bit more about Nil, via some tests (Welcome to Rakudo™ v2022.02.):

  1 # use Nil to reset $x on assignment
  2 my $x = Nil;
  3 say $x;                 #(Any)
  4 
  5 # is the new $x Nil? (hint - no since it was reset by assigning Nil)
  6 say $x ~~ Nil;      #False
  7 
  8 # what does [].pop make?
  9 dd [].pop;             #Failure.new(exception => X::Cannot::Empty.new(, ...
 10 
 11 # what does [].pop do? (hint - throws the exception)
 12 # [].pop;              #Cannot pop from an empty Array ...
 13 
 14 # what happens when I assign that to $y? (hint - get's swallowed)
 15 my $y = [].pop;
 16 dd $y;                  #Failure $y = Failure.new(exception => X::Cannot::Empty...
 17 
 18 # do the contents of $y smartmatch Nil?
 19 say $y ~~ Nil;      #True
 20 
 21 # what happens when I assign this 'Nil' (hint - does not do a reset)
 22 my $a = 42;
 23 $a = $y;            
 24 dd $a;                 #Failure $a = Failure.new(exception => X::Cannot::Empty...
 25 
 26 # what does a pure (none failing) sub look like?
 27 sub does-nowt() {}  
 28 my $w = does-nowt;  # ''
 29 say $w;               #(Any)
 30 
 31 # how about if I pop a Nil? (hint - same as $v = Nil)
 32 my $v = [Nil].pop;  
 33 dd $v;                  #Any $v = Any
 34 say $v ~~ Nil;      #False
 35 
 36 # remind me what is a child of what
 37 say Nil ~~ Failure; #False
 38 say Failure ~~ Nil; #True

Having hacked on raku for a bit, the new learnings from this for me were:

• assignment of a Failure gets swallowed which prevents it from throwing
• the Failure ~~ Nil smartmatch is quite surprising (not sure if this has any use)*

Overall, while I have some sympathy with the OP, I think these behaviours are pretty sensible and I would vote for not going through the pain of a change.

The takeaway (for me) is that [].pop wants to throw Cannot pop from an empty Array ... or return a Failure if it is assigned whereas [Nil].pop does not throw and just resets the lvalue Scalar if it is assigned.

Maybe one day I'll RTFM the Exceptions and Try/Catch bits of the raku doc ;-)

• EDIT: see comment from @salortiz below to very neatly explain this

[CIAvash]

The bottom line is that [].pop throws Cannot pop from an empty Array ... whereas [Nil].pop does not throw and just resets if it is assigned.

Because [].pop is a Failure which will throw when goes unchecked, whereas [Nil] resets before doing anything else and is [Any], because Arrays have containers.

[CIAvash]

Not sure why one of my comments wasn't submitted.
Even though this issue is about the behavior of Nil, here it goes:

Based on discussions here and on IRC and the fact that similar things(on typing) have been brought up before, I think we'll need a pragma called use StaticTypes or something like that to tell the compiler to do strong typing. Then it should act like Rust, Haskell, …. More types will probably be needed as well.

I don't know how feasible it is though.

[librasteve]

@CIAvash - this is what rust does:

// An output can have either Some value or no value/ None.
enum Option<T> { // T is a generic and it can contain any type of value.
    Some(T),
    None,
}

// A result can represent either success/ Ok or failure/ Err.
enum Result<T, E> { // T and E are generics. T can contain any type of value, E can be any error.
    Ok(T),
    Err(E),
}

// a minimal example
fn main() {
    let o: Result<i8, &str> = Ok(8);
    let e: Result<i8, &str> = Err("message");
    
    assert_eq!(o.ok(), Some(8)); // Ok(v) ok = Some(v)
    assert_eq!(e.ok(), None);    // Err(v) ok = None
    
    assert_eq!(o.err(), None);            // Ok(v) err = None
    assert_eq!(e.err(), Some("message")); // Err(v) err = Some(v)
}

Not sure you can just enable this (assuming that you really want to) with a pragma

[2colours]

Nil is both a Type and a value, thinking of it as "absence of value" lacks a fundamental part: It is "absence of defined value".

I don't think this is fundamental. What would the "absence of undefined value" even mean? Also, Type objects are generally values as well - Nil is not special by being a valid value of the Nil type.

(...) is special in two important and orthogonal ways

By pointing out that they are orthogonal, you are basically making my point. If it is, using your words, a fundamental property of Nil that it is the absence of a value that can implicitly fall back to a default, then picking the orthogonal property that it can avoid type checks as the sole basis of the Failure analogy, is conceptually faulty, and actually this doesn't seem hard to fix at all, if there is nothing hidden in this logic: separate the two orthogonal properties into two roles/classes. Nil inherits them both, Failure only inherits the second. Solved.

I was thinking about how I could phrase my problem with the way vrurg (and now implicitly you as well) assumed that sharing the "type check dodging" property is just enough reason to mix them together. It's as if you modelled a car as a ship with additional wheels, arguing that they are both machines that are used for transportation. This is true but a ship has other fundamental properties, too, like that it travels on the surface of water, and this is something that isn't true for a car. They are both vehicles, though.

And now I'd like to know it even more: what's wrong with my proposal? Honestly, I don't think I'm the first one who come up with it as it is rather commonsensical. What got in the way?

[librasteve]

much ado about nothing (?)

[2colours]

much ado about nothing (?)

Ask Heidegger. ;)

[2colours]

@TimToady it's always insightful to get some feedback from you, and I don't think anyone can have as distant memories about the design of Raku anyway as you do.

I still have the impression that we agree quite a lot on the concepts - this stands for @vrurg as well, concerning this "common ancestor".

We could conceivably redefine Nil and Failure as both forms of Singularity
(the property which subverts the return type system of the universe by the
permanent absence of a return value), but the universe discourages people
from creating their own singularities, and I suspect we should too.

This part is a good example: the conclusion I draw from this presupposed Singularity type is that it would be rather internal; even if not completely hidden, definitely discouraged to use directly, in any shape or form. We can have different reasons to come to this conclusion: I perceive it as a technicality while someone else could consider it really too "meta" and Heidegger-ish to be used by "mere mortals",

We do not want people thinking of Nil as a value, or as a type from which other user-defined types can be derived, other than user-defined Failures. We want them to think of Nil as the least-marked form of failure

On the other hand, this is something I don't get. Why are Failures good to go? Why are they allowed to break this principle that you shouldn't treat Nil as a kind of value? Where does the "absence" nature of Nil, backed up by its behavior when assigned to a container, get lost and turned into "the least-marked form of failure"? The same goes for the "accretion disk" analogy - if we start with the assumption that Nil is essentially the proto-failure, then the "accretion disk" that carries additional data makes sense, however, if we start with Nil being the complete lack of a value to be used, it is paradox: Failures do have data, just not valid data. Their data should be kept (and propagated in case), and I suppose this is why they don't reset the container to the default value, like Nil does.

The "complete absence of a value" interpretation isn't only backed up by the documentation and the assignment with containers - as it stands, a "void function" also returns Nil, not Empty or (Any) or anything of the likes. Conceptually, I think it's hardly tenable to say that all subroutines or blocks that don't return a value, "fail silently".
One step forward: what about sink? Would you say that it turns everything into a "failure" and the happy ending is when this "failure" stays silent (Nil value)? To make things more complicated, sink propagates Failures as Exceptions, hence highlighting how much "present" Failures really are compared to Nil, and also contradicting the "Nil is a failure without any additional data" narrative.

I elaborated on this point long but this is simply because sorting off disagreement or misunderstandings require the most care. I understand that Nil shouldn't be treated like something in the universe of values and hence it makes no sense to try to subject them to type checks. However, "the lack of values", which Nil apparently is and is not, at the same time, can be integrated to a type system, similarly to container assignments.

This, in particular:

sub demo(-->Int) {}
say demo # Nil

could and should work better i.e properly typed in my opinion. I haven't even written down Nil in the code, yet that's what I get, when I have a function with a type constraint. This shouldn't even have to know about Nil if that didn't have the "absence" semantics - anyway, aiming for a valid Int result and staying consistent with containered variables would be very desirable, in which case we should get (Int).

This is one of my main points: the burden Nil carries by having to act both as absence and failure - two things that are to be treated differently - is too big for it to get either of these tasks well.

[Kaiepi]

I don't like the semantics of Failure:D as a return value with regards to the type system in particular:

for (
    anon only gives(--> Int) { 0 },
    anon only sinks(--> Int) { Nil },
    anon only fails(--> Int) { fail },
) -> &value {
    CATCH { default { say .^name } }
    say my Int $ = value;
} # OUTPUT:
# 0
# (Int)
# X::TypeCheck::Assignment

But there is a consistency there somewhere. I see the properties of instances, type objects, Nil, and Failure as existing on a sort of spectrum:

0     1     2     3
o-----o-----o-----o
|=Instances |     |
      |=Types     |
            |=Nil |
                  |=Failure

With repects to type smileys:

• 0 is a valid :D
• 1 is a valid :U in a binding, in an assignment, and as a return value
• 2 is a valid :U in an assignment and as a return value
• 3 is a valid :U as a return value

That :_ covers in its entirety.

I haven't even written down Nil in the code, yet that's what I get, when I have a function with a type constraint.
This, in particular:

sub demo(-->Int) {}
say demo # Nil

could and should work better i.e properly typed in my opinion. I haven't even written down Nil in the code, yet that's what I get, when I have a function with a type constraint. This shouldn't even have to know about Nil if that didn't have the "absence" semantics - anyway, aiming for a valid Int result and staying consistent with containered variables would be very desirable, in which case we should get (Int).

Because no assignment is being made with &demo, you get the return semantics of :U instead of the more specific assignment semantics, so the default Nil return value is typed and remains as-is.

But note the complicated nature of :U in comparison to :D. Personally, when I write a :U, what I really want to write is some shade of grey towards 0 that would need to be typed with a dynamic subset, and is awkward to produce generically. Would adding :I, :N, :F type smileys to represent 0-1, 0-2, and 0-3 explicitly help the situation somewhat?

[salortiz]

@Kaiepi,

The important characteristic of Failure, inherited of Nil, is that any Failure:D is undefined, so as a return value is trivially testable;

for (
    anon only gives(--> Int) { 0 },
    anon only sinks(--> Int) { Nil },
    anon only fails(--> Int) { fail },
) -> &value {
    with (value) {
	say my Int $ = $_;
    } else {
	say "Undefined";
    }
}
# OUTPUT
# 0
# Undefined
# Undefined

The Raku documentation should put more emphasis in the difference between definite and defined,

[salortiz]

BTW, given the facts that

say Slip ~~ List; # True
say Empty ~~ List; # True

But, as expected

my @a is List = (), () ,();
my @b is List = Empty, Empty, Empty;
say @a; # (() () ())
say @b; # ()

In the OP's line of thought, an LSP violation can be argued: "Among all Lists, the actual Empty is the one that is not assigned to …", then "Empty don't act compatible with List".

Does that warrant creating another type for Empty? I don't think so.

There are certainly some subtle asymmetries in Raku, but as @TimToady once commented, "It's the asymmetries that make this universe an interesting one"

[2colours]

@Kaiepi I gotta respect the effort you have put into making it look more sensible but I have to say I don't find it sane that one needs to take assignment, binding and return values as three distinct phenomena to put it into any consistent table, and mainly that return values aren't any of the other two, is in my opinion a big enough red flag to begin with.

Because no assignment is being made with &demo, you get the return semantics of :U instead of the more specific assignment semantics, so the default Nil return value is typed and remains as-is.

Anyway, I think the important part is: "the default Nil return value". So it is the default value, after all, and we can ask once again: does it make conceptual sense that functions by default return "a silent error"? I'd argue that it makes no sense, hence we'd have to accept that it is an essential property of Nil is that it does represent the absolute lack of a value, QED. This conclusion doesn't have to be informed about the technicalities.

Then back to DEFINITE and defined.

This is a whole different issue but these two things would really deserve two different names that is harder to mix up, also to better see which one is involved in which situation.

From what I understand, .defined is higher level, and it doesn't take to be Nil to act as a missing value with by the .defined semantics (confer the example from Lizmat). The docs also just say:
A few types (like [Failure](https://docs.raku.org/type/Failure)) override defined to return False even for instances:
What about .DEFINITE, then? Well, even less data available in the docs but it seems to me it really just means whether you have a ("described") instance of a class, or the type object:
You can use the .DEFINITE method to find out if what you have is an instance or a type object:

And here is another part where Nil has a certain behavior that Failure (and as far I'm concerned, no other class?) shares: the instances of Nil are not DEFINITE: Nil.new.DEFINITE returns False. Failure.new.DEFINITE returns true, on the other hand. This is probably related to Nil being such a universal default value, but it really reads like "Nil is a type that is its own only instance as well", and this is such a special property which again discourages any sort of inheritance at all from Nil.

A few examples:

Nil eqv Nil.new #True
Failure eqv Failure.new #False
class CoolNil is Nil {}
CoolNil eqv CoolNil.new #False
CoolNil.raku #Nil
CoolNil eqv Nil #False
CoolNil.new eqv Nil #True
CoolNil.new.DEFINITE #False

So CoolNil in itself is a "Nil" that isn't the real OG Nil, but its instances are, and its instances also refuse to acknowledge they are instances... aargh. This really just doesn't seem like something that should be done for any reason - I wonder how it's achieved that Failure can have proper instances that don't lie that they aren't instances.

[librasteve]

So my questions are:

why are Failures subclassed from Nil?
is it really worth it, given the also established "Nil is absence" semantics?
since Failures already don't act compatible with Nil, what would it take if they got torn from Nil? What would break from it in the core language? What would break at "end users" apart from the shown smartmatch (that is a really strong WAT competitor)?

IMO these points have been answered in this thread. N'est ce pas?

Nothing to see here, move along...

EDIT: I apologise for my over-emotional and intolerant comment.

[2colours]

The discussion is over when there is nothing more to talk about, or nobody to talk. I don't think any of these apply here. If you have nothing else to add, that's perfectly fine, you don't have to.

On CoolNil, it's existence is a consequence of a consistent and open "everything is a class" approach ... but it is not intended to be a user facing language feature.

And by the way, this is absolutely not true: Failures show exactly that inheriting from Nil is a legitimate and common thing to come across - it is exactly my point that Nil is, very blantly put, so weird that for this sole reason one should have stayed away from subclassing it. Also, it doesn't just straight follow from "everything is a class" that some subclasses of Nil will have non-definite instances and they will have two different Nils for the type object and the instance, the latter being the real Nil. Actually it's so weird that I would consider sharing it, if for no other reason, to see if this is known behavior at all.

[salortiz]

@2colours,

The Nil constructor is special, it deliberately returns its own type:

Nil.new === Nil; # True (eqv isn't the best tool for value identity testing)

Your CoolNil, a new different class, inherits Nil constructor so CoolNil === Nil is False but CoolNil.new === Nil is True.
BTW, the "reset to default" effect of Nil (as a value) in Scalar assignment isn't an intrinsic property of the Nil class, it is hardwired behavior of "scalar assignment" (but not "list assignment"), so your CoolNil.new will trigger it.

class CoolNil is Nil {}
my $a is default(0) = 8;
($a = CoolNil.new) ~~ Nil; # False
say $a; # 0
my &func is default(sub { 'def' }) = sub { 1 };
say func; # 1
&func = CoolNil.new;
say func; # 'def'
# Because
dd &func.VAR.WHAT; # Scalar

I wonder how it's achieved that Failure can have proper instances that don't lie that they aren't instances.

Any subclass can define its own constructor:

class OtherNil is Nil {
    method new { self.bless }
}
say OtherNil === Nil; # False (obvious)
say OtherNil ~~ Nil; # True
say OtherNil.new ~~ Nil; # True (by definition)
say OtherNil.new.DEFINITE; # True (expected)
dd OtherNil.new; # OtherNil.new

[2colours]

@salortiz I deliberately used eqv because if I used ===, someone would have immediately said "that acts upon pointer values and isn't a good idea to rely on it".... By the way, correct me if I'm wrong but eqv uses === by default anyway, so in this case it's just an abstraction on top.

Thank you for the explanation. At the end of the day, there is always something useful to learn. In the big picture of this issue, though, I think only the terms change as we keep digging deeper: if Nil silently but deliberately refuses to make actual instances, and hence it can never be DEFINITE, it doesn't seem to be a good idea to replace it with something that does make actual instances and can be DEFINITE...

say OtherNil === Nil; # False (obvious)

One more: this is not so obvious for CoolNil - also returned False but .raku gave 'Nil'.

[vrurg]

I don't find it sane that one needs to take assignment, binding and return values as three distinct phenomena

But they are different. Assignment and binding are different in what happens to a value with regard to the destination symbol. Return is not about assignment whatsoever. The only thing they have in common is type checking which is done the same way in all three cases.

Upd My mistake, assignment and binding do it differently from return. I mean, the type checking is not different, but what each operation does to the value is different. Where return is a transmission and it bypasses Nil as-is, assignment and binding are the terminal stages and as such they have to deal with a value somehow. But either way, they all know about Nil and in each case make their own decision as to how to deal with it.

does it make conceptual sense that functions by default return "a silent error"?

Have you tried to think of it as taking a value from a non-returning function is an error? Not a critical one though, hence the silence.

it doesn't seem to be a good idea to replace it with something that does make actual instances and can be DEFINITE

Again, the problem is that your scope of the problem is different, from what scope has been initially applied to Nil. Yet, speaking of Failures, they cannot be non-DEFINITE due to they carrying on a value which is expected to be an error description. As long as values are individual objects, their containers has to be different too.

And yet, a Failure isn't a useful value on its own because it is representing a problem not a solution. This is why it is not .defined.

Yes, these differences are subtle, but aren't we operate with subtle differences in our natural languages? We call them nuances, for example; or synonyms; or there are some other language constructs which allow us to differentiate between slightly different term, or emotional coloring, or whatever else one could think of.

One more: this is not so obvious for CoolNil - also returned False but .raku gave 'Nil'.

Nil is a black hole, remember? It doesn't fir the classical laws. CoolNil is your shiny foil wrap for it. You just left a gap in that wrap. Cover it:

class CoolNil is Nil {
    method new { self.bless }
    method raku { "[" ~ self.^name ~ " wrap around Nil]" }
}
class OtherNil is CoolNil { }
say OtherNil.new.raku;
say OtherNil.raku;

Because .raku is just a method sticking out on the edge of the event horizon...

[2colours]

But they are different. Assignment and binding are different in what happens to a value with regard to the destination symbol. Return is not about assignment whatsoever. The only thing they have in common is type checking which is done the same way in all three cases.

As I said: assignment and binding, so be it, but returning must be either binding or assignment, there is no third option. These are the two things you can set up a return value and these are the two options to tie the return value to a symbol from the caller's side. I think this made perfect sense in C++ - but anyway, what was the motive to create a third behavior that doesn't fit into the two "productive", actually available options?

Have you tried to think of it as taking a value from a non-returning function is an error? Not a critical one though, hence the silence.

I tried - it doesn't work, for me at least. Why would you be thinking that all functions start from a "soft error" position, if you can instead think that they are missing a return value altogether, which is not a failure, just a corner case?
And mind you, this is how Raku also behaves. If you assign this value, it will do something that has no signs of being invalid: resets the value. This doesn't make sense if we treat Nil as explicitly invalid - we don't want invalid things to hide behind valid behavior.

Yet, speaking of Failures, they cannot be non-DEFINITE due to they carrying on a value which is expected to be an error description. As long as values are individual objects, their containers has to be different too.

Exactly! This makes sense. A Failure is a value, just not a valid one. A true Nil is not a value.

Yes, these differences are subtle, but aren't we operate with subtle differences in our natural languages? We call them nuances, for example; or synonyms; or there are some other language constructs which allow us to differentiate between slightly different term, or emotional coloring, or whatever else one could think of.

First off, I don't like the general idea - we create languages to make things easier, not to repeat the same inconveniences natural languages develop because of their heavy exposure to social and historical events. Natural languages are not good enough for programming; I would argue they often aren't good enough for their own purpose but for programming, surely not.
Second, I still think this is about the ship and the car. "ship" isn't a synonym of "car", even though they are both vehicles. It's not a nuance that Nil is a lack of value (while Failure isn't) - it is part of the core behavior of Nil, anywhere you look.

.defined has somewhat fragile semantics in my opinion, given that an empty Slip can also be "undefined". I'm not saying it's bad per se, it's just hard to come up with a consistent interpretation, if valid BUT empty values can also be "undefined".
Maybe one could say: something is .defined if and only if it is a both present and valid value. In this interpretation:

• Empty is valid but not present
• Nil is not present (and it's a part of the argument whether it is valid or not)
• Failure is present but not valid

Therefore I think it makes sense that no matter how we look at Nil, with Failure they will both be not .defined. The difference is that I think it's a way too fundamental difference between Nil and Failure that Nil is not present and Failure is present. This is a part of the reason why .DEFINITE works differently for them, I suppose.

Nil is a black hole, remember? It doesn't fir the classical laws.

I didn't want to intervene yesterday but I did note in your presentation when you said "what if Nil can also be a valid value of " certain attribute. Maybe not with these exact words but you probably remember. :P So honestly, let us decide if Nil is something legitimate or a weird special value that one shouldn't build API's around.

[vrurg]

but returning must be either binding or assignment, there is no third option

"Even if you've been eaten up there is still two exits from the situation..."

Return is a transmission. It is certainly not an assignment or binding because what is returned doesn't end up bound to a symbol or an attribute.

when you said "what if Nil can also be a valid value of " certain attribute

A black hole is a physical object. Nil is a type object.

Otherwise it feels to me like we're starting a second lap here. So, my conclusion would be to give another reason to rattle about:

my $a = 1;
say $a === $a.Int;
$a = <1>;
say $a.WHAT; # IntStr
say $a ~~ Int; # True
say $a === $a.Int; # False

What would the Lipskov principle in the pure flat world say about it? It is broken because the allomorph cannot be used exactly everywhere where Int can be. But somehow we exclude the coercion case from the set of conditions and – voila! – allomorph conforms to the principle!

That brings me to the cause of all this misunderstanding: the scope. Acceptance or denial of Failure as a subclass of Nil is purely a matter of expectations or following the documentation and the specs (I simplify here, but it's not a manuscript format to cover all the nuances).

There is a point about disappointment or confusion. Well, the problem is: this world is a disappointment for all of us. If we can ever speak about expectations of a fetus, they are most certainly get broken after the birth.

I have my own disappointments, speaking of Raku, BTW.

So, I personally not convinced that there is real need in splitting the two. 🤷🏼‍♂️

[2colours]

It is certainly not an assignment or binding because what is returned doesn't end up bound to a symbol or an attribute.

Why should that matter? This arguably violates a bigger "substitution principle" than the one from Liskov. It means that you cannot exactly reproduce the behavior of a function return value. Literally. You had two ways to capture it and it does a third thing. Again, for what purpose? I again don't know a language that does this and it seems to go against any unwritten design principles in any language. Maybe the xvalue of C++ could be compared - but that falls into a different conceptual layer and it also has a low-level purpose.

A black hole is a physical object. Nil is a type object.

I don't know what you tried to say with this but my point is that you cannot say both "Nil can be a value worthy of storing" and "Nil cannot be expected to work like a proper type of value(s)" without either contradicting yourself or implying that you are one of the harbringers. 😂

What would the Lipskov principle in the pure flat world say about it? It is broken because the allomorph cannot be used exactly everywhere where Int can be.

This sounds like a strawman to me, not gonna lie. By the same chance, you might as well have said the Liskov's Substitution Principle is literally against inheritance because whenever you want to check the exact type of the object, you won't get the same value.
Now, you did something that strongly relies on exact type and the exact type is different by definition. But I mean... do you sincerely not feel the difference between "the exact types don't match" and "the thing users want to do most of the time - assignment, mind you - has contradicting outcomes for the two"?

Acceptance or denial of Failure as a subclass of Nil is purely a matter of expectations or following the documentation and the specs (I simplify here, but it's not a manuscript format to cover all the nuances).

It can be very well concluded from the documentation that this was a bad idea - and anyway, these things aren't either set into stone, nor any sort of authority to appeal to. "Fix the language, not the user", "Torture implementors on the user's behalf", just two thoughts from Larry Wall himself.

I would say that this is a "disappointed but not surprised" situation - the same reason I didn't push harder around ne and != not being proper operators, and a few others. As I "predicted" on the IRC:

• there will be some sentiments about the technical reason why it works this way (even when it's explicitly pointed out that this is a deliberate design decision and I'm targeting it as such)
• some unconvincing arguments to chew (I have pointed it out on several occasions that I see the main contradiction around the "just missing" behavior and the "documented error" behavior and I have given up on waiting for a response better than "but have you tried thinking that it's right behavior")
• eventually we will get to "c'est la vie, time to move on, deal with it" sort of remarks

This is indeed just a waste of time for all of us. Perhaps it would be better if I just collected these sort of eyebrow-raiser behaviors for myself and published a post somewhere whenever I find another ten of them.

[niner]

The "complete absence of a value" interpretation isn't only backed up by the a "void function" also returns Nil, not Empty or (Any) or anything of the likes. Conceptually, I think it's hardly tenable to say that all subroutines or blocks that don't return a value, "fail silently".

But you could say that those void functions or blocks "fail to return a value".

[CIAvash]

Based on discussions here and on IRC and the fact that similar things have been brought up before, I think we'll need a pragma called `use StaticTypes` or something like that to tell the compiler to do strong typing. Then it should act like Rust, Haskell, …. More types will probably be needed as well.

[2colours]

But you could say that those void functions or blocks "fail to return a value".

Indeed, "could say" that, however I think that's mostly a play with words, and even as such, it can be argued against: as someone who never climbed Chimborazo, it would be odd for me to state that I "failed to climb Chimborazo". I was never meant to do that.
This is a typical case for "void functions" - they don't "fail" in any sense "Failure" implies, they are simply invoked for the side effect, and continuing with the execution flow is a sign of apparent success. It doesn't have to be this way but Raku, along with all mainstream imperative languages, is designed this way.

By the way, just a couple of days ago, I noticed that sub empty { Nil with Nil }; empty returns Empty indeed. This seems to be because it's the same thing as the "list comprehension" mentioned in the transition guide from Python.
So now, all of sudden, the empty function doesn't fail to return a value...

I think what could still be a meaningful question is what MasterDuke already hinted: when to use Nil or Empty in the first place (let alone type objects :D).

CIAvash is also making a good point but I think eventually we'll need to define new topics for new discussions because "Failure is Nil" is a dead horse... For the time being, I have enough open topics that nobody seemed to care about - even though I think the one about hyper metaoperators would be generally important - so personally, I'm more trying to focus on refurbishment of the resources we already have. Still, please don't leave me out if those discussions actually happen. :P

[salortiz]

This is a typical case for "void functions" - they don't "fail" in any sense "Failure" implies

Any function that returns Nil in fact "fail" in the same sense that Failure implies.

In Raku that "fail" is well defined and observable using the phasers KEEP and UNDO:

sub nothing() {}

sub ToSucceed() {
    KEEP say 'Raku knows this succeeds';
    UNDO say 'Raku knows this failed';
    say "Executed";
}

sub ToFail() {
    KEEP say 'Raku knows this succeeds';
    UNDO say 'Raku knows this failed';
    say "Executed";
    Nil # This can be substituted by a call to nothing
}

This is important, for example, to the semantic of the let declarations.

In Raku the proper way to declare a "void function" is by explicitly annotate Nil as its return type. That results in the compiler optimize it forcing a sink context to its last statement (if any).

On another side, you should not be surprised by the value returned by Foo with Nil. In Raku, any conditional statement when not executed, as a expression returns Empty:

('hi' with Failure.new()) === Empty; # True
(42 with Any) === Empty; # True
(Any if False) === Empty; # True
(die 'Humm' unless True) === Empty; # True
(say 'Foo' without 42) === Empty; # True
Empty === do if False { die }; # True

Very useful because loops can constructs lists.

I'm beating the dead horse because this thread shows that, in some topics, Raku´s documentation needs a more formal treatment.

[2colours]

Again - these are "it works exactly as it is implemented" kind of technical details, not the user's concern. The way Raku is used builds upon "void functions" not causing any unexpected trouble, only expected side effects. If you are trying to use the return value of one, perhaps you weren't looking for "void function semantics" in the first place.

I have never seen let declarations used - now, checking them, it seems to me it produces the right result for the wrong reason.
Reading the docs:
"Refers to a variable in an outer scope whose value will be restored if the block exits unsuccessfully, implying that the block returned a defined object."
also
"Restores the previous value if the block exits unsuccessfully. A successful exit means the block returned a defined value or a list."
So the docs define success as "returning a defined value or a list", which is a stricter criterion than not returning Nil and its derivatives.

In Raku the proper way to declare a "void function" is by explicitly annotate Nil as its return type. That results in the compiler optimize it forcing a sink context to its last statement (if any).

It wasn't immediately clear what you mean but I think I got it eventually. By the way, this also implies that anything executed in sink context "produces Nil", and hence could be interpreted as a "silent failure". Once again: I don't know if this is true technically, but I'm confident it's not a useful mental framework, and the implications shouldn't show up at the user.

On another side, you should not be surprised by the value returned by Foo with Nil. In Raku, any conditional statement when not executed, as a expression returns Empty:

Honestly, it's not about the surprise. I rather like this behavior, even. However, it's very counter-intuitive that non-executed code produces Empty while no code produces Nil. It shows how fragile the "failed to produce value" interpretation is.

So yes you are right, it would be nice to make the documentation cover the concepts and corner cases better - however, my complaint is that the concepts and the behavior themselves feel wrong, no matter whether they are specced or written down. To use a harsh comparison: esoteric languages can be specified and documented, that doesn't stop them being esoteric. Of course the situation isn't as bad here but nevertheless, the reason I opened the issue was to initiate discourse around the concepts and semantics, rather than to trace the implementation.