Consider `protected`, `abstract`, and `friend` about proposal-decorators

[littledan]

Initially, in this proposal, let's not build in protected, abstract, etc, and leave these to be implemented via decorators and transpilers. Then, with that experience, we can consider adding them in a follow-on proposal.

[bakkot]

Here's an earlier comment of mine on the topic of additional modifiers which I think is pretty relevant to this thread.

[rbuckton]

@bakkot, from that comment:

If there is any way at all we can avoid getting into that mess, I would like to do so. It seems far better to me to provide the minimum necessary and then allow user code to define other modifiers to fit their needs, including modifiers which have yet to be invented, and maybe someday provide more after we've gotten some experience in how this bare minimum plays out in JavaScript. I'll stand behind that as good language design any day.

The decorators in the example in friend.js are an attempt to do this in user code, but have a number of issues with leaking private state (primarily the "friend" example), being fairly non-ergonomic to use (a lot of repetitive boilerplate in the "friend" example), or cannot provide certain capabilities (such as overriding an @Protected method and calling super to call the superclass version).

Even if we never implement these modifiers, it makes sense to discuss their implications. Understanding how "protected" or "friend" access should work, with or without syntax, helps to direct the actual implementation of private-state in a way that allows user code to emulate the behavior using something like a decorator.

The upshot of syntax is that it gives the engine the opportunity to establish and lock down member access in ways that may not be feasible in user code.

[rbuckton]

Here's an example of code using "friend" via decorators: https://gist.github.com/rbuckton/0aebbdc9c723cab0b39c0a0c6684ebb5

Here's the same example using the syntax strawman above: https://gist.github.com/rbuckton/a7736d4c061446df6590c4b5dfb2dccd

[bakkot]

The decorators in the example in friend.js are an attempt to do this in user code, but have a number of issues with leaking private state (primarily the "friend" example),

I'm not sure what you mean w.r.t. the friend example leaking private state. As to ergonomics and capabilities, I expect the examples could be improved on both fronts - just as an example, the friend decorator could trivially be implemented such that you could do key(myClassInstance)._x = 2 in place of key.set(myClassInstance, '#x', 2). They shouldn't be taken as the best possible implementations, just proofs of concept.

Even if we never implement these modifiers, it makes sense to discuss their implications.

Agreed. I just prefer to frame the discussion as "what sort of accessibility modification do we want to do and how can we ensure decorators are flexible enough to provide that", for the moment.

On that note, a comment on friend - one nice thing about the example in friend.js is that it allows exposing only select private properties. It would be easy enough to rewrite it to expose all of them, if that's a use case, but I like the granularity provided by the current design.

[littledan]

There's a lot of details to work out here, but I want to talk first about the high-level design. I completely agree that we should talk this through.

For protected, I don't quite understand all the problems. Was this only about protected super calls (which I think will be implementable somehow), or are there other issues?

Is there a way for two classes to be mutually friends of each other with your proposal? The FriendKey functionality in friend.js would permit that.

[azz]

Do people actually think the general concept offriend is a good idea? To me the idea of having a unit X with private internal state accessible only to itself and thing F has always been a code smell. Why not either put that state in a Map or somewhere where both can access it? Or just make it public and don't expose X to consumers.

I feel like the reason no languages have nailed these accessibility modifiers is that they are usually workarounds to problems inherent in OOP, not first class language design concepts. JavaScript has excellent, intuitive modules and closures that alleviate the need for these modifiers.

As for abstract, I think protocols can fill this void while providing a great primitive that can be used in FP, too.

[ljharb]

I do not think the concept of friend is a good idea; particularly because there's no way besides closures to safely grant access to a desired friend, without also granting access to anything else.

Similarly, I agree that protocols seem like they obviate any use cases for abstract classes - is there a use case I'm missing?

[dead-claudia]

Just a thought: friends should be designed in a way that they can be used both ways between two classes. For example, in a subject observable, the subject needs to access the private subscriber to invoke its method, but that shouldn't be exposed publicly. Similarly, the private subscriber needs to access the subject's private listener list to remove itself on unsubscribe.

Also, JSDOM probably has a blue million different cases where it needs to access private data from another object, and IIRC they currently use weak maps extensively for that reason.

[dead-claudia]

@ljharb

I do not think the concept of friend is a good idea; particularly because there's no way besides closures to safely grant access to a desired friend, without also granting access to anything else.

Or at least in the sense that's being proposed here. It appears the design of friend.js (define a scope outside the class) would suffice to fix that, so if you instead did something like this, it might work better (as a bonus, this could be exportable as a binding):

Edit: The error is early (parse or module instantiation time) when defined incorrectly, and friends may only have their methods defined in the same function-level scope as they're defined or imported. Also, exported friend bindings don't show up in dynamic imports, and it's a load error if an exported friend isn't defined as a member by the dynamically loaded module or its dependents.

// It's an early error at parse or module instantiation time if `#x` or `#y` are
// defined multiple times, or if either variable is defined with the incorrect
// member type.
friend scope { #x, protected #y }

class B {
  friend {#x, #y} from scope;

  getX(obj) {
    return obj.#x;
  }
  setX(obj, value) {
    obj.#x = value;
  }
  callY(obj) {
    return obj.#y();
  }
}

class A {
  friend {#x, #y} from scope;
  #x;
  protected #y() { return this.#x + 1; }
  constructor(x) { this.#x = x; }
}

Similarly, I agree that protocols seem like they obviate any use cases for abstract classes - is there a use case I'm missing?

I think it's just that abstract classes are easier to subclass, but also, you can't define instance state for protocols, and even if you could, you can't initialize them with arguments without putting them near feature-parity with classes (which would make them no longer protocols, but multiple-inheritance classes instead). And if you were to go the latter route, you might as well just support multiple inheritance with normal classes - it'd be easier and cleaner.

[dead-claudia]

As for clarification, I chose to include protected in the friend descriptor to make it easier for the engine to write the correct bytecode (it doesn't have to wait for the eventual class definition to make the correct call type). That part could change, though, and I'm not terribly attached to it. (If it happens to be feasible to omit it, I'd be happy to do it.)

[littledan]

@isiahmeadows Interesting idea. We could also accomplish something similar with "rebind decorators", see #17 . However, I'm not pursuing that right now, because it seems like enough is possible with the current proposal, e.g., see friends.js.

[dead-claudia]

That'd work, too. Mine would allow for better engine optimization, but if all you're looking for is pure structure, that'd be sufficient (and slightly more concise).

[littledan]

@isiahmeadows I don't understand how yours would lead to better engine optimization.

[rbuckton]

Just a thought: friends should be designed in a way that they can be used both ways between two classes.

One option is to define an intermediary:

// friends.js
class AB {
  static getX(a) { return a.A#x; }
  static getY(b) { return b.B#y; }
}

export class A friend AB {
  #x = 1;
  getY(b) { return AB.getY(b); }
}
export class B friend AB {
  #y = 2;
  getX(a) { return AB.getX(a); }
}

// main.js
import { A, B } from "./friends";
const a = new A();
const b = new B();
a.getY(b); // 2
b.getX(a); // 1

Another option for friend would be to hold onto the binding of the friend (rather than the value) and only evaluate the binding when checking access:

// friends.js
export class A friend B {
  #x = 1;
  getY(b) { return b.B#y; }
}
export class B friend A {
  #y = 2;
  getX(a) { return a.A#y; }
}

// main.js
import { A, B } from "./friends";
const a = new A();
const b = new B();
a.getY(b); // 2
b.getX(a); // 1

[ljharb]

I agree that if friend only operates in a single scope, that it works fine.

However, what’s the utility if it means you can’t split your classes up into separate modules? Can you help me understand the use case where you need more than one class to share private data, and these classes are small enough that cramming them into the same file is practical and maintainable?

[rbuckton]

With the proposed syntax you can also split across files:

// a.js
import { B } from "./b";
class A friend B {
  #x;
}

// b.js
import { A } from "./a";
class B {
  getX(a) { return a.A#x; }
}

However, this introduces circularity. While circular dependencies are permitted with ES modules, they are a possible source of subtle errors around evaluation order.

[ljharb]

Ah, right. While in my projects i intend to lint against circularity, that would indeed allow friends to be useful across modules. Thanks for clarifying!

[rbuckton]

I wonder if it would make sense to propose a mechanism to provide a hint to the loader about dependency order, something like:

// a.js
import defer { B } from "./b"; 
// creates the binding for `B`, but prefer b.js to load later.
// possibly an early error/TDZ to access `B` before a.js module body has finished evaluation.

// To avoid TDZ, `friend` would need to capture the binding for `B` but not read the value
// until access checks are made.
class A friend B {
  #x;
}

// b.js
import { A } from "./a";
class B {
  getX(a) { return a.A#x; }
}

With import defer, "b.js" would be given a lower priority than "a.js", allowing "a.js" to evaluate first. If both "a.js" and "b.js" were to use import defer, they both have the same priority and imports resolve as per usual, but with the added effect of the early error/TDZ for any use of an import before the module body finishes evaluation.

[littledan]

@concavelenz was suggesting something similar to this "import defer" thing.

[littledan]

@rbuckton I'm still having trouble understanding the advantages of these modifiers being built-in as opposed to using decorators.

Is this a categorical thing, that for something as core as access modifiers, it's inappropriate to put it in a library? Or that the syntax is nicer without an @, and friend element access is more ergonomic as x.A#b than key.get(x, A, "b")?

One thing that makes me a bit worried about adding access modifiers at this point is that they are complicated, detailed, and the proposal differs from what we have now (e.g., these are somewhat tighter than what TypeScript provides). A lot of the complexity is in protected, but many programmers have been giving the feedback that friend is more important.

The upshot of syntax is that it gives the engine the opportunity to establish and lock down member access in ways that may not be feasible in user code.

You mentioned stronger guarantees, but I don't quite understand. What wouldn't be feasible in user code? Is this about the distribution of the FriendKey?

The protected modifier allows a subclass to access all of the private names of a superclass that were declared as part of a protected member.

Protected increases the dynamism of private names significantly. In particular, the lookup of an identifier #x becomes no longer lexical but instead there are other sources to consult (super-classes, friends) in a certain order.

One option is to define an intermediary:

For friends, a very common case is that class A is allowed to see the private elements of class B, and A is defined after B. For example, this comes up when class A is test code accessing the internals of class B. It would be pretty awkward to define an intermediate class AB as you're suggesting above, as opposed to using a @testable decorator which just exposes everything for metaprogramming.

Another option for friend would be to hold onto the binding of the friend (rather than the value) and only evaluate the binding when checking access:

Seems like this only works if they're in the same scope. For example, if I have a class, and then tests in another file, I'm not sure how to use this mechanism.

[dead-claudia]

Inline. Although it wasn't directed at me, I thought I'd drop in a little input.

On Sat, Nov 11, 2017, 06:44 Daniel Ehrenberg ***@***.***> wrote: @rbuckton <https://github.com/rbuckton> I'm still having trouble understanding the advantages of these modifiers being built-in as opposed to using decorators. Is this a categorical thing, that for something as core as access modifiers, it's inappropriate to put it in a library? Or that the syntax is nicer without an @, and friend element access is more ergonomic as x.A#b than key.get(x, A, "b")?

Having them as native features lends an obvious optimization opportunity. For one, it becomes a simple hidden property access rather than a weak map load or whatever. As for the syntax, I do find `class A friend B` to be a bit too limiting.

One thing that makes me a bit worried about adding access modifiers at this point is that they are complicated, detailed, and the proposal differs from what we have now (e.g., these are somewhat tighter than what TypeScript provides). A lot of the complexity is in protected, but many programmers have been giving the feedback that friend is more important.

I'll join that crowd. Also, you can provide most of the `protected` functionality with `friend`.

The upshot of syntax is that it gives the engine the opportunity to establish and lock down member access in ways that may not be feasible in user code. You mentioned stronger guarantees, but I don't quite understand. What wouldn't be feasible in user code? Is this about the distribution of the FriendKey? The protected modifier allows a subclass to access all of the private names of a superclass that were declared as part of a protected member. Protected increases the dynamism of private names significantly. In particular, the lookup of an identifier #x becomes no longer lexical but instead there are other sources to consult (super-classes, friends) in a certain order.

Yes, but this should still be statically discernable in practically every case.

One option is to define an intermediary: For friends, a very common case is that class A is allowed to see the private elements of class B, and A is defined after B. For example, this comes up when class A is test code accessing the internals of class B. It would be pretty awkward to define an intermediate class AB as you're suggesting above, as opposed to using a @testable decorator which just exposes everything for metaprogramming.

This is why I created my `friend` proposal the way I did - it was for this very reason.

…

Another option for friend would be to hold onto the binding of the friend (rather than the value) and only evaluate the binding when checking access: Seems like this only works if they're in the same scope. For example, if I have a class, and then tests in another file, I'm not sure how to use this mechanism. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#25 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AERrBKdSj3bUQ-F5tw8OFGR9nL_-YCbcks5s1Yi2gaJpZM4P_-TT> .

[littledan]

Yes, but this should still be statically discernable in practically every case.

I'm not really sure how to do that for the protected proposal above; maybe you can explain more. One detail here is that the superclass is mutable.

This is why I created my friend proposal the way I did - it was for this very reason.

Where can I find your friend proposal?

[dead-claudia]

@littledan As long as you make sure private is hard-private and clearly non-overridable, you have to make protected members expose themselves differently, and engines can use that information to statically determine the difference between protected and private. In particular, C++ operates this way for the most part. For example:

class Foo {
    protected #bar() {}
}

class Bar extends Foo {
    method() {
        // Engines can detect this case statically to not be private, because
        // it's not defined in this scope.
        #bar();
    }
}

class Baz extends Foo {
    // Engines can detect this case statically to not be private, because it's
    // marked with `protected`.
    protected #bar() {}
}

class Qux extends Foo {
    protected { #bar }
    method() {
        // Engines can detect this case statically to not be private, because
        // it's marked as `protected`.
        #bar();
    }
}

If you don't require protected, then you're now back to soft-private, but with all the inheritance concerns as well. (That, I would expect to be a no-go.)

Where can I find your friend proposal?

I was referring to this comment of mine. I was replying via email, or I would've linked it from there.

[bmeck]

@isiahmeadows

I think it's just that abstract classes are easier to subclass, but also, you can't define instance state for protocols

@rbuckton

I have various concerns about the actual need for protected in the presence of private and an exposing constructor pattern for now. I would like to defer the idea of adding it until we get more experience in user land to see if private and exposing constructors are sufficient. It would keep the line clear about what can access properties. In particular I am concerned about:

class A {protected #foo}
class B extends A {}
const b = new B;
class C extends b.constructor {
   get foo() {return #foo;}
}
Object.getOwnPropertyDescriptor(C.prototype, foo).get.call(b);

Being a method to circumvent protected to an extent that it is effectively public.

I fear that friendly faces the same problem of getting a hold of a privileged class and extending it.

[dead-claudia]

@bmeck

I fear that friendly faces the same problem of getting a hold of a privileged class and extending it.

If friends are tied to scopes rather than classes, that problem mostly disappears. But I do agree that protected still carries a risk.

[littledan]

@bmeck are you concerned about friends as done through decorators for that privacy risk?

[pzuraq]

This seems to be less relevant to this proposal or possible extensions, and should instead be its own proposal related moreso to fields/private methods. I'm going to close this, but feel free to reopen if you disagree.