sebmarkbage / ecmascript-generator-expression

Proposal for do Generator Expressions in ECMAScript. Work in progress. Edit Add topics

Generator Expressions for ECMAScript

A convenient way of generating arrays/sets/maps of values in a contained expression. It's an alternative to array comprehensions and designed to pair well with do expressions. I think of it as the multi-value form of do-expressions.

Motivation

A strong argument allowing traditional for comprehensions in the language is that when they become complex, they're often better structured as combinator methods with arrow functions. Therefore it's better to just start with combinators from the start to make that transition easier.

However, we've observed that when combinator chains become complex, rather than staying in the combinator form, people tend to break out of the combinator form and switch to imperative loops.

The idea around this proposal is to allow a more natural transition by making a convenient shorthand imperative style that can be expanded to more complex cases.

Generator Expressions

The primitive is a new kind of expression that allows yielding. It evaluates to a generator.

let gen = *{
  yield 1;
  yield 2;
};

This desugars to an immediately invoked generator function:

let gen = (function*() {
  yield 1;
  yield 2;
})();

This can be combined with various initializers to create an Arrays, Maps or Sets.

let arrayOfUsers = [...*{
  for (let user of users)
    if (user.name.startsWith('A'))
      yield user;
}];

let arrayOfUsers = Array.from(*{
  for (let user of users)
    if (user.name.startsWith('A'))
      yield user;
});

let setOfUsers = new Set(*{
  for (let user of users)
    if (user.name.startsWith('A'))
      yield user;
});

let mapOfUsers = new Map(*{
  let i = 0;
  for (let user of users)
    if (user.name.startsWith('A') && (i++ % 2) === 0)
      yield [user.id, user];
});

Code Evolution

Since these generator expressions naturally expand to more complex examples, you can keep expanding these with more complex logic as requirements expand. While still remaining in an isolated expression.

let allUsers = new Set(*{
  let i = 0;
  for (let user of activeUsers) {
    i++;
    let isEven = (i % 2 === 0);
    let id = user.id;
    let name = user.name;
    if (id && name !== 'DELETED') {
      yield { id, name, isEven };
    }
  }
  for (let user of inactiveUsers) {
    i++;
    let isEven = (i % 2 === 0);
    yield { id: null, name: user.name, isEven };
  }
});

If the code complexity keeps expanding to requiring a temporary array, the next refactor step is to switch to a do expression.

let allUsers = new Set(do {
  let tmp = [];
  let i = 0;
  for (let user of activeUsers) {
    i++;
    let isEven = (i % 2 === 0);
    let id = user.id;
    let name = user.name;
    if (id && name !== 'DELETED') {
      tmp.push({ id, name, isEven });
    }
  }
  for (let user of inactiveUsers) {
    i++;
    let isEven = (i % 2 === 0);
    tmp.push({ id: null, name: user.name, isEven });
  }
  tmp.reverse();
});

Completion Values

Just like do-expressions, the completion value is like a return value. That means that the completion value is the final value in the generator. This is an advanced and uncommon feature. In most common uses of this feature, the completion value is ignored. E.g. when used to construct Arrays or Sets.

This means that this yields a generator whose final value is z.

let xyz = *{
  for (let x in a)
    yield x;
  for (let y in b)
    yield y;
  z;
};

It desugars to:

let xyz = (function* {
  for (let x in a)
    yield x;
  for (let y in b)
    yield y;
  return z;
})();

This can be useful for scheduling code like Task.js.

let promise = Task(*{
  const [foo, bar] = yield Task.join(
    read("foo.json"),
    read("bar.json")
  );
  foo.x + bar.y;
});

break Statements

break without a label will break out of the generator. It behaves like returning the completion value of a generator function. Early returns can be accomplished using break.

let promise = Task(*{
  const [foo, bar] = yield Task.join(
    read("foo.json"),
    read("bar.json")
  );
  if (!bar) {
    foo.x;
    break;
  }
  foo.x + bar.y;
});

You can break to labels inside of the generator expression just like normal. However, with a label defined outside the generator expression, the control flow doesn't make as much sense since in a free standing generator, that scope doesn't exist anymore. That's a syntax error.

foo: {
  let items = [...*{
    break foo; // SyntaxError!
  }];
}

return Statements

It's unclear what the return statement should do inside these generators. It could either return out of the outer function or abruptly stop the iteration of the generator. I'm leaning to just forbidding return for now. Early returns are awkward but works, which seems fine because the use case for the completion value is rare anyway.

throw works just fine though.

do * { ... } Syntax Alternative

An alternative syntax could use the do prefix since it is effectively in the same category as do expressions and this might help explain it once you are familiar with do-expressions. I tend to prefer the shortest possible syntax when possible though.

Related Proposals

do expressions

Array comprehensions

Prior Art

F# Sequence Expressions

Scala Sequence Comprehensions

Status of this Proposal

This has not yet been presented to TC39. I'm still gathering feedback.

ecmascript-generator-expression's Issues

Babel Transform

I made a ECMAScript generator transform Babel plugin:
https://github.com/clemmy/babel-plugin-transform-generator-expression.

Since it requires a forked parser to dabble with, I made a CodePen where you can easily try out the syntax:
https://codepen.io/clemmy/pen/zpGaNv?editors=1000

Perhaps this can be incorporated as a part of the proposal draft?

About implicit do or do* expressions

I was also posting this in the do proposal:
tc39/proposal-do-expressions#9

To me, it sounds most logical that if any of the if, switch, for, while, try keywords are used in an expression context, then it will be an implicit do expression construction:

So the following

const num  = if (cond) {
  100; 
} else {
  10;
};

would be equivalent to:

const num  = do {
  if (cond) {
    100; 
  } else {
    10;
  }
}

Similarly, the following

const iterable = for (let user of users) {
  if (user.name.startsWith('A'))
    yield user;
}

would be equivalent to

const iterable = do* {
  for (let user of users) {
    if (user.name.startsWith('A'))
      yield user;
  }
};