tc39 / proposal-function-pipe-flow

A proposal to standardize helper functions for serial function application and function composition.

Function.pipe and flow for JavaScript

ECMAScript Stage-0 Proposal (withdrawn). J. S. Choi, 2021.

⚠️ This proposal is withdrawn. In the plenary on July 21, proposal-function-pipe-flow was formally presented to the Committee, and it was rejected for Stage 1. The Committee generally found its use cases either easily solved by userland functions, such as:

function pipe (input, ...fnArray) {
  return fnArray.reduce((value, fn) => fn(value), input);
}

…or also solved by the pipe operator. Its champion subsequently withdrew it from consideration. (Eventually, after the pipe operator gains users, pain points with the pipe operator may be enough motivation to revive this proposal, but that would not occur for a long time.)

• Proposal history

Original proposal

Serial function application and function composition are useful and common in JavaScript. Developers often divide code into many smaller unary callbacks, which are then sequentially called with some initial input—or which are composed into larger callbacks that will sequentially call those functions later.

To do this, they often combine these callbacks with helper functions: pipe (serial function application) and flow and/or compose (function composition). It would be useful to standardize these metafunctions.

The problem space here is the application and composition of serial callbacks. Much of the JavaScript community already has been serially applying or composing callbacks.

From StoplightIO Prism v4.5.0 packages/http/src/validator/validators/body.ts.

return pipe(
  specs,
  A.findFirst(spec => !!typeIs(mediaType, [spec.mediaType])),
  O.alt(() => A.head(specs)),
  O.map(content => ({ mediaType, content }))
);

From [email protected] packages/strapi-admin/services/permission/permissions-manager/query-builers.js:

const transform = flow(flattenDeep, cleanupUnwantedProperties);

From [email protected]/docs/static/utils/getInfoForSeeTags.js:

const getInfoForSeeTags = _.flow(
  _.get('docblock.tags'),
  _.filter((tag) => tag.title === 'see'),
  _.map((tag) => { /* … */ }),
)

If this proposal is approved for Stage 1, then we would explore various directions for serially applying and composing unary callbacks. Additionally, we would assemble as many real-world use cases as possible and shape our design to fulfill them.

(This problem space is already being explored to some extent by the pipe operator’s proposal. However, unlike in this proposal, the pipe operator can “apply” any kind of expression operation, not only unary function calls. The tradeoff is that the pipe operator involves new syntax; additionally, the pipe function is more concise for unary function calls and can handle dynamic arrays of callbacks. In Stage 1, we would continue to examine cross-cutting concerns and overlap with the pipe operator and other “dataflow” proposals.)

Solutions

We could add various combinations of the following static functions:

• Function.pipe and Function.pipeAsync (for serial application).
• Function.flow and Function.flowAsync (for LTR serial composition).
• Function.compose and Function.composeAsync (for RTL serial composition).

(LTR = left to right. RTL = right to left.)

There are eight possible combinations of `pipe`, `flow`, and `compose`.

• Choice #0: Status quo

• Choice #1: LTR flow

  Function.flow(f, g, h);
  Function.flowAsync(f, g, h);

• Choice #2: RTL compose

  Function.compose(h, g, f);
  Function.composeAsync(h, g, f);

• Choice #3: LTR flow & RTL compose

  Function.flow(f, g, h);
  Function.flowAsync(f, g, h);
  Function.compose(h, g, f);
  Function.composeAsync(h, g, f);

• Choice #4: Pipe

  Function.pipe(x, f, g, h);
  Function.pipeAsync(x, f, g, h);

• Choice #5: Pipe & LTR flow

  Function.pipe(x, f, g, h);
  Function.pipeAsync(x, f, g, h);
  Function.flow(f, g, h);
  Function.flowAsync(f, g, h);

• Choice #6: Pipe & RTL compose

  Function.pipe(x, f, g, h);
  Function.pipeAsync(x, f, g, h);
  Function.compose(h, g, f);
  Function.composeAsync(h, g, f);

• Choice #7: Pipe, LTR flow & RTL compose

  Function.pipe(x, f, g, h);
  Function.pipeAsync(x, f, g, h);
  Function.flow(f, g, h);
  Function.flowAsync(f, g, h);
  Function.compose(h, g, f);
  Function.composeAsync(h, g, f);

---

What happened to the F# pipe operator?

F#, Haskell, and other languages that are based on auto-curried unary functions have a tacit-unary-function-application operator. The pipe champion group has presented F# pipes for Stage 2 twice to TC39, being unsuccessful both times due to pushback from multiple other TC39 representatives’ memory performance concerns, syntax concerns about await, and concerns about encouraging ecosystem bifurcation/forking. (For more information, see the pipe proposal’s HISTORY.md.)

Given this reality, TC39 is much more likely to pass a Function.pipe helper function than a similar syntactic operator.

Standardizing a helper function does not preclude standardizing an equivalent operator later. For example, TC39 standardized binary ** even when Math.pow existed.

In the future, we might try to propose a F# pipe operator, but we would like to try proposing Function.pipe first, in an effort to bring its benefits to the wider JavaScript community as soon as possible.

Function.pipe

The Function.pipe static method applies a sequence of callbacks to a given input value, returning the final callback’s result.

Function.pipe(input, ...fns);

const { pipe } = Function;

// f2(f1(f0(5))).
pipe(5, f0, f1, f2);

// 5.
pipe(5);

// undefined.
pipe();

The following real-world examples originally used fp-ts’s pipe function.

// From @gripeless/[email protected]/source/inline.ts:
return pipe(
  download(absoluteURL),
  mapRej(downloadErrorToDetailedError),
  chainFluture(responseToBlob),
  chainFluture(blobToDataURL),
  mapFluture(dataURL => `url(${dataURL})`)
)

// From StoplightIO Prism v4.5.0 packages/http/src/validator/validators/body.ts:
return pipe(
  specs,
  A.findFirst(spec => !!typeIs(mediaType, [spec.mediaType])),
  O.alt(() => A.head(specs)),
  O.map(content => ({ mediaType, content }))
);

The first callback is applied to input, then the second callback is applied to the first callback’s result, and so forth. In other words, function piping occurs from left to right.

Each callback is expected to be a unary function.

If Function.pipe receives only one argument, then it will return input by default.
If Function.pipe receives no arguments, then it will return undefined.

Precedents include:

• fp-ts: import { pipe } from 'fp-ts/function';

Function.pipeAsync

The Function.pipeAsync static method applies a sequence of potentially async callbacks to a given input value, returning a promise. The promise will resolve to the final callback’s result.

Function.pipeAsync(input, ...fns);

const { pipeAsync } = Function;

// Promise.resolve(5).then(f0).then(f1).then(f2).
pipeAsync(5, f0, f1, f2);

// Promise.resolve(5).
pipeAsync(5);

// Promise.resolve(undefined).
pipeAsync();

The input is first awaited. Then the first callback is applied to input and then awaited, then the second callback is applied to the first callback’s result then awaited, and so forth. In other words, function piping occurs from left to right.

Each callback is expected to be a unary function.

If any callback returns a promise that then rejects with an error, then the promise returned by Function.pipeAsync will reject with the same error.

If Function.pipeAsync receives only one argument, then it will return Promise.resolve(input) by default.
If Function.pipeAsync receives no arguments, then it will return Promise.resolve(undefined).

(LTR) Function.flow

The Function.flow static method creates a new function by combining several callbacks in left-to-right order.

Function.flow(...fns);

const { flow } = Function;

const f = flow(f0, f1, f2);
f(5, 7); // f2(f1(f0(5, 7))).

const g = flow(g0);
g(5, 7); // g0(5, 7).

const h = flow();
h(5, 7); // 5.

The following real-world examples originally used lodash.flow.

// From [email protected]/packages/gatsby-plugin-sharp/src/plugin-options.js:
flow(
  mapUserLinkHeaders(pluginData),
  applySecurityHeaders(pluginOptions),
  applyCachingHeaders(pluginData, pluginOptions),
  mapUserLinkAllPageHeaders(pluginData, pluginOptions),
  applyLinkHeaders(pluginData, pluginOptions),
  applyTransfromHeaders(pluginOptions),
  saveHeaders(pluginData)
)

// From [email protected]
// packages/strapi-admin/services/permission/permissions-manager/query-builers.js:
const transform = flow(flattenDeep, cleanupUnwantedProperties);

// From [email protected]/docs/static/utils/getInfoForSeeTags.js:
const getInfoForSeeTags = flow(
  _.get('docblock.tags'),
  _.filter((tag) => tag.title === 'see'),
  _.map((tag) => { /* … */ }),
)

Any function created by Function.flow applies its own arguments to its leftmost callback. Then that result is applied to its next callback.

The leftmost callback may have any arity, but any subsequent callbacks are expected to be unary.

If Function.flow receives no arguments, then, by default, it will return a unary identity function.

Precedents include:

• lodash: lodash.flow is individually downloaded from NPM about 600,000 times weekly
• fp-ts: import { flow } from 'fp-ts/function';
• Ramda: import { pipe } from 'ramda/src/pipe';
• RxJS: import { pipe } from 'rxjs';

(LTR) Function.flowAsync

The Function.flowAsync static method creates a new function by combining several potentially async callbacks in left-to-right order; the created function will always return a promise.

Function.flowAsync(...fns);

const { flowAsync } = Function;

// async (...args) => await f2(await f1(await f0(...args))).
flowAsync(f0, f1, f2);

const f = flowAsync(f0, f1, f2);
await f(5, 7); // await f2(await f1(await f0(5, 7))).

const g = flowAsync(g0);
await g(5, 7); // await g0(5, 7).

const h = flowAsync();
await h(5, 7); // await 5.

Any function created by Function.flowAsync applies its own arguments to its leftmost callback. Then that result is awaited before being applied to its next callback.

The leftmost callback may have any arity, but any subsequent callbacks are expected to be unary.

If Function.flowAsync receives no arguments, then, by default, it will return Promise.resolve.

(RTL) Function.compose

The Function.compose static method creates a new function by combining several callbacks in right-to-left order.

Function.compose(...fns);

const { compose } = Function;

const f = compose(f2, f1, f0);
f(5, 7); // f2(f1(f0(5, 7))).

const g = compose(g0);
g(5, 7); // g0(5, 7).

const h = compose();
h(5, 7); // 5.

Any function created by Function.compose applies its own arguments to its rightmost callback. Then that result is applied to its next callback.

The rightmost callback may have any arity, but any subsequent callbacks are expected to be unary.

If Function.compose receives no arguments, then, by default, it will return a unary identity function.

(RTL) Function.composeAsync

The Function.composeAsync static method creates a new function by combining several potentially async callbacks in right-to-left order; the created function will always return a promise.

Function.composeAsync(...fns);

const { composeAsync } = Function;

// async (...args) => await f2(await f1(await f0(...args))).
composeAsync(f2, f1, f0);

const f = composeAsync(f2, f1, f0);
await f(5, 7); // await f2(await f1(await f0(5, 7))).

const g = composeAsync(g0);
await g(5, 7); // await g0(5, 7).

const h = composeAsync();
await h(5, 7); // await 5.

Any function created by Function.composeAsync applies its own arguments to its rightmost callback. Then that result is awaited before being applied to its next callback. In other words, async function composition occurs from left to right.

The rightmost callback may have any arity, but any subsequent callbacks are expected to be unary.

If Function.composeAsync receives no arguments, then, by default, it will return Promise.resolve.

proposal-function-pipe-flow's Issues

Should compose/flow return a variadic function (and what should its length be)?

const countArgs = (...args) => args.length;
const test = compose(countArgs);
console.log(test.length); // 1? Or same as countArgs.length?
console.log(test()); // 1 or 0?
console.log(test(null)); // 1, surely
console.log(test(null, null)); // 1 or 2?

The returned function might be variadic (having the same arity as the innermost function, and forwarding as many arguments as it receives), or it might be unary (having an arity of 1, and forwarding only the first argument, or undefined if none is passed).

Both have their pros and cons.

• The variadic solution would permit composing functions with multiple arguments without jumping through unnecessary hoops:

  const add = (a, b) => a + b;
  const addThenSqrt = Function.compose(Math.sqrt, add); // (x, y) => Math.sqrt(add(x, y))
  console.log(addThenSqrt(140, 4)); // 12

  It would also fit in with the variadic behaviour of the bind method. On the other hand, such usage might be confusing to anyone coming from functional programming, all prior art I know works on unary functions exclusively. It would also be a bit harder to declare a correct type for it.

• A unary solution would follow prior art and have less surprises.

  arrOfStrings.map(Function.compose(Math.sqrt, parseInt))

  would just work like .map(x => Math.sqrt(parseInt(x))) and not have the .map(parseInt) problem. It could also be abused to create unary functions like in arr.map(Function.compose(parseInt)) (not sure if that's a good or bad thing?).

Function composition: LTR flow vs. RTL compose

What about the standard function composition? I.e. flow with order of arguments reserved.
Why is it not part of this proposal? It would fit right in.

Function-name bikeshedding

The current proposal uses flow and pipe as names for very similar concepts however it's not clear how "flow" distinguishes the behaviour from pipe.

Given .pipe(initial, ...fns) is essentially just .flow(...fns)(initial) I feel like a terminology like .pipeline(...fns) would make more sense. i.e. .pipeline creates a pipeline which can be passed a value, .pipe would be treatable as a shorthand for passing a value into a pipeline.

How much does this help with point-free programming?

Soon (hopefully) we'll be receiving the pipeline operator, which is great! They're choosing to go the hack-style route, which is great for interoping with existing APIs, but it leaves a nasty taste in your mouth if you're a lover of point-free/tacit style programming.

The primary purpose of this proposal seems to be addressing the need of the tacit-programming side of our community - they were promised a pipeline operator, it was assumed to be tacit-style-friendly, and now they're not getting one, which is frustrating I'm sure. A new API, like Function.pipe() could be used to address the needs of this side of the community.

But, I'm left wondering how helpful this is really going to be. The only benefit of using Function.pipe() over the standard pipe operator is for tacit-programming, which depends heavily on the presence of curried functions. The standard library is not curried, and generally, our community-provided libraries are not curried either (unless they're somehow related to functional programming). So even if we provide a standard, tacit-programming-friendly pipe function, is it really going to help much? tacit-style-programmers are still going to have to hand-curry everything the standard and third-party libraries offer, or they're going to have to install a functional library that basically provides its own, substitute standard library (like Rambda). In either scenario, it's not that hard to hand-add yet another utility function (a one-line definition for pipe()), or to use a pipe function provided by Rambda.

Perhaps the answer is one of these?

• Maybe the value of a built-in tacit-friendly pipe function is much more important than I'm realizing? If so, would someone be able to explain why a Function.pipe() is important to have, despite the non-curried nature of the standard library you have to deal with. To me, adding a Function.pipe() just seems like a small drop of improvement in a sea of missing tacit-friendly functionality.
• Or, maybe this is a sign of things to come, and the language designers are wanting to better support tacit programming in JavaScript? I haven't heard of anything along these lines, nor do I know how reasonably possible this is.
• Or, perhaps there's other uses for Function.pipe() besides helping our tacit-programming friends? If so, what? Are these other use-cases important enough to give enough value to Function.pipe() to introduce it into the language?
• Or is there something else I'm missing?

Forgot to define Function.identity?

If Function.flow receives no arguments, then, by default, it will return Function.identity (which is defined later in this proposal).

But the word identity isn't found anywhere else in the README. I can't take this kind of suspense! ;-)

flow/compose when given no input functions

Let’s assume that the Committee approves Function.flow and/or Function.compose (#5).

flow() by default returns an identity function, so flow()(1) === 1.

But should flow() === flow()`?
Should there be one identity function instance or should it be constructed each time? The specification must decide.

Same question for compose, if it gets approved. And in the event that the Committee approves both, then we must also decide whether flow() === compose().

I’m currently inclined to make the answer yes (return the same identity-function instance every time).

Stage 1 slides feedback

@js-choi since you suggested over on the pipeline operator proposal to leave feedback on your slides on this repo, but I don't see an issue for it, I thought I'd create one. Please let me know if you think this feedback belongs on a specific other issue, though.

One minor thought I had, specifically about the slides you linked: something that might make sense to add to the last page dealing with whether it competes with or complements the pipeline operator proposal. This might be more of a social benefit than an objective language benefit -i t could conceivably be a decent consolation to the decent number of people still wishing that F# syntax could be revived from the dead. This proposal plus the hack token is definitely no F# pipe operator, but the two combined do have some similar properties:

'foo' |> Function.pipe(^, someFunction, someCurriedFunction(somethingElse))

Which arguably isn't so different from what the Fsharpies want:

'foo' |> someFunction |> someCurriedFunction(somethingElse)

in that they can pass as many unary functions as they like, and they only have to type the dreaded ^ once. They have to also type Function.pipe, and wrap their list of unary functions in parentheses, which they wouldn't with F#, but crucially, they don't have to wrap the original expression ('foo' above) in parentheses. They also don't need to pay an ugly syntax tax on every unary function, which opens the door slightly more to purist functional data-last libraries like fp-ts.

It certainly won't totally satisfy everybody (or even anybody?), but as someone who badly wishes the committee had decided on F# rather than Hack, (but has accepted that they... didn't) - this makes me feel a little better. And maybe I'm not alone?