sindresorhus / promise-fun

Promise packages, patterns, chat, and tutorials

promise-fun

I intend to use this space to document my promise modules, useful promise patterns, and how to solve common problems. For now though, you can see all my promise modules below.

Contents

• Packages
• FAQ

Packages

Not accepting additions, but happy to take requests.

• pify: Promisify a callback-style function
• delay: Delay a promise a specified amount of time
• yoctodelay: Delay a promise a specified amount of time
• p-map: Map over promises concurrently
• p-all: Run promise-returning & async functions concurrently with optional limited concurrency
• p-queue: Promise queue with concurrency control
• p-event: Promisify an event by waiting for it to be emitted
• p-debounce: Debounce promise-returning & async functions
• p-throttle: Throttle promise-returning & async functions
• p-timeout: Timeout a promise after a specified amount of time
• p-retry: Retry a promise-returning or async function
• p-any: Wait for any promise to be fulfilled
• p-some: Wait for a specified number of promises to be fulfilled
• p-locate: Get the first fulfilled promise that satisfies the provided testing function
• p-limit: Run multiple promise-returning & async functions with limited concurrency
• p-series: Run promise-returning & async functions in series
• p-memoize: Memoize promise-returning & async functions
• p-pipe: Compose promise-returning & async functions into a reusable pipeline
• p-props: Like Promise.all() but for Map and Object
• p-waterfall: Run promise-returning & async functions in series, each passing its result to the next
• p-cancelable: Create a promise that can be canceled
• p-progress: Create a promise that reports progress
• p-reflect: Make a promise always fulfill with its actual fulfillment value or rejection reason
• p-filter: Filter promises concurrently
• p-reduce: Reduce a list of values using promises into a promise for a value
• p-settle: Settle promises concurrently and get their fulfillment value or rejection reason with optional limited concurrency
• p-map-series: Map over promises serially
• p-each-series: Iterate over promises serially
• p-times: Run promise-returning & async functions a specific number of times concurrently
• p-lazy: Create a lazy promise that defers execution until .then() or .catch() is called
• p-whilst: While a condition returns true, calls a function repeatedly, and then resolves the promise
• p-do-whilst: Calls a function repeatedly while a condition returns true and then resolves the promise
• p-forever: Run promise-returning & async functions repeatedly until you end it
• p-wait-for: Wait for a condition to be true
• p-min-delay: Delay a promise a minimum amount of time
• p-try: Promise.try() ponyfill - Starts a promise chain
• p-race: A better Promise.race()
• p-immediate: Returns a promise resolved in the next event loop - think setImmediate()
• p-time: Measure the time a promise takes to resolve
• p-defer: Create a deferred promise
• p-is-promise: Check if something is a promise
• p-state: Inspect the state of a promise
• make-synchronous: Make an asynchronous function synchronous

.then/.catch-based packages

You should generally avoid using .then except in edge cases.

• p-catch-if: Conditional promise catch handler
• p-if: Conditional promise chains
• p-tap: Tap into a promise chain without affecting its value or state
• p-log: Log the value/error of a promise
• p-break: Break out of a promise chain

FAQ

How can I run 100 async/promise-returning functions with only 5 running at once?

This is a good use-case for p-map. You might ask why you can't just specify an array of promises. Promises represent values of a computation and not the computation itself - they are eager. So by the time p-map starts reading the array, all the actions creating those promises have already started running. p-map works by executing a promise-returning function in a mapper function. This way the promises are created lazily and can be concurrency limited. Check out p-all instead if you're using different functions to get each promise.

import pMap from 'p-map';

const urls = [
	'https://sindresorhus.com',
	'https://avajs.dev',
	'https://github.com',
	…
];

console.log(urls.length);
//=> 100

const mapper = url => fetchStats(url); //=> Promise

const result = await pMap(urls, mapper, {concurrency: 5});

console.log(result);
//=> [{url: 'https://sindresorhus.com', stats: {…}}, …]

promise-fun's Issues

Debounce so that there's only one pending promise

async function action() {
	delay(3000);
	alert('Done')!
}

button.onclick = action;

If the user double-clicks the button, action is run twice. This usually debounced/throttled with timeouts, but how would you throttle this so that action calls don't pile up instead?

This is different from p-debounce since that's timeout-based and completely ignores the state of the promise.

How to access the state of a promise synchronously?

Officially there's no way to check whether a promise has resolved without awaiting it. Is there a module for this already?

Like a synchronous reflect: https://github.com/sindresorhus/p-reflect

const promise = new OpenPromise(resolve => setTimeout(resolve, Math.random() * 100000));

await delay(1000);
if (promise.isFulfilled) {
	console.log('Hurray!')
}

p-* version for promise-queue(max-concurrent=1, max-lenght=1)

It's useful for the situation like "queue some processing like building up to 1, and do not run at the same time".

naming candidates

I didn't came up with good name.

• p-serial

implementation with promise-queue

import Queue from "promise-queue";

const pBlah1 = (promiseGetter) => {
  const maxConcurrent = 1;
  const maxQueue = 1;
  const queue = new Queue(maxConcurrent, maxQueue);
  return () => queue.add(promiseGetter).catch((e) => {if (e.message !== 'Queue limit reached') throw e;})
}

implementation from scratch

NOTE: this is fully written by me and publishing under public domain.

const pBlah2 = (promiseGetter) => {
  let waitProm = null;
  let waitResolve = null;
  let waitReject = null;
  let running = false;
  const done = () => {
    const waitResolveOld = waitResolve;
    const waitRejectOld = waitReject;
    running = false;
    waitProm = null;
    waitResolve = null;
    waitReject = null;
    if (waitResolveOld) {
      promiseGetter()
        .then((v) => {
          done();
          waitResolveOld(v);
        })
        .catch((e) => {
          done();
          waitRejectOld(e);
        });
    }
  };
  return () => {
    if (running) {
      if (!waitProm) {
        waitProm = new Promise((resolve, reject) => {
          waitResolve = resolve;
          waitReject = reject;
        });
      }
      return waitProm;
    }
    running = true;
    return promiseGetter()
      .then((v) => {
        done();
        return v;
      })
      .catch((e) => {
        done();
        throw e;
      });
  };
};

usage example

const wait = (ms) =>
  new Promise((resolve) => {
    setTimeout(resolve, ms);
  });

const a = async () => {
  await wait(2000);
  console.log("hi");
};

const f = pBlah1(a);
// const f = pBlah2(a);

(async () => {
  f(); // start immediately
  await wait(500);
  f(); // queued because running
  await wait(500);
  f(); // ignored because running and queue is full
})();

hi is only shown 2 times.

vs others

• vs p-limit: this will run all queued promises.
• vs p-debounce: this will run concurrently
• vs p-queue: no option to limit max length
• vs promise-queue: easy for the specific case

Run async function return values

An array of async functions that are called at the same time with each returning async functions that are run in order. Useful for executing a list of tasks where you have to do some preparation before each one that can be done without doing the actual task. For example:

await pAsyncLoadedTasks([
	async () => {
		const converted = await convertPart(foo)
		return async () => play(converted)
	},
	async () => {
		const converted = await convertPart(bar)
		return async () => play(converted)
	}
])

In this case, the first async function is executed and the function it returns is executed whilst the second async function is executed and then its returned function is executed after.

Maybe like this?

import pMap from "p-map"
import PQueue from "p-queue"

const pAsyncLoadedTasks = async (jobs, options) => {
	const returnValues = []

	const queue = new PQueue({ concurrency: 1 })

	await pMap(jobs, async createJob => {
		const job = await createJob()

		queue.add(async () => {
			returnValues.add(await job)
		})
	}, options)

	await queue.onEmpty()

	return returnValues
}

Move .then/.catch-based modules to their own section

Modules like p-log and p-tap don't fit the "avoid .then" suggestion, so they aren't as useful anymore.

What do you think about moving them to their own "old-style modules" section or something like that? With dozens of packages these IMHO are just clutter among packages are still useful.

They were even just updated for Node 12 support but their examples still have then/catch: https://github.com/sindresorhus/p-log

p-map for a function or p-whilst with concurrency

Hi,

Is there any package here that works like p-map but with a function instead of an array? Or like p-whislt with concurency.

Something that takes a function returning a promise or null. As long as the function returns a promise then it keeps going with concurency managed.

Like this package https://www.npmjs.com/package/es6-promise-pool

Thanks

Add caolan/async

I'm kinda surprised this isn't on here yet: https://caolan.github.io/async/v3/

It's basically the async equivalent of lodash.

p-prefetch or p-iterable?

I have created a module that seems to fit well here, as it was initially based off of p-map. The module is itself an async/sync iterable and it takes the same inputs as pMap() with an additional maxUnread option.

If you configure it with concurrency 4 and maxUnread 8, it will spin up 4 runners, push their mapper result or wrapped exception into a queue, then start another mapper. When the number of mappers running + queue depth == maxUnread then the runners will exit.

When the caller then iterates an item out of the unread result queue it will start another runner, and so on.

This essentially allows the consumer to "prefetch" results of promises with back pressure. An example would be when you're reading 10,000 files from, say, S3, but you don't want to fill up your disk or spend all your time fetching files that you will never get to: the prefetcher can be configured to always have, say, no more than 20 files ready and can use up to 20 mappers to fetch more when needed.

There is some overlap with p-queue in that the starting/stopping of runners is similar to what p-queue is doing to adjust concurrency when the queue drains / hits empty / refills. I think it could be possible to implement this iterable with back pressure using p-queue under the hood with the event function passed to p-queue being a function that writes into the unread queue and awaits a Promise that indicates that an item has been read from the queue when the queue is full (thus preventing further items from being iterated if maxUnread has been hit).

Is there interest in including another module here with these capabilities? The module is currently written in TypeScript (500 lines) and has 1,000 lines of TS Jest tests. The TypeScript can be easily converted to plain JS and the tests have been easy to convert back and forth from Jest to Ava, so that should not be a problem if it's desired to use JS/Ava.

p-map + p-queue

Hi @sindresorhus

Maybe I haven't catch right module but it would be great to have mix of p-map + p-queue to be able to run the same promise-returning & async function on inputs queue.

For example:

const queue = new Queue((item) => {
  return got('https://sindresorhus.com/?unicorn=' + item);
}, { concurrency: 2 });

(async () => {
  await queue.add('🦄');
  await queue.add(async () => '🦄');
  await queue.add('🐴');
  await queue.add( '🐙');
})();