nicolas-van / modern-async

A modern JavaScript tooling library for asynchronous operations using async/await, promises and async generators

modern-async

A modern JavaScript tooling library for asynchronous operations using async/await, promises and async generators.

This library is a modernized alternative to a lot of libraries like Async.js that were created using the legacy callback style to handle asynchronous operations. Its goal is to be as complete as any of those libraries while being built from the very beginning with async/await and promises in mind.

See the documentation.

• Exclusively uses async/await, promises and async generators in its code, tests and documentation.
• Has low bundle size.
• Has 100% code coverage.
• Bundled for ESM modules, CommonJS and UMD.
• Works in node >= 8 and in the vast majority of browsers (very old browser compatibility can be achieved using Babel and shims).
• Has Typescript support.

Installation

npm install --save modern-async

Or use jsDelivr to get the UMD version. The content of the library will be available under the modernAsync global variable.

Usage

import { asyncMap, asyncSleep } from 'modern-async'

const array = [1, 2, 3]
const result = await asyncMap(array, async (v) => {
  await asyncSleep(10)
  return v * 2
})
console.log(result)

See the documentation for the rest.

Migrating from version 1.X to version 2.X

See the migration guide.

Changelog

The changelog.

Contribution Guide

The contribution guide

License

The license.

modern-async's Issues

flatMap

Please explain the feature or improvement you would like:

Please add async flatMap

Please describe the use case where you would need that feature (the general situation or type of program where that would be helpful):

Currently the best counterpart to

arr.flatMap((elt) => f(elt))

is

(await asyncMap(arr, async (elt) => await f(elt))).flatMap(elt => elt)

Which is janky. I would prefer

await asyncFlatMap(arr, async (elt) => await f(elt))

Please explain why you think that feature would be helpful to other people:

Because flatMap is in stdlib Array

Could we add a `mapValues` implementation into the collection?

Please explain the feature or improvement you would like:

I would love to see the addition of mapValues to the collection. This method would asynchronously transform the values of an object by applying the iteratee to each.

Please describe the use case where you would need that feature (the general situation or type of program where that would be helpful):

I often use records to organize my data, such as a record of user IDs paired with an access token:

const accessTokensByUserId: Record<Uuid, string> = {
    [Uuid(...)]: "token1",
    [Uuid(...)]: "token2",
}

.mapValues would be useful here if I'd like to make a request for each user using their relative access-key:

async function fetchBirthday(accessToken: string): Promise<Date> { ... }

const birthdayByUser: Record<Uuid, Date> = await async.mapValues(accessTokensByUserId, fetchBirthday)

This would be opposed to the manual implementation:

async function fetchBirthday(accessToken: string): Promise<Date> { ... }

const birthdaysByUser: Record<Uuid, Date> = Object.fromEntries(
    await async.map(Object.entries(birthdaysByUser), async ([uuid, token]) => {
        const birthday = await fetchBirthday(token)
        return [uuid, token]
    })
) as Record<Uuid, Date> // Needed since Object.fromEntries types its keys as `string` where before they were `Uuid`

If you know another library similar to this one that already propose that feature please provide a link:
Async's mapValues

forEachLimit is ignoring the concurrent paramter

See this Code:

//@ts-ignore
var {forEachLimit}=require('modern-async')

var ar=[]
for(let i=0;i<=100;i++) ar.push(i)

forEachLimit(ar,async item=>{
  await delay(1000)
  console.log(item)
},5)
function delay(ms){return new Promise(resolve=>setTimeout(resolve,ms))}

It should show every 1 seconds 5 items. But it shows after 1 second all the items.
I found the bug and I wanted to fix it:

In the file forEachLimit.js:

async function forEachLimit (iterable, iteratee, concurrency) {
  await mapLimit(iterable, async (v, i, t) => {
    iteratee(v, i, t)
  }, concurrency)
}

missing await before iteratee(v, i ,t)

I wanted to fix it but the current version in GitHub is different (in development).
Can you provide hotfix and publish it to NPM as 1.0.4?
It's really a security issue for servers to handle too many requests at the same time.

By the way, thank you for you awesome library.

Test with parcel

tsc error "library may need to update its package.json"

Describe the bug

Getting the following error from tsc:

file.ts:4:23 - error TS7016: Could not find a declaration file for module 'modern-async'. 'node_modules/modern-async/src/modern-async.mjs' implicitly has an 'any' type.
  There are types at 'node_modules/modern-async/modern-async.d.ts', but this result could not be resolved when respecting package.json "exports". The 'modern-async' library may need to update its package.json or typings.

import { sleep } from 'modern-async';

To Reproduce

Any import from modern-async

Expected behavior

No error

Additional context

I believe this only happens under ESM with TypeScript 5.

Fix should be as simple as adding the correct value in export.types in package.json.

https://stackoverflow.com/questions/57383049/typescript-esnext-compiler-option-destroys-es6-import-from-external-lib

Question about Synchronous Long Executions and Code breaking

Dear Nicolas and contributors, I have a question about blocking sync operations.

I have a long array, lets say for example 10000 in length. And I have to execute simple functions such as running a multiplication or simply squaring values and manipulating the objects inside of this array.

On a web server the main goal is to avoid blocking code at all costs. So the alternatives available are:

1. Use a for loop THIS WILL BLOCK EXECUTION UNTIL DONE! NOT A SOLUTION.

2. Use a .map / .forEach on the array to execute the sync function on each element.

It does not block the event loop, however I am not sure of the impact on the execution as it would be thousands of sync functions piling.

3. Use Modern Async with Limit this way, the sync functions would be transformed in async and setting the limit for example 1 Promise at a time will permit the code to act as one big promise that will allow for each execution of the sync function for the event loop to run and other functions or requests to also be executed and queued without blocking any part of the code and running as many functions as possible when possible.

This method in my opinion might be slight less performant because of all the overhead of promises and callbacks, but it would permit the breaking of a long iterator into small pieces pieces with small intervals to prevent the blocking of the execution.

Please let me know with your knowledge what would you do ? Or if I am wrong about sync .map .forEach

Thanks
Pedro

TypeScript Support

Please explain the feature or improvement you would like:
TypeScript type support, mainly the type declaration.

Please describe the use case where you would need that feature (the general situation or type of program where that would be helpful):
For projects that use TypeScript, type declaration is needed for type checking and autocomplete.

Please explain why you think that feature would be helpful to other people:
A lot of projects are using and migrating to TypeScript. For example, Visual Studio Code is using TypeScript.

If you know another library similar to this one that already propose that feature please provide a link:

• native-promise-util, I am the maintainer of it :)
• promise-fun

Security issues

I'm evaluating using modern-async for a production application but I'm concerned about the security issues that Socket.dev identified.

Aside from a number of npm audit issues, Socket.dev says modern-async include install scripts and that it can access the network. Maybe a security audit of modern-async could uncover why these issues are flagged.

https://socket.dev/npm/package/modern-async/overview/1.1.3

Getting modern-async's security story up-to-date could help adoption. The library looks amazing otherwise, thanks for the hard work!

Aggregate error wrapper like map/mapSeries

Please explain the feature or improvement you would like:

I'd like to see a variant of forEach/forEachSeries which completes all items (rather than cancelling on first failure), collects all errors (rather than failing on the first), and rejects with an AggregateError (if more than one failed? or even if just one failed?)

Please describe the use case where you would need that feature (the general situation or type of program where that would be helpful):

I find myself using this pattern a lot when writing some types of backend batch processing code:

var errors = [];
for (…) {
  try {
    // Do some async work
  catch(err) {
    errors.push(err)
  }
}

if (errors.length) {
  throw new AggregateError(errors);
}

Please explain why you think that feature would be helpful to other people:

I imagine I am not the only one trying to handle partial failures in a backend 🙂

For `reduce` and `reduceRight` it should be possible to use any type for accumulator

For now it is strictly fixed to iterator item type

Include a utility similiar to Async's "reflect"

Please explain the feature or improvement you would like:
The "Async" library includes a utility call "reflect" which essentially wraps a callback with a try-catch. The resulting callback is then guaranteed to succeed with a results object that either contains the result of the function within a value property, or the returned error within a failure property. It would be incredibly useful to see the inclusion of a similar utility in modern-async that wraps promises or promise-generating methods.

Their example:

async.parallel([
    async.reflect(function(callback) {
        // do some stuff ...
        callback(null, 'one');
    }),
    async.reflect(function(callback) {
        // do some more stuff but error ...
        callback('bad stuff happened');
    }),
    async.reflect(function(callback) {
        // do some more stuff ...
        callback(null, 'two');
    })
],
// optional callback
function(err, results) {
    // values
    // results[0].value = 'one'
    // results[1].error = 'bad stuff happened'
    // results[2].value = 'two'
});

Please describe the use case where you would need that feature (the general situation or type of program where that would be helpful):"
"Reflect" would reduce the amount of boilerplate code required for error management when using functions like .map, which is otherwise pretty streamlined.

For example:

//  type ReflectResult<T> = {
//    value?: T;
//    error?: unknown;
//  };

function dangerouslyGetBirthday(username: string): Promise<Date> {}

const usernames = ["user1","user2"]
const birthdayResults: ReflectedResult<Date>[] = 
    async.map(usernames, reflect(username => dangerouslyGetBirthday(username)))

const birthdays = birthdayResults.map(result => result.value).filter(value => !!value)
const failures = birthdayResults.map(result => result.error).filter(error => !!error)

As for alternatives, I'm actually unsure what tools modern-async offers for error handling.

If you know another library similar to this one that already propose that feature please provide a link:
Async's Reflect

We need replaceAsync and replaceAsyncLimit

There is already string-replace-async library on npm: https://www.npmjs.com/package/string-replace-async with 12,000 weekly downloads but it run all in parallel.
We need replaceAsyncLimit and replaceAsyncSeries

I think it will be very fit this library