GithubHelp home page GithubHelp logo

voxpelli / buffered-async-iterable Goto Github PK

View Code? Open in Web Editor NEW
9.0 2.0 1.0 126 KB

Buffered parallel processing of async iterables / generators

License: MIT License

JavaScript 99.81% Shell 0.19%
async-generator async-generators async-iterable async-iterables async-iterator async-iterators buffering

buffered-async-iterable's Introduction

Buffered parallel processing of async iterables / generators.

npm version npm downloads Module type: ESM Types in JS neostandard javascript style Follow @voxpelli@mastodon.social

Usage

Simple

import { bufferedAsyncMap } from 'buffered-async-iterable';

async function * asyncGenerator() {
  yield ...
}

const mappedIterator = bufferedAsyncMap(asyncGenerator(), async (item) => {
  // Apply additional async lookup / processing
});

for await (const item of mappedIterator) {
  // Consume the buffered async iterable
}

Array input

import { bufferedAsyncMap } from 'buffered-async-iterable';

const mappedIterator = bufferedAsyncMap(['foo'], async (item) => {
  // Apply additional async lookup / processing
});

for await (const item of mappedIterator) {
  // Consume the buffered async iterable
}

Async generator result

import { bufferedAsyncMap } from 'buffered-async-iterable';

const mappedIterator = bufferedAsyncMap(['foo'], async function * (item) => {
  // Apply additional async lookup / processing
  yield ...
  yield * ...
});

for await (const item of mappedIterator) {
  // Consume the buffered async iterable
}

API

bufferedAsyncMap()

Iterates and applies the callback to up to bufferSize items from input yielding values as they resolve.

Syntax

bufferedAsyncMap(input, callback[, { bufferSize=6, ordered=false }]) => AsyncIterableIterator

Arguments

  • input – either an async iterable, an ordinare iterable or an array
  • callback(item) – should be either an async generator or an ordinary async function. Items from async generators are buffered in the main buffer and the buffer is refilled by the one that has least items in the current buffer (input is considered equal to sub iterators in this regard when refilling the buffer)

Options

  • bufferSizeoptional – defaults to 6, sets the max amount of simultanoeus items that processed at once in the buffer.
  • orderedoptional – defaults to false, when true the result will be returned in order instead of unordered

mergeIterables()

Merges all given (async) iterables in parallel, returning the values as they resolve

Syntax

mergeIterables(input[, { bufferSize=6 }]) => AsyncIterableIterator

Arguments

  • input – an array of async iterables, ordinare iterables and/or arrays

Options

  • bufferSizeoptional – defaults to 6, sets the max amount of simultanoeus items that processed at once in the buffer.

Similar modules

buffered-async-iterable's People

Contributors

github-actions[bot] avatar renovate[bot] avatar voxpelli avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

edad95

buffered-async-iterable's Issues

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

  • chore(deps): update test dependencies (@types/chai, chai, husky, mocha)
  • chore(deps): update type dependencies (@types/node, typescript)
  • chore(deps): update dependency @voxpelli/tsconfig to v13
  • chore(deps): lock file maintenance

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

github-actions
.github/workflows/codeql-analysis.yml
.github/workflows/compliance.yml
  • mtfoley/pr-compliance-action 11b664f0fcf2c4ce954f05ccfcaab6e52b529f86
.github/workflows/dependency-review.yml
.github/workflows/lint.yml
.github/workflows/nodejs.yml
.github/workflows/release-please.yml
.github/workflows/ts-internal.yml
npm
package.json
  • @types/chai ^4.3.16
  • @types/chai-as-promised ^7.1.8
  • @types/chai-quantifiers ^1.0.4
  • @types/mocha ^10.0.7
  • @types/node ^18.19.40
  • @types/sinon ^17.0.3
  • @types/sinon-chai ^3.2.12
  • @voxpelli/eslint-config ^20.0.0
  • @voxpelli/tsconfig ^12.0.1
  • c8 ^10.1.2
  • chai ^4.4.1
  • chai-as-promised ^7.1.2
  • chai-quantifiers ^1.0.18
  • eslint ^9.7.0
  • husky ^9.0.11
  • installed-check ^9.3.0
  • knip ^5.26.0
  • mocha ^10.6.0
  • npm-run-all2 ^6.2.2
  • sinon ^18.0.0
  • sinon-chai ^3.7.0
  • type-coverage ^2.29.1
  • typescript ~5.5.3
  • validate-conventional-commit ^1.0.3
  • node >=18.6.0

  • Check this box to trigger a request for Renovate to run again on this repository

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.