benji6 / virtual-audio-graph

:notes: Library for declaratively manipulating the Web Audio API

virtual-audio-graph

Overview

Small and dependency-free library for declaratively manipulating the Web Audio API.

virtual-audio-graph manages the state of the audio graph so this does not have to be done manually. Simply declare what you would like the audio graph to look like and virtual-audio-graph takes care of the rest.

Inspired by virtual-dom and React.

Installation

npm i -S virtual-audio-graph

virtual-audio-graph is distributed as a bundled CJS module by default, however, there is also a module property in its package.json so tools like Rollup and Webpack can consume an ES modules build.

Size

virtual-audio-graph is designed to be small and weighs in at 11.9kB minified (3.3kB minified & gzipped) according to bundlephobia (https://bundlephobia.com/[email protected]).

Docs

First check out the virtual-audio-graph guide for working examples and to understand how the library works.

Then see the API docs for all supported virtual audio node factory functions.

For an example of virtual-audio-graph working in a real application you can take a look at Andromeda which is one of my other projects.

virtual-audio-graph's Issues

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

---

Invalid npm token.

The npm token configured in the NPM_TOKEN environment variable must be a valid token allowing to publish to the registry https://registry.npmjs.org/.

If you are using Two-Factor Authentication, make configure the auth-only level is supported. semantic-release cannot publish with the default auth-and-writes level.

Please make sure to set the NPM_TOKEN environment variable in your CI with the exact value of the npm token.

---

Good luck with your project ✨

Your semantic-release bot 📦🚀

LFO

Hi! Thanks again for the great lib. I've quickly hacked a ClojureScript wrapper around it and it is a dream come true. While working on it I got to the point when I'd like to have an LFO to automate an audio parameter. Is it possible to replicate this case with Virtual Audio Graph?

var saw = context.createOscillator(),
      sine = context.createOscillator(),
      sineGain = context.createGainNode();

//set up our oscillator types
saw.type = saw.SAWTOOTH;
sine.type = sine.SINE;

//set the amplitude of the modulation
sineGain.gain.value = 10;

//connect the dots
sine.connect(sineGain);
sineGain.connect(saw.frequency);

A custom node inside of a custom node

Hello @benji6,

I'm having another problem that could be a bug in the library.
I'm trying to extract setting a curve on a gain node into an AR component.
It looks like my code should work, but it doesn't. Please, take a look when you have some time for it.
This works:
https://codesandbox.io/s/relaxed-browser-lbhzz?file=/src/index.ts
This doesn't:
https://codesandbox.io/s/vibrant-buck-hlidp?file=/src/index.ts

Thanks!
/Nik

defineNodes to replace defineNode

instead of:

virtualAudioGraph.defineNode(pingPongDelayParamsFactory, 'pingPongDelay');
virtualAudioGraph.defineNode(oscillatorBankParamsFactory, 'oscillatorBank');

this seems nicer:

virtualAudioGraph.defineNodes({oscillatorBank, pingPongDelay});

VAG samples not working in safari

Hello Ben,

I tried examples from https://virtual-audio-graph.netlify.com in safari (12.1.2) today and none of them worked.

I see this in console.
ReferenceError: Can't find variable: AudioContext

Implement support for OfflineAudioContext?

Only 1 layer of custom nodes?

In my experiments when I create custom nodes that themselves contain custom nodes I don't hear any output. Is that a limitation? I don't think it's a big problem this way just to know.

Custom Nodes and AudioWorklets

Really liking the declarative configurable functional way virtual-audio-graph works. An AudioWorklet is yet another type of custom node, can virtual-audio-graph integrate them?

Use Rollup

Rollup

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

---

The push permission to the Git repository is required.

semantic-release cannot push the version tag to the branch master on remote Git repository.

Please refer to the authentication configuration documentation to configure the Git credentials on your CI environment.

---

Good luck with your project ✨

Your semantic-release bot 📦🚀

Persisting audio graph definitions using functions

Hi

Using versions before v0.20.0, where audio graphs are defined as arrays, it's easy to save the definitions, in e.g. a document database. But after the change to using functions for the audio graph definitions, I'm unsure how to go about persisting them:

Any ideas on how to achieve that?

virtual-hyperscript inspired API change?

Considering adopting a new and potentially more elegant API:

const proposedAPI = {
  0: ['oscillator', 'output'],
  1: ['gain', {key: 0, destination: 'detune'}, {gain: 0.5}, 'input'],
  2: ['oscillator', 1, {frequency: 110}],
};

const currentAPI = {
  0: {
    node: 'oscillator',
    output: 'output',
  },
  1: {
    input: 'input',
    node: 'gain',
    output: {key: 0, destination: 'detune'},
    params: {
      gain: 0.5,
    },
  },
  2: {
    node: 'oscillator',
    output: 1,
    params: {
      frequency: 110,
    },
  },
}

Access to currently playing sound?

Hi Benji6,
I'm building an app to apply filters to soundtracks in order to imitate the world of the hearing impaired.
It'd be nice to do that to the currently playing sound in a browser, so from another app or source. Is there any way I can access that sound (without going through the microphone) ??
Likely not - it would be a security issue - but I'm just checking anyway.
Thanks
Mark

Example connect to a source from a file

Hi

I'm trying to implement the library to filter existing sound tracks (as examples of hearing loss).
But I'm having having trouble finding the correct method to load an existing audio file so that I can apply the biquad (notch) filter. Where can I find examples where existing files are loaded and used as input for this library?

Thanks
Mark

Is it possible to produce a diff without explicitly providing keys?

I'm wondering if you've thought about this.

Virtual DOMs can infer the identity of an element based on the surrounding tree structure. Would this be possible with graphs?

For context, I should mention I'm currently using your library in dpren/elm-audio-graph. I'd like to be able to pass nodes as values without assigning string/int keys.

How to use in browser.

Hi, probably a simple answer to this but I cant find a .js file to include so that I can start using this. I am not sure how to do the import stuff. Can someone help me?

Support multiple inputs and outputs

Unless I've missed something, there doesn't seem to be support for AudioNodes that have multiple inputs or outputs such as ChannelSplitterNode (https://developer.mozilla.org/en-US/docs/Web/API/ChannelSplitterNode) and ChannelMergerNode (https://developer.mozilla.org/en-US/docs/Web/API/ChannelMergerNode).

This is essential for stereo effects.

With the current design, by using an array you can route a mono output to multiple different nodes (which have mono inputs), but for a node that supports multiple inputs (stereo, 6 channel surround, etc), there is no way of specifying the specific input to connect the output to.

For example, here is an example stereo delay using the Web Audio API:
https://github.com/web-audio-components/delay/blob/master/index.js#L154.
In this example, the first output of rightFilter is connected to the second input of _merge ( a ChannelMergerNode that takes two inputs in this configuration).

Possible bug with offset

Hello Ben,

I'm not really sure but there might be a bug with offset in bufferNode. But I can't figure out the reason. Basically everything looks the same. Offset is passed into audioNode.start(startTime, offsetTime);

Here is the example repo
https://github.com/catz/vag-offset

Plays with correct offset

        ....
        source.buffer = sampleBuffer;
        source.playbackRate.value = 0.19971255630630633;
        source.offsetTime = null;
        source.start(context.currentTime, 0.5813333333333334);
        ...

But this one plays always from the start

          ....
          virtualAudioGraph.update({
                                   0: gain('output', { gain: 0.2 }),
                                   1: bufferSource(0, {
                                        buffer: sampleBuffer,
                                        playbackRate : 0.19971255630630633,
                                        loop: false,
                                        startTime: currentTime,
                                        offsetTime: 0.5813333333333334
                                      })
                                   });

Thank you for your library. Works fine with Elm lang

v1.1.1 release don't include the compiled module

Hello,

The latest v1.1.1 npm package don't include dist/index.js and I failed building my app. I needed to do cd node_modules/virtual-audio-graph, npm i -g rollup and npm run build to build. v1.1.0 includes dist/index.js. Could you fix it?

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

---

The push permission to the Git repository is required.

semantic-release cannot push the version tag to the branch master on remote Git repository.

Please refer to the authentication configuration documentation to configure the Git credentials on your CI environment.

---

Good luck with your project ✨

Your semantic-release bot 📦🚀

create node with no output?

is it possible to create a node with no output? it seems like it is explicitly disallowed, but the spec respects the option. for instance an AnalyserNode may exist with no output.

Redundant `exponentialRampToValueAtTime` calls cause click noises

Thanks for this great library.

I found some problems in AudioParam. When virtual-audio-graph updates AudioParam by function such as exponentialRampToValueAtTime multiple times, redundant function calls causes click noise. I confirmed it in Chrome 74 and Firefox 66, Windows 10 and Android 8. My SSCCE and demo:

import createVirtualAudioGraph, * as V from 'virtual-audio-graph';

document.getElementById("start").addEventListener("click", () => {
    const virtualAudioGraph = createVirtualAudioGraph();
    const currentTime = virtualAudioGraph.currentTime;
    setInterval(() => {
        virtualAudioGraph.update({
            0: V.gain('output', {
                gain: [
                    ['setValueAtTime', 1, currentTime + 1],
                    ['exponentialRampToValueAtTime', 0.0001, currentTime + 2]
                ]
            }),
            1: V.oscillator(0, { stopTime: currentTime + 2 })
        });
    }, 100);
});

I also found a following quick fix using a flag like audioNode[key].applied = true;:

if (Array.isArray(param)) {
    if (this_2.params && !(0, _utils.equals)(param, this_2.params[key])) {
        audioNode[key].cancelScheduledValues(0);
        audioNode[key].applied = false;
    }
    if (!audioNode[key].applied) {
        audioNode[key].applied = true;
        var callMethod = function (_a) {
        var methodName = _a[0],
            args = _a.slice(1);

        return (_b = audioNode[key])[methodName].apply(_b, args);

        var _b;
        };

        Array.isArray(param[0]) ? param.forEach(callMethod) : callMethod(param);
    }

} else {
    audioNode[key].value = param;
}

I can make a pull request about it, but it looks so dirty. I have not deep-dived into the source codes of this library, so I have no good idea to fix it yet. I believe AudioWorkletVirtualAudioNode also have same issue.

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

---

The push permission to the Git repository is required.

semantic-release cannot push the version tag to the branch master on remote Git repository.

Please refer to the authentication configuration documentation to configure the Git credentials on your CI environment.

---

Good luck with your project ✨

Your semantic-release bot 📦🚀

Smoothing API

Been getting a lot of these warnings in Chrome:

[Deprecation] GainNode.gain.value setter smoothing is deprecated and will be removed in M64, around January 2018. Please use setTargetAtTime() instead if smoothing is needed. See https://www.chromestatus.com/features/5287995770929152 for more details.

I'm thinking about whether an option for default smoothing could be added to the API?

Support OfflineAudioContext (repeat #8)

Looking to do some audio processing with WebAudio API and need OfflineAudioContext to do it!

Tried forking the library and contributing myself, but honestly I never got into typescript... started changing types wholesale from AudioContext to AudioContext | OfflineAudioContext and then the ts compiler started yelling at me for an unrelated type issue.

Beats me, honestly... It wasn't satisfied even after my any typecast :( Guess I'm too dumb to prove all this stuff to the compiler lol

Thank you so much for virtual-audio-graph

Hello @benji6 sorry to open a ticket but I couldn't figure out another way to contact you. I just wanted to say thank you very much for virtual-audio-graph, it's really wonderful! 🙏

Code in custom node is interpreted in a different way

Hi, the name of the bug is actually an assumption. I have isolated the problem in two code sandboxes.

This works perfectly fine. Top level audio graph definition is updated on setInterval:
https://codesandbox.io/s/relaxed-ramanujan-7qdmr?file=/src/index.js

This one doesn't work as expected. Seemingly, the same code produces only two events:
https://codesandbox.io/s/elegant-heisenberg-leht2?file=/src/index.js

I do hope it's my mistake, but I can't find it.

Otherwise, thanks for the great work. I'm having a crush on this library and going to base my audio experiments on it (replacing tonejs, that does too much).

defineNodes to accept data structure instead of factory

Currently defineNode is called with a factory like this:

const pingPongDelayParamsFactory = ({
  decay = 1 / 3,
  delayTime = 1 / 3,
  maxDelayTime = 1 / 3,
} = {}) => ({
  zero: ['stereoPanner', 'output', {pan: -1}],
  1: ['stereoPanner', 'output', {pan: 1}],
  2: ['delay', [1, 'five'], {delayTime, maxDelayTime}],
  3: ['gain', 2, {gain: decay}],
  4: ['delay', ['zero', 3], {delayTime, maxDelayTime}],
  five: ['gain', 4, {gain: decay}, 'input']
});

However, it would be good if instead it could be called with a data structure and have the same behaviour. This opens doors to safely importing 3rd party custom nodes. Propose an API something like this:

const pingPongDelayCustomNode = {
  zero: ['stereoPanner', 'output', {pan: -1}],
  1: ['stereoPanner', 'output', {pan: 1}],
  2: ['delay', [1, 'five'], {delayTime: '@@param/delayTime', maxDelayTime: '@@param/maxDelayTime'}],
  3: ['gain', 2, {gain: '@@param/decay'}],
  4: ['delay', ['zero', 3], {delayTime: '@@param/delayTime', maxDelayTime: '@@param/maxDelayTime'}],
  five: ['gain', 4, {gain: '@@param/decay'}, 'input']
};

Better documentation

The demos could be improved and written a bit more like a tutorial and it would be good if the API documentation could be made simpler

Working with the Configuration Object

As you know the world of possible timbres is very large, many configurations are possible. To manage this efficiently it would be great if these timbre configurations could be stored as JSON on a backend then fetched and injected into a virtual-audio-graph at runtime. It does look like the virtualAudioGraph.update function is already designed for such a scenario.

For storing and querying timbre configurations it would be useful to add optional descriptive tags to a configuration. This way a configuration could be named, have a suitable pitch range specified, make note of the author and include other info tags that help fetch and sort and apply timbre configuration objects. This would be extremely helpful both for standard orchestra timbres as well as newly invented ones.

With this in place it is then much easier to fetch and load any timbres that a newly activated score document would reference. For instance, an app can load a score and then subsequently also resolve the timbre configurations it uses, so they could be queried, loaded and updated in the AudioContext.

This would make timbre management so much better, what do you think?

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

---

Invalid npm token.

The npm token configured in the NPM_TOKEN environment variable must be a valid token allowing to publish to the registry https://registry.npmjs.org/.

If you are using Two-Factor Authentication, make configure the auth-only level is supported. semantic-release cannot publish with the default auth-and-writes level.

Please make sure to set the NPM_TOKEN environment variable in your CI with the exact value of the npm token.

---

Good luck with your project ✨

Your semantic-release bot 📦🚀

Upgrade to Babel 6

AudioContext in Safari always 'suspended'

Hi
Just asking if you have a solution to kickstart the audiocontext in Safari ?? It always starts in state 'suspended' and the usual call to resume() has no effect (so waiting for the promise or onStateChange doesnt happen).
So, like this is not helping:

    audioContext = new window.webkitAudioContext()
    audioContext.resume()

If you have any experience with this, I'm all ears! Thanks
Mark

EDIT
fixed, this blog helped: https://www.mattmontag.com/web/unlock-web-audio-in-safari-for-ios-and-macos
Thnaks

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

---

No npm token specified.

An npm token must be created and set in the NPM_TOKEN environment variable on your CI environment.

Please make sure to create an npm token and to set it in the NPM_TOKEN environment variable on your CI environment. The token must allow to publish to the registry https://registry.npmjs.org/.

---

Good luck with your project ✨

Your semantic-release bot 📦🚀