meyda / meyda Goto Github PK
View Code? Open in Web Editor NEWAudio feature extraction for JavaScript.
Home Page: https://meyda.js.org/
License: MIT License
Audio feature extraction for JavaScript.
Home Page: https://meyda.js.org/
License: MIT License
Lets clean up the source of the WAC2015 paper on Meyda and open source it either on a branch of this repo or on another repo. I'm not sure I have it anymore - was it in Overleaf or did we keep it somewhere else?
there's already an mfcc, so adding a chroma isn't too much further work.
Here's a good primer on chroma: http://labrosa.ee.columbia.edu/matlab/chroma-ansyn/
It seems that anything not outputting numbers or arrays is broken, looks like the canvas ID set at https://github.com/hughrawlinson/meyda/blob/master/index.html#L213 is wrong, as on https://github.com/hughrawlinson/meyda/blob/master/index.html#L252 you request meyda.featureInfo[v].arrayNames["1"]
instead of meyda.featureInfo[v]["1"]
I suspect the line on 213 should be accessing arrayNames
instead.
It shouldn't do that, in case someone mistakenly includes it somewhere for some reason.
Hello,
After a quick chat with @jakubfiala I mentioned that modularisation would be great for my use-case, so I thought it would be a good idea to post an issue here.
I'm a big fan of commonJS, I've been using it in many different libraries and it has worked great so far, and it allows a really nice workflow…
I've gone ahead and did some basic separations here:
https://github.com/vectorsize/meyda/tree/commonJS
to see how it works and it seems to be working perfectly!
I also did a few performance optimisations. Everything is in a commonJS branch in case u guys like it and feel like incorporating it.
That said if you decide to go for other type of modules, I think what I did is a good starting point.
Also the small optimisations in the main function still make sense.
Let me know what you think.
As this is not very useful (maybe just for visualisations) we should remove this from meyda.js, possibly to a separate file.
Any useful functions and transforms we come up with for the CLI should probably be split out into lib files and exposed for developers to use for their convenience, rather than having people reimplementing things. This is part of the next issue, to do with reconsidering the packaging structure.
Following our discussion – related to #85 – I think the best course of action is what @hughrawlinson suggested.
This would be an offline version of the get
method (couldn't think of a hot name for it yet), which would behave as follows:
get
, i.e. a String
denoting the desired feature, or an Array
of such strings. Argument no. 2 will be a Float32Array containing the signal buffer to analyze.get
would then:
get
wouldMeyda
should be able to run without a specified AudioContext/source nodeAs discussed in #85 and #84 this would create a simple API for offline AFE, and also extend the functionality of a single Meyda
object, as it could now in theory analyze two simultaneous audio streams.
… as they're only dependent on bufferSize and sample rate, which wouldn't change unless a new Meyda is instantiated.
suggested solution:
the scales themselves should be stored as Float32Arrays, e.g. self.barkScale
. They can be calculated once, before the onaudioprocess
handler is attached to the SPN. There can also be conversion functions, as are now in the mfcc extractor. I think it'd be great to keep this all as public variables, so that people could for example have a mel converter built with Meyda, or visualize the bark scale bounds alongside some other things.
This is bound to be controversial, but...
Javascript packages have been more and more moving towards small, core packages to reduce bloat in their dependent packages. Two examples of this are grunt, and babel, who have respectively split out functionality between ['grunt', 'grunt-cli', 'grunt-contrib-sass', 'gulp-contrib-whatever','etc']
and ['babel','babel-es2015','babel-jsx','etc']
. I believe we should at least consider this approach for meyda, purely because there are many different use cases for our users, and a growing one-size-fits-all package might not be the best way to serve users. This isn't a case like C/C++, where a library is included but only the linked-to code remains in the binary after compilation, when a package depends on Meyda it must (some caveats, but usually does) deploy with the entire package, including files that will never get used in the production deployment.
Use cases for meyda include:
Cleary those who wish to contribute to Meyda should use the git repository, but the other usecases are not so straightforward. Frontend developers can use both npm or bower to install packages, so it's not entirely clear which files should be in the npm package. It's pretty clear that the bower package shouldn't include the node distribution files (for those who may be unfamiliar, we compile into a minified concatenated file for the web, and to unminified unconcatenated files with module loading for node), but should the node package contain the distribution files? Should we include instructions in the documentation for web developers using the npm distribution to link to the web distribution file instead of the node one, or should we point out that if they include via npm and therefore get the node distribution from require they should take care that browserify (or whatever equivalent tool) is properly requiring the nested files for their build process? Probably neither of those need to be distributed the source ES6 code or the tests, or the config files for the CI services/linter/etc, or the CLI. Users who want to use the CLI probably don't need the source code, just the distribution code, but users who want to use the node module don't necessarily need the cli.
All of this is fine, if meyda is only used infrequently on a small scale, but if meyda is ever included on a project with a very high deploy frequency, the extra cost of deploying the extraneous files could really add up. Given our new focus on offline, I don't think that scenario is too far fetched. In my opinion, we should probably discuss the above in order to make sure meyda serves the widest variety of use cases in the best way possible.
It seems to vary between 0 and 1.13ish. It needs to be between 0 and 1.
It'd be great to automatically grab a minified version of meyda including the dependencies (fftjs, complex_array) whenever we commit to master. Alternatively, if we just run the build system before we commit to master, that'd work, but a way to test that would be excellent.
I can't believe I'm saying this, but... http://flowtype.org/ is a static type checker for Javascript. It could save us a lot of time writing unit tests. I suggest using weak
mode for now, I'll check out how that might work this evening - I'm hoping to combine it with linting and testing.
This needs to happen once browsers have implemented Audio Workers, but before ScriptProcessorNode is removed from browsers.
It's not immediately clear to me how I hook in a source. Giving an example with a creative commons mp3 would help a lot :)
A script to test speed and accuracy vs other frameworks would be really useful in helping us gauge what use-cases meyda is appropriate for.
Alright so having attempted to use our MFCC extractor as a standalone function in node, I realised that actually I cannot use any spectral features unless the FFT is calculated internally by the Meyda object. We may say this is by design, but in order for the "standalone" extractors to work, we need the ability to calculate the amplitude spectrum without having to run an SPN/audioworker. Otherwise the simplicity of using Meyda in node is compromised by the fact that I first have to FFT my data with a non-meyda method, and then run the extractor on it.
This would be an easy thing to add, and I'll assign this issue to myself if approved. The only ugly part is that the proposed extractor is then basically a wrapper for Nick Jones' jsfft, but whatever.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.