GithubHelp home page GithubHelp logo

msqr1 / vosklet Goto Github PK

View Code? Open in Web Editor NEW
13.0 1.0 1.0 247.5 MB

A speech recognizer that can run on the browser, inspired by vosk-browser

License: Apache License 2.0

JavaScript 28.12% C++ 35.86% Shell 36.01%
api javascript speech-recognizer vosk webassembly

vosklet's Introduction

Overview

  • A speech recognizer built on Vosk that can be run on the browser, inspired by vosk-browser, but built from scratch and no code taken!
  • Designed with basic/nothrow exception safety
  • See the examples folder for examples on using the API.
  • See API.md for the API reference
  • See test for a developer build script for just the JS
  • Note: The examples uses examples/Vosklet.js because I can't set the Response headers for my model for browsers to decompress correctly. Instead, I used DecompressionStream to decompress manually, so examples/Vosklet.js only works for the examples. In practice, please use the outside Vosklet.js instead.

Compared to vosk-browser

  • Support multiple models
  • Has models' storage path management
  • Has models' ID management (for model updates)
  • Has smaller JS size (>3.1MB vs 1.2MB gzipped)
  • Has all related files (pthread worker, audio worklet processor,...) merged
  • Has faster processing time
  • Has shorter from-scratch build time
  • Has more Vosk functions exposed

Basic usage (microphone recognition in English)

  • Result are logged to the console.
  • Copied from examples/fromMic.html
<!DOCTYPE html>
<html>
  <head>
    <script src="https://cdn.jsdelivr.net/gh/msqr1/[email protected]/examples/Vosklet.min.js" async defer></script>
    <script>
      async function start() {
        // Make sure sample rate matches that in the training data
        let ctx = new AudioContext({sampleRate : 16000})

        // Setup mic with correct sample rate
        let micNode = ctx.createMediaStreamSource(await navigator.mediaDevices.getUserMedia({
          video: false,
          audio: {
            echoCancellation: true,
            noiseSuppression: true,
            channelCount: 1,
            sampleRate: 16000
          },
        }))

        // Load Vosklet module, model and recognizer
        let module = await loadVosklet()
        let model = await module.createModel("https://ccoreilly.github.io/vosk-browser/models/vosk-model-small-en-us-0.15.tar.gz","model","ID")
        let recognizer = await module.createRecognizer(model, 16000)

        // Listen for result and partial result
        recognizer.addEventListener("result", ev => {
          console.log("Result: ", ev.detail)
        })
        recognizer.addEventListener("partialResult", ev => {
          console.log("Partial result: ", ev.detail)
        })

        // Create a transferer node to get audio data on the main thread
        let transferer = await module.createTransferer(ctx, 128 * 150)

        // Recognize data on arrival
        transferer.port.onmessage = ev => {
          recognizer.acceptWaveform(ev.data)
        }
        // Connect to microphone
        micNode.connect(transferer)
      }
    </script>
    <!-- Start and create audio context only as a result of user's action -->
    <button onclick="start()">Start</button>
  </head>
</html>

vosklet's People

Contributors

msqr1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

Forkers

tdcook

vosklet's Issues

"TypeError: cannot convert undefined to object" in VoskletTransferer

Sometimes, VoskletTransferer will throw a TypeError: cannot convert undefined to object exception immediately after initialization. It won't always happen right away, but usually will after a few reloads. It seems there is a race condition between the connection of the source and processor nodes and the execution of VoskletTransferer's process method.

The error is being thrown here:

this.buffer.set(inputs[0][0], this.count * 128)

Adding a falsiness check here should fix it:

const data = inputs[0][0]
if (!data) return true
this.buffer.set(data, this.count * 128)

ReferenceError: Can't find variable: SharedArrayBuffer

[Error] Unhandled Promise Rejection: ReferenceError: Can't find variable: SharedArrayBuffer
	(anonymous function) (Vosklet.js:9:9494)
	(anonymous function) (index.html:21)

Safari Version 17.3.1 (19617.2.4.11.12)

Happens after clicking start and speaking via the example

Thanks for the lib, still failing to get browser offline speech recognition working with any example from any offering ๐Ÿ™ƒ

Not work with different model

On the step of model's creation (malloc) JS throw error without text but with 1 number, it looks like number of bytes tried to allocate.

Have you built model with fixed memory size? doesn't it allow memory_grow?

evaluation give worse metrics then server-based VOSK. Way to run it sync for test?

I have a dataset of syllables on which I perform the evaluation.
Each wav = 1 syllable (0.2 sec). Just read one by one, recognize, check with ground true.

Vosk on the host gives ~80% accuracy.

Vosklet give ~65%, Possible it's because async nature of the js wrapper.
I use a timeout 2 sec to divide different wav recognition, but 11% of records have nothing as recognized result, and 8% contains 2 syllables.

Other things all the same between hosted Vosk and browser Vosklet.
Maybe running in main thread synchronously will help?

Import via module: Can't find variable window

I've installed the library via NPM and load with import loadVosklet from 'vosklet';

Get the following error running the demo code from the readme

vosklet_demo.js:23 worker sent an error! http://localhost:4000/assets/app.js:2855: Uncaught ReferenceError: window is not define
worker.onerror	@	Vosklet.js:9
error (async)		
(anonymous)	@	Vosklet.js:9
loadWasmModuleToWorker	@	Vosklet.js:9
loadWasmModuleToAllWorkers	@	Vosklet.js:9
(anonymous)	@	Vosklet.js:9
callRuntimeCallbacks	@	Vosklet.js:9
preRun	@	Vosklet.js:9
run	@	Vosklet.js:9
runCaller	@	Vosklet.js:9
removeRunDependency	@	Vosklet.js:9
receiveInstance	@	Vosklet.js:9
receiveInstantiationResult	@	Vosklet.js:9
Promise.then (async)		
instantiateArrayBuffer	@	Vosklet.js:9
instantiateAsync	@	Vosklet.js:9
createWasm	@	Vosklet.js:9
(anonymous)	@	Vosklet.js:9
startRecognition	@	vosklet_demo.js:23
await in startRecognition (async)		
(anonymous)	@	vosklet_demo.js:7
Show less

working example?

I tried to run on IDE devserver, different services but get error about SharedBufferArray
I created a simple Flask with necessary CORS headers, but now it fails at

let model = await module.createModel("en-model.tgz","model","ID")

with

Uncaught (in promise) TypeError: Failed to fetch

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.