GithubHelp home page GithubHelp logo

breathe's People

Contributors

auzel avatar jiaming1999 avatar mrob95 avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar

Forkers

wolfmanstout haughki dsw88 sturmianseq jiaming1999 suzzy777 pretty-parrot versatilus

breathe's Issues

Kaldi issues

Breathe works well currently on natlink and the dragonfly test engine, but there are still a few issues getting it to work with kaldi:

  1. On start-up:
Traceback (most recent call last):
  File "kaldi_module_loader_plus.py", line 161, in <module>
    main()
  File "kaldi_module_loader_plus.py", line 151, in main
    engine.prepare_for_recognition()
  File "C:\Users\Mike\Documents\GitHub\new_merger2\winpython\python-2.7.13.amd64\lib\site-packages\dragonfly\engines\backend_kaldi\engine.py", line 243, in prepare_for_recognition
    self._compiler.prepare_for_recognition()
  File "C:\Users\Mike\Documents\GitHub\new_merger2\winpython\python-2.7.13.amd64\lib\site-packages\kaldi_active_grammar\compiler.py", line 421, in prepare_for_recognition
    self.process_compile_and_load_queues()
  File "C:\Users\Mike\Documents\GitHub\new_merger2\winpython\python-2.7.13.amd64\lib\site-packages\kaldi_active_grammar\compiler.py", line 396, in process_compile_and_load_queues
    for kaldi_rule in results:
  File "C:\Users\Mike\Documents\GitHub\new_merger2\winpython\python-2.7.13.amd64\lib\site-packages\concurrent\futures\_base.py", line 641, in result_iterator
    yield fs.pop().result()
  File "C:\Users\Mike\Documents\GitHub\new_merger2\winpython\python-2.7.13.amd64\lib\site-packages\concurrent\futures\_base.py", line 462, in result
    return self.__get_result()
  File "C:\Users\Mike\Documents\GitHub\new_merger2\winpython\python-2.7.13.amd64\lib\site-packages\concurrent\futures\thread.py", line 63, in run
    result = self.fn(*self.args, **self.kwargs)
  File "C:\Users\Mike\Documents\GitHub\new_merger2\winpython\python-2.7.13.amd64\lib\site-packages\kaldi_active_grammar\compiler.py", line 390, in <lambda>
    results = executor.map(lambda kaldi_rule: kaldi_rule.finish_compile(), self.compile_queue)
  File "C:\Users\Mike\Documents\GitHub\new_merger2\winpython\python-2.7.13.amd64\lib\site-packages\kaldi_active_grammar\compiler.py", line 99, in finish_compile
    self.compiler._compile_agf_graph(compile=True, nonterm=self.nonterm, input_data=self._fst_text, filename=self.filepath)
  File "C:\Users\Mike\Documents\GitHub\new_merger2\winpython\python-2.7.13.amd64\lib\site-packages\kaldi_active_grammar\compiler.py", line 283, in _compile_agf_graph
    compile_command()
  File "C:\Users\Mike\Documents\GitHub\new_merger2\winpython\python-2.7.13.amd64\lib\site-packages\ush.py", line 585, in __call__
    return wait(procs, raise_on_error)
  File "C:\Users\Mike\Documents\GitHub\new_merger2\winpython\python-2.7.13.amd64\lib\site-packages\ush.py", line 189, in wait
    result = tuple(iterate_outputs(procs, raise_on_error, status_codes))
  File "C:\Users\Mike\Documents\GitHub\new_merger2\winpython\python-2.7.13.amd64\lib\site-packages\ush.py", line 221, in iterate_outputs
    raise ProcessError(process_info)
ush.ProcessError: One or more commands failed: [(['C:\\Users\\Mike\\Documents\\GitHub\\new_merger2\\winpython\\python-2.7.13.amd64\\lib\\site-packages\\kaldi_active_grammar\\exec\\windows\\fstcompile', '--isymbols=kaldi_model_zamia\\words.txt', '--osymbols=kaldi_model_zamia\\words.txt'], 1868, 0), (['C:\\Users\\Mike\\Documents\\GitHub\\new_merger2\\winpython\\python-2.7.13.amd64\\lib\\site-packages\\kaldi_active_grammar\\exec\\windows\\compile-graph-agf', '--arcsort-grammar', '--grammar-prepend-nonterm=999995', '--grammar-append-nonterm=999996', '--nonterm-phones-offset=993', '--read-disambig-syms=kaldi_model_zamia\\disambig.int', '--verbose=0', 'kaldi_model_zamia\\tree', 'kaldi_model_zamia\\final.mdl', 'kaldi_model_zamia\\L_disambig.fst', '-', 'kaldi_model_zamia.tmp\\f7f2e5a8f575f98a65a4cf94151db0eaf3b63981.fst'], 2092, -1)]
Press any key to continue . . .

Appears to be a problem with loading a grammar with an empty list, disabling the context switcher fixes this.

FIXED by adding an "enable breathe" command context so that the list is never empty on start-up. I'm not 100% sure this is actually a good feature but for now it fixes the problem.

  1. I think a similar patch to dragonfly 149 will need to be applied to the kaldi engine to get rule management working correctly, but this produces some new errors.
engine (INFO): Listening...
Speech started.
engine (INFO): Loading grammar SG8
kaldi.wrapper (WARNING): wrong len(grammars_activity) = 10 != 9 = num_grammars
Press any key to continue . . .

FIXED No modifications to the engine necessary, issue is fixed by passing lazy_compilation=False in the get_engine call in kaldi_module_loader_plus.

Example with nested grammar?

I'd love to see an example with a nested grammar. I'm trying to hack up something for latex that could produce something like

 \frac{\sin(x^2)}{3+x}

from an utterance like "fraction sin x squared divided by 3 + x". I'm thinking the spec could look like "fraction divided [by] " but I'm not sure how to make numerator and denominator both also be math expressions. This seems like maybe a job for RuleRef, but I'm not sure how to manage that and some sort of example would be lovely.

P.S. I'm enjoying Breathe so far!

Support weights for commands?

Is it possible to support the "weight" feature of the Kaldi backend for commands? I'm having issues with some commands being recognized when I don't want them to be. It looks like add_commands could just pass this value along. I'm not sure how well this will work but it would be nice to be able to try it.

When does enabling or disabling CommandContext take effect?

I'm wondering whether within a CCR it is effective to disable/enable a CommandContext? I'm thinking of using this to enable changes between text and math modes for latex, but am unsure of the implications of such a trick. If there is a better approach for such a "mode switching" grammar I'd love to hear about it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.