Comments (6)
Thanks for your interest. I merged ff1e2fc that has a script to convert the whisper model weights to an npz file. @RoboMagus please verify and close the issue.
from useful-transformers.
Thanks for the script. It works without problem. Though I've noticed a couple of issues.
- Quite erratic behavior when running in non-english language mode. This works fine for the same model with regular old whisper, though this could easily be because of the number of additional parrameters available to that script. I'm still running some tests to further figure out the cause of this difference.
- Running with any model larger than tiny errors are thrown:
- Running
transscribe_wav
with base model (English language):
E RKNN: [19:58:52.477] failed to convert handle(1019) to fd, ret: -1, errno: 24, errstr: Too many open files
- Running
transscribe_wav
with small.en:
E RKNN: [20:01:32.183] rknn_matmul_create, matmul K must be less than or equal 2048! Segmentation fault
- Running
The repo readme shows that the base model has been covered though, so the fact that RKNN throws errors on me is a bit unexpected. Is there something else required to get that model running?
from useful-transformers.
E RKNN: [19:58:52.477] failed to convert handle(1019) to fd, ret: -1, errno: 24, errstr: Too many open files
This error is due to the Linux limit on the number of fds for the process. You can run this command to increase the number limit
ulimit -n 10000
But when I continued to run the base model, the following error occurred.
n_vocab=51865 python: /userdata/asr/useful-transformers/examples/whisper/whisper.cc:100: TextDecoder::TextDecoder(int, int, int, int, int, int): Assertion
n_vocab % 3 == 0' failed.`
I am still trying to figure out what happened.
from useful-transformers.
Multilingual model needs some work, as n_vocab % 3 != 0
is not yet supported. I have a branch https://github.com/usefulsensors/useful-transformers/tree/keveman/non_multiple_of_3 where the support is implemented. However, I am still testing it with tiny
and base
models.
from useful-transformers.
I changed to using base.en and it runs very well now. Thank you for your help.
from useful-transformers.
Im tinkering with the dev
branch which seems to already have the non_multiple_of_3
features included.
The ulimit trick helped bypass the too many files error, but the behavior is still a bit wonky. It doesn't crash or throw errors, but it only seems to translate the first word and just return that response, or get caught in an interesting state where it behaves like a broken record just repeating a single word / token (just random nonsence) over and over again:
Vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem, vindem,
from useful-transformers.
Related Issues (20)
- Providing help and FLOSS stack HOT 1
- How to build for development? HOT 3
- 请教模型是如何利用3588的npu计算的 HOT 1
- 提问 HOT 1
- please help me HOT 1
- help
- Using my own wav file to transcribe the results was wrong HOT 6
- How to run the transcribe_wav.py code directly instead of using the release wheel package?
- How to fix issue when i build filters_vocab_gen_util.ipnb?
- react native
- Hello, author. Does it support converting chatglm6b into the corresponding model? grateful
- Project Raodmap or Project abandoned? HOT 1
- Support shorter n_audio_ctx HOT 1
- hello,author,Can I ask you exactly how you optimized the model performance of the transformer?
- Is it possible to set input language with whisper-base.tflite? HOT 1
- RK3566 support? HOT 1
- How to run the Whisper on GPU
- Home Assistant addon support?
- Get confidence values?
- Understanding the use of taskset HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from useful-transformers.