Comments (14)
It does not support LSTM with projections.
from sherpa-onnx.
@csukuangfj How can I export these lstm models? I don't care it support or not, I just need a workable version
from sherpa-onnx.
Please see the doc
I posted it before in another thread.
from sherpa-onnx.
In this version, models of pruned_transducer_stateless3 are used. In this script, models are exported separately, but projection parts of joiner should also be exported separately. You can use the below function to export joiner
def export_joiner_model_onnx(
joiner_model: nn.Module,
joiner_filename: str,
opset_version: int = 11,
) -> None:
"""Export the joiner model to ONNX format.
The exported model has two inputs:
- encoder_out: a tensor of shape (N, encoder_out_dim)
- decoder_out: a tensor of shape (N, decoder_out_dim)
and has one output:
- joiner_out: a tensor of shape (N, vocab_size)
"""
encoder_out_dim = joiner_model.encoder_proj.weight.shape[1]
decoder_out_dim = joiner_model.decoder_proj.weight.shape[1]
encoder_out = torch.rand(1, 1, 1, encoder_out_dim, dtype=torch.float32)
decoder_out = torch.rand(1, 1, 1, decoder_out_dim, dtype=torch.float32)
project_input = False
# Note: It uses torch.jit.trace() internally
torch.onnx.export(
joiner_model,
(encoder_out, decoder_out, project_input),
joiner_filename,
verbose=False,
opset_version=opset_version,
input_names=["encoder_out", "decoder_out", "project_input"],
output_names=["logit"],
dynamic_axes={
"encoder_out": {0: "N"},
"decoder_out": {0: "N"},
"logit": {0: "N"},
},
)
torch.onnx.export(
joiner_model.encoder_proj,
(encoder_out.squeeze(0).squeeze(0)),
str(joiner_filename).replace(".onnx", "_encoder_proj.onnx"),
verbose=False,
opset_version=opset_version,
input_names=["encoder_out"],
output_names=["encoder_proj"],
dynamic_axes={
"encoder_out": {0: "N"},
"encoder_proj": {0: "N"},
},
)
torch.onnx.export(
joiner_model.decoder_proj,
(decoder_out.squeeze(0).squeeze(0)),
str(joiner_filename).replace(".onnx", "_decoder_proj.onnx"),
verbose=False,
opset_version=opset_version,
input_names=["decoder_out"],
output_names=["decoder_proj"],
dynamic_axes={
"decoder_out": {0: "N"},
"decoder_proj": {0: "N"},
},
)
logging.info(f"Saved to {joiner_filename}")
@csukuangfj , I might update onnx exporting script ?
from sherpa-onnx.
@EmreOzkose So the model is conformer not lstm? Please update export script, I want have a tried on onnx.
from sherpa-onnx.
@csukuangfj , I might update onnx exporting script ?
Yes, please.
from sherpa-onnx.
@EmreOzkose Hi, I wanna using wenet Chinese mode, how should I download pretrained model and convert toonnx? Does there any necessary to change thecode?
from sherpa-onnx.
@EmreOzkose So the model is conformer not lstm? Please update export script, I want have a tried on onnx.
Yes, it is not LSTM. I made a PR.
@EmreOzkose Hi, I wanna using wenet Chinese mode, how should I download pretrained model and convert toonnx? Does there any necessary to change thecode?
Actually, I did experiments with only English pre-trained model. I have never worked on a Chinese model. If Chinese model doesn't have extra changes in greedy search, I think it will work.
from sherpa-onnx.
You can export English models as below
git lfs install
git clone https://huggingface.co/csukuangfj/icefall-asr-librispeech-pruned-transducer-stateless3-2022-05-13
cd exp
cp pretrained-iter-824000-avg-18.pt epoch-1.pt
cd path/to/icefall/egs/librispeech/ASR/
./pruned_transducer_stateless3/export.py\
--exp-dir /path/to/icefall-asr-librispeech-pruned-transducer-stateless3-2022-05-13/exp \
--bpe-model /path/to/icefall-asr-librispeech-pruned-transducer-stateless3-2022-05-13/data/lang_bpe_500/bpe.model \
--epoch 1 \
--avg 1 \
--onnx 1
from sherpa-onnx.
This one is for non-streaming Conformer models. It should work for English as well as Chinese models.
from sherpa-onnx.
@csukuangfj Hello, I got an error when load wenet model to inference:
for decoder.embedding.weight: copying a param with shape torch.Size([5537, 512]) from checkpoint, the shape in current model is torch.Size([5539, 512]).
do u know why?
from sherpa-onnx.
I get vocabsize from token got max + 1 : : vocab size: 5539
but model shape is 5537, why?
from sherpa-onnx.
Please show the complete code. It is hard to figure out what goes wrong without seeing the code.
from sherpa-onnx.
Please see
https://k2-fsa.github.io/icefall/model-export/index.html
from sherpa-onnx.
Related Issues (20)
- int8 quantized TTS model slower than fp32 HOT 7
- sherpa-onnx-alsa Failed to set number of channels to 1 or Failed to set sample rate to 16000
- ASR TTS Merge into one apk HOT 6
- Can't generate lexicon HOT 3
- TTS Highlight Current Text HOT 8
- sherpa_onnx.OnlineRecognizer( TypeError: OnlineRecognizer() takes no arguments HOT 12
- speaker-identification.py的speaker-file参数具体是指? HOT 3
- Crash After Package Name Changed HOT 7
- This library cannot reconcile with the react-native onnxuruntime official library: unresolved OrtGetApiBase symbol HOT 17
- how to solve the channel problem and bit rate problem? HOT 2
- ANR On Android HOT 3
- 这边项目也支持语音唤醒吗? HOT 12
- Cannot instantiate Java OnlineRecognizer on Windows HOT 3
- Requred File? HOT 1
- Issue with sampling rate requirement for online_websocket_client_decode_file.py HOT 2
- whisper model recognition is unstable and error
- 在SherpaOnnxKws(Android)运行过程中发现的一个问题 HOT 4
- 热词影响onnx端点识别问题 HOT 5
- 单词写错需要修改 HOT 2
- 麻烦问下endpoint识别与vad识别有什么区别吗? HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sherpa-onnx.