k2-fsa / kaldi-decoder Goto Github PK

Hi, I have tried to use your package to decode audio files with provided pre-trained model in icefall-asr-librispeech-conformer-ctc-jit-bpe-500- but sometimes i get error and sometimes the decoding result is wrong.

This is the procedure I have followed: first, i have converted HLG.pt to HLG.fst by running
python3 convert-k2-to-openfst.py --olabels aux_labels ./lang_bpe_500/HLG.pt ./lang_bpe_500/HLG.fst

then i used to decodet test_wavs using following command:
python3 decode_with_HLG.py --nn-model ./lang_bpe_500/cpu_jit.pt --HLG ./lang_bpe_500/HLG.fst --words ./lang_bpe_500/words.txt test_wavs_1089-134686-0001.wav test_wavs_1221-135766-0001.wav test_wavs_1221-135766-0002.wav

finally, this is the output:

2024-02-14 10:25:54,563 INFO [decode_with_HLG.py:175] device: cpu
2024-02-14 10:25:54,563 INFO [decode_with_HLG.py:177] Loading torchscript model
2024-02-14 10:25:55,153 INFO [decode_with_HLG.py:182] Loading HLG from ./lang_bpe_500/HLG.fst
2024-02-14 10:25:58,930 INFO [decode_with_HLG.py:187] Constructing Fbank computer
2024-02-14 10:25:58,930 INFO [decode_with_HLG.py:198] Reading sound files: ['test_wavs_1089-134686-0001.wav', 'test_wavs_1221-135766-0001.wav', 'test_wavs_1221-135766-0002.wav']
2024-02-14 10:25:58,991 INFO [decode_with_HLG.py:204] Decoding started
2024-02-14 10:26:03,312 INFO [decode_with_HLG.py:139] test_wavs_1089-134686-0001.wav, torch.Size([165, 500])
2024-02-14 10:26:03,484 INFO [decode_with_HLG.py:139] test_wavs_1221-135766-0001.wav, torch.Size([417, 500])
2024-02-14 10:26:03,957 INFO [decode_with_HLG.py:147] failed to decode test_wavs_1221-135766-0001.wav
2024-02-14 10:26:03,959 INFO [decode_with_HLG.py:139] test_wavs_1221-135766-0002.wav, torch.Size([120, 500])
2024-02-14 10:26:04,116 INFO [decode_with_HLG.py:147] failed to decode test_wavs_1221-135766-0002.wav
2024-02-14 10:26:04,119 INFO [decode_with_HLG.py:235] 
test_wavs_1089-134686-0001.wav:
OLD SNIPER ATE WITH BE THAT NOTHING CUP P AAGE PET WERE HE THE PROUD MISSUS SPENCER GIED HOD ADAR LIFE SEN S LIFEBOAT AT THE TO AGNE AND HA THE

test_wavs_1221-135766-0001.wav:


test_wavs_1221-135766-0002.wav:



2024-02-14 10:26:04,119 INFO [decode_with_HLG.py:237] Decoding Done

What is wrong?
Thanks for your contributions.
@csukuangfj

Hi

is there a specific reason why the acoustic-scale option from kaldi decode-faster has not been exported as well?
using kaldi decoding with icefall models allows to use some desirable properties like fst memory mapping with const fst but I have not been able to get same performance (in terms of WER) than icefall 1 best decoding (I observe higher deletion, for example)
in icefall playing with the hlg-scale option seems to make same difference (like 10% relative from 0.2 to 0.8, for example) and I was wondering if I could observe the same difference by playing with the acoustic-scale option with kaldi decoder, but it is absent (it looks to me they are supposed to both play on the relative weight of graph and model scores during best path computation, isn't it?)
I actually commented here
k2-fsa/icefall#1287 (comment)
but exporting a lattice decoder seems way more complicated, if possible at all

This is a very small change that helps quite a lot I've found. The WeNet decoder has an option "length_penalty" which really helps control deletion rates which by default tend to be really high.

A good reference for the code change is in the WeNet code base. The relevant lines are linked below. Basically you add "length_penalty" (a float param with a negative value like -3.0 - so a bonus...) the following two places in ProcessEmitting() in kaldi-decoder/csrc/faster-decoder.cc and of course add the corresponding config option to kaldi-decoder/csrc/faster-decoder.h and kaldi-decoder/python/csrc/faster-decoder.cc:

runtime/core/kaldi/decoder/lattice-faster-decoder.cc#L781-L783

runtime/core/kaldi/decoder/lattice-faster-decoder.cc#L811-L813