AssertionError: Got inconsistent logprobs length : 23 != 22 about whisper-timestamped

[Jeronymous]

OK thanks for your help. This should be fixed now

[MohammedMehdiTBER]

OK, sorry for the inconvenience. I pushed a new version "1.7.0" that solves several issues including one related to the assertion failure you saw.

Can you please try?

Note: this version use different default options, so it is longer to run, and the transcription results should be better. If you were OK with the accuracy of the transcription, and want to run faster, you should use option --efficient.

Tried on 1.7.1. It worked via Google Colab now. I did not know about--efficient, so I did not use it, but I would like if the subtitles are more precise and do not fade before the speech ends.

[Jeronymous]

Can you check with the last version please?

Also can you share the file and tell me the full set of options for me to reproduce?

[MohammedMehdiTBER]

Can you check with the last version please?

Also can you share the file and tell me the full set of options for me to reproduce?

Same error, sir
On this video, you can replicate:
https://www.youtube.com/watch?v=zVYaS_qXM3Y

[Jeronymous]

Thanks 👍
But it seems that you're not taking the full video.
Just the first 2 minutes or something like this?

[MohammedMehdiTBER]

No I have just given you the first video I had the issue with. Not all tracks suffer from it for some reason I am not sure, maybe the subtitles are long or something like that?

…

On Thu, Jan 19, 2023, 9:41 PM Jérôme Louradour ***@***.***> wrote: Thanks 👍 But it seems that you're not taking the full video. Just the first 2 minutes or something like this? — Reply to this email directly, view it on GitHub <#12 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASU2BOHBUFSQ2FHJ3PYMEODWTGRH7ANCNFSM6AAAAAAUAUIAWA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[MohammedMehdiTBER]

OK thanks for your help. This should be fixed now

Affirmative, the alignment is currently operational. My gratitude is extended to you, Sir. However, it has been noted that the sentences within the command line interface outputs are not properly aligned until the output file is opened. It is suggested that this issue be addressed to prevent confusion for users.

[MohammedMehdiTBER]

OK thanks for your help. This should be fixed now

I see that the starting time is good for songs but when longing the voice the subtitle does not keep with until It stops.

[MohammedMehdiTBER]

OK thanks for your help. This should be fixed now

The error re happened when I used a very large audio 33min:
[32:48.000 --> 32:53.000] في كل منواضع

Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Program Files\Python310\Scripts\whisper_timestamped.exe\__main__.py", line 7, in <module>
  File "C:\Program Files\Python310\lib\site-packages\whisper_timestamped\transcribe.py", line 1085, in cli
    result = transcribe_timestamped(
  File "C:\Program Files\Python310\lib\site-packages\whisper_timestamped\transcribe.py", line 524, in transcribe_timestamped
    assert abs(segment["avg_logprob"] - avglogprob) < 1e-2, f"Fatal Error: Got inconsistent logprob for segment {i}: {avglogprob} != {segment['avg_logprob']}"
AssertionError: Fatal Error: Got inconsistent logprob for segment 543: -0.34706705808639526 != -0.2806609970571749

[Jeronymous]

This is not the same error.
I fixed something related to this yesterday: see issue #19
Do you have a version higher or equal to 1.6.5?

[MohammedMehdiTBER]

This is not the same error. I fixed something related to this yesterday: see issue #19 Do you have a version higher or equal to 1.6.5?

No, I have lower

[Jeronymous]

Then upgrade and it should be fixed

[MohammedMehdiTBER]

This is not the same error. I fixed something related to this yesterday: see issue #19 Do you have a version higher or equal to 1.6.5?

Now It did not produce anything:

PS C:\Users\MohammedMehdiTBER> whisper_timestamped "C:\Users\MohammedMehdiTBER\Downloads\نقد أغنية Begin Again Purity Ring.mp3" --model large-v2 --output_dir "C:\Users\MohammedMehdiTBER\Documents" --language Arabic --device cpu --threads 4 --verbose True
C:\Users\MohammedMehdiTBER\AppData\Roaming\Python\Python310\site-packages\numpy\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:
C:\Users\MohammedMehdiTBER\AppData\Roaming\Python\Python310\site-packages\numpy\.libs\libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll
C:\Users\MohammedMehdiTBER\AppData\Roaming\Python\Python310\site-packages\numpy\.libs\libopenblas64__v0.3.21-gcc_10_3_0.dll
  warnings.warn("loaded more than 1 DLL from .libs:"
2023-01-25 10:11:42.931828: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2023-01-25 10:11:42.933690: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Program Files\Python310\Scripts\whisper_timestamped.exe\__main__.py", line 7, in <module>
  File "C:\Users\MohammedMehdiTBER\AppData\Roaming\Python\Python310\site-packages\whisper_timestamped\transcribe.py", line 1368, in cli
    result = transcribe_timestamped(
  File "C:\Users\MohammedMehdiTBER\AppData\Roaming\Python\Python310\site-packages\whisper_timestamped\transcribe.py", line 210, in transcribe_timestamped
    (transcription, words) = _transcribe_timestamped_efficient(model, audio, **alignment_options, **whisper_options, **other_options)
  File "C:\Users\MohammedMehdiTBER\AppData\Roaming\Python\Python310\site-packages\whisper_timestamped\transcribe.py", line 618, in _transcribe_timestamped_efficient
    assert abs(segment["avg_logprob"] - avglogprob) < 1e-2, f"Fatal Error: Got inconsistent logprob for segment {i}: {avglogprob} != {segment['avg_logprob']}"
AssertionError: Fatal Error: Got inconsistent logprob for segment 543: -0.34706705808639526 != -0.2806609970571749

[Jeronymous]

What do you mean "It did not produce anything"?
It's failing with the exact same error as before.

What gives pip show whisper-timestamped now?
whisper-timestamped -v?
Did you upgrade to a newer version ? (for instance by running pip install --upgrade --no-deps --force-reinstall git+https://github.com/linto-ai/whisper-timestamped)

[MohammedMehdiTBER]

What do you mean "It did not produce anything"? It's failing with the exact same error as before.

What gives pip show whisper-timestamped now? whisper-timestamped -v? Did you upgrade to a newer version ? (for instance by running pip install --upgrade --no-deps --force-reinstall git+https://github.com/linto-ai/whisper-timestamped)

PS C:\Users\MohammedMehdiTBER> pip show whisper-timestamped
Name: whisper-timestamped
Version: 1.6.7
Summary: Add to OpenAI Whisper the capability to give word timestamps
Home-page: https://github.com/Jeronymous/whisper-timestamped
Author: Jeronymous
Author-email:
License: MIT
Location: c:\users\mohammedmehditber\appdata\roaming\python\python310\site-packages
Requires: Cython, dtw-python, openai-whisper
Required-by:
PS C:\Users\MohammedMehdiTBER>

I mean no file, no transcription output in the CLI. Only retuned that error. But I did not use --force-reinstall --no-deps to upgrade but:

pip3 install git+https://github.com/linto-ai/whisper-timestamped --isolated --ignore-installed -U --upgrade

[MohammedMehdiTBER]

Same error even with using python env:

PS C:\Users\MohammedMehdiTBER\whisper-timestamped> pipenv --python "C:\Program Files\Python310\python.exe" install git+https://github.com/linto-ai/whisper-timestamped#egg=whisper-timestamped
Installing git+https://github.com/linto-ai/whisper-timestamped#egg=whisper-timestamped...
Pipfile.lock (5a35e3) out of date, updating to (30cd9b)...
Locking [packages] dependencies...
Locking [dev-packages] dependencies...
Updated Pipfile.lock (bcad4f480615786d5960b2ede41d5e61cddf802ff86a1fbbd9513f6d9b30cd9b)!
Installing dependencies from Pipfile.lock (30cd9b)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
PS C:\Users\MohammedMehdiTBER\whisper-timestamped> pipenv run
Usage: pipenv run [OPTIONS] COMMAND [ARGS]...
Try 'pipenv run -h' for help.

Error: Missing argument 'COMMAND'.
PS C:\Users\MohammedMehdiTBER\whisper-timestamped> pipenv shell
Launching subshell in virtual environment...
PowerShell 7.3.1
PS C:\Users\MohammedMehdiTBER> pipenv run -h
Usage: pipenv run [OPTIONS] COMMAND [ARGS]...

  Spawns a command installed into the virtualenv.

Options:
  --python TEXT       Specify which version of Python virtualenv should use.
  --three             Use Python 3 when creating virtualenv. Deprecated
  --clear             Clears caches (pipenv, pip).  [env var: PIPENV_CLEAR]
  -q, --quiet         Quiet mode.
  -v, --verbose       Verbose mode.
  --pypi-mirror TEXT  Specify a PyPI mirror.
  -h, --help          Show this message and exit.
PS C:\Users\MohammedMehdiTBER> whisper_timestamped "C:\Users\MohammedMehdiTBER\Downloads\نقد أغنية Begin Again Purity Ring.mp3" --model large-v2 --output_dir "C:\Users\MohammedMehdiTBER\Documents" --language Arabic --device cpu --threads 4 --verbose True
Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\MohammedMehdiTBER\.virtualenvs\MohammedMehdiTBER-xIZJGCn3-python.exe\Scripts\whisper_timestamped.exe\__main__.py", line 7, in <module>
  File "C:\Users\MohammedMehdiTBER\.virtualenvs\MohammedMehdiTBER-xIZJGCn3-python.exe\lib\site-packages\whisper_timestamped\transcribe.py", line 1379, in cli
    result = transcribe_timestamped(
  File "C:\Users\MohammedMehdiTBER\.virtualenvs\MohammedMehdiTBER-xIZJGCn3-python.exe\lib\site-packages\whisper_timestamped\transcribe.py", line 211, in transcribe_timestamped
    (transcription, words) = _transcribe_timestamped_efficient(model, audio, **alignment_options, **whisper_options, **other_options)
  File "C:\Users\MohammedMehdiTBER\.virtualenvs\MohammedMehdiTBER-xIZJGCn3-python.exe\lib\site-packages\whisper_timestamped\transcribe.py", line 627, in _transcribe_timestamped_efficient
    assert abs(segment["avg_logprob"] - avglogprob) < 1e-2, f"Fatal Error: Got inconsistent logprob for segment {i}: {avglogprob} != {segment['avg_logprob']}"
AssertionError: Fatal Error: Got inconsistent logprob for segment 543: -0.34706705808639526 != -0.2806609970571749

[Jeronymous]

OK, sorry for the inconvenience.
I pushed a new version "1.7.0" that solves several issues including one related to the assertion failure you saw.

Can you please try?

Note: this version use different default options, so it is longer to run, and the transcription results should be better.
If you were OK with the accuracy of the transcription, and want to run faster, you should use option --efficient.