Whisper is skipping quotes about stable-ts

[jianfch]

What were the arguments you used? Have you tried to use beam search (e.g. beam_size=5)?

[furqan4545]

Yes i can feel the same issue. Previously it was working perfect but with new updates, it has started making issues.

[antiframes]

What were the arguments you used? Have you tried to use beam search (e.g. beam_size=5)?

I'll try with beam search and let you know

[antiframes]

What were the arguments you used? Have you tried to use beam search (e.g. beam_size=5)?

Yes, it works with beam_size=5. However, now it worked without it as well.
Maybe the issue was fixed by one of the recent updates. I'll try new examples in the future, then let you know.

At least, it worked for the original example I was using.

[antiframes]

I've found a way to reproduce the issue, it's still happening.

Issue is still happening.
The commands to replicate it with the exact same material are present in this Gist. You just need to run the cells and look for gaps in the generated SRT file.

https://gist.github.com/antiframes/fc9160ff30bb66fb8febd334da406a53#file-whisper-stable-ipynb

Most of the gaps take around one minute.
The conference is longer than 2 hours, so the bug will most probably be replicated at some point.

Although it's not a pattern, some of the gaps occur when the author is quoting a text, then making pauses to comment about it. Sometimes, the quotations are in Latin. Latin is not a problem for Whisper, sometimes the transcription is really precise. But confusion about languages may be related to the issue itself.

[jianfch]

Try model.transcribe_minimal() instead of model.transcribe(). If the issue persists, then this is likely an issue with the model itself which can only be fixed by using different model that performs better with this material (e.g. large-v3 performs worse than large-v2 in some cases so try it, or try a smaller model such as medium).

[antiframes]

That seems to be the case. Issue persists with model.transcribe_minimal() , but there's no problem when using base model.

So the issue is probably due to Whisper algorithm instead of stable-ts.

[antiframes]

There is no issue with large-v2!

I'm closing this case as solved.