Hallucination loop about whisper-standalone-win

[JDTR75]

It worked with -prompt=auto!
I know it's hard if there's no sample 😓. Sorry about that.

[Purfview]

It affects only srt and vtt files.
You can delete all those "tests".

[Purfview]

Can you you share a sample with the issue?

[JDTR75]

Unfortunately I can't (NDA). Seems like it's only happening with this particular audio. But still got me curious as to why only large-v2 was having this issue. All the other models worked just fine. You can safely close this 🙂.

[Purfview]

Try -prompt=None.
Or -prompt=auto, this works only if --language is specified.

But still got me curious as to why only large-v2 was having this issue.

Some hallucination loop, could be some bug, dunno without a sample.

[Purfview]

@JDTR75
I've just remembered one bug from vanilla Whisper, I wonder if it's triggering this hallucination for you.

Bugfix is incorporated in new r167.3 release.

Try it on that file with the same settings when you was getting that hallucination loop.

[JDTR75]

Hello! It's still hallucinating with r167.3 if I don't use -prompt.
Let me use this to ask you about this as well - maybe it's not the the right thread, lmk-. I'm seeing this on the console all the time: Could not load preprocessor_config.json: Expecting value: line 1 column 1 (char 0). Might be affecting something?

[Purfview]

It's still hallucinating

Could you run it with --verbose=true and share the output directly from the console?
Transcription of the few minutes.

Let me use this to ask you about this as well - maybe it's not the the right thread, lmk-. I'm seeing this on the console all the time: Could not load preprocessor_config.json: Expecting value: line 1 column 1 (char 0). Might be affecting something?

This doesn't look right. With what commands you are getting it?

[JDTR75]

So the weirdest (or maybe not) thing happened. I updated SE to the latest beta before reading this. It's now running large-v2 but it omitted part of the first seconds of the audio even though the VAD clearly detects audio there.

I'm running it like this from SE, nothing special at the moment: whisper-faster.exe --language es --model "large-v2" "D:\TEMP\7e7359a6-df52-4fe1-a828-e4cfb1210154.wav"

whisper_log.txt

[Purfview]

So the weirdest (or maybe not) thing happened. I updated SE to the latest beta before reading this. It's now running large-v2 but it omitted part of the first seconds of the audio

You didn't said that you are running it in SE...
If you want to see what is going on then you need to run it directly.
Do you mean that it stopped hallucinating after you updated SE?

whisper_log.txt

This is SE's log, it's unreadable. I meant the output directly from the whisper-faster.exe.

[JDTR75]

Here you go... ran directly from CLI.

purfview.txt

[Purfview]

Was you doing anything with the models, like copying files manually?

Try to delete large-v2 folder and get model re-downloaded.

[JDTR75]

Nope, not doing anything with the models - not savvy enough to be messing around with that. I usually download them from SE's list.

Followed your lead and deleted and re-downloaded the model directly from CLI. Here you go, nothing changed.

purfview.txt

[Purfview]

Followed your lead and deleted and re-downloaded the model directly from CLI. Here you go, nothing changed.

It changed, there is no more: Could not load preprocessor_config.json: Expecting value: line 1 column 1 (char 0).
I wonder what that was about, did you downloaded that model manually from SE links or SE downloaded it automatically?

Try -prompt=default2.

[JDTR75]

You're right about the preprocessor message, that changed. On SE you just use the GUI.

I'll be back shortly with the results for -prompt=default2.

[Purfview]

Maybe there is some bug with SE's autodownload, try to delete the model again and download it now with SE.

[JDTR75]

Downloading the model from SE.

purfview.txt

Results with -prompt=default2

purfview_prompt_default2.txt

The different paths are because I used a separate, clean instance of SE and downloaded the large-v2 from there - same SE beta version to not mess with the tests.

[Purfview]

...the model from SE.

Hmm, that message there again, must be some bug in SE, reported it there -> SubtitleEdit/subtitleedit#7813

purfview_prompt_default2.txt

Looks like default prompt is triggering that hallucination loop, "default2" was previous default, maybe I need to swap it back.
Or wait, I'll try some other thing.

[Purfview]

OK then... I'll make "default2" back as default.

I'll cleanup all this spam. Btw for you GPU -ct=float16 should be fastest.

[JDTR75]

Happy to help! Thanks for the tip on the compute type 👍

[Purfview]

I noticed there are lots of prompt resets in "purfview_prompt_default2.txt" because of - * Reset prompt. prompt_reset_on_no_end is triggered.

There can be discontinuations in transcription between chunks/windows on prompt reset.

Could you share one more test with -prompt=auto?

[JDTR75]

test4_prompt_auto.txt

[Purfview]

Thx. There "prompt_reset_on_no_end" was triggered only once. I wonder if "auto" is better than "default2".

If you have feedback on that then post there -> #142

Btw, you can output srt files formatted by --sentence or --standard. "--sentence" makes easier to spot punctuation.

[JDTR75]

You're welcome! I use --sentence for everything, actually. I didn't use it in these tests to leave the arguments as clean as possible. I'll make sure to post about it on the right thread.
So should I stay with this build or will you upload something new? Thanks again!