Comments (10)
Thanks both !
I figured out how to support shared models and safetensors format.
@BlahBlah314 You should be able to use your "BlahBlah314/whisper_LargeV3FR_ft-V1" with the new version (1.14.4)
from whisper-timestamped.
Can you give some code to reproduce?
from whisper-timestamped.
from transformers import WhisperForConditionalGeneration
model=WhisperForConditionalGeneration.from_pretrained("your_model_dir")
model.save_pretrained("your_model_dir", safe_serialization=False, max_shard_size= '10GB')
from whisper-timestamped.
What is "your_model_dir" in your case?
You mentioned "finetuned whisper model on hugging face", I thought you were using a model that is on Hugging Face.
Also there is no whisper_timestamped in your code.
Is it possible to have a code to reproduce how you load the model in whisper_timestamped? (the code that throws an error related to a missing "pytorch_model.bin" file I guess)
from whisper-timestamped.
I think I found a way to reproduce:
import shutil
from transformers import WhisperForConditionalGeneration
import whisper_timestamped as whisper
audio_file = "XXX.wav" # use an audio file here
shutil.rmtree("tmp_model", ignore_errors=True)
model = WhisperForConditionalGeneration.from_pretrained("qanastek/whisper-tiny-french-cased")
model.save_pretrained("tmp_model", safe_serialization=False, max_shard_size= '100MB')
model = whisper.load_model("qanastek/whisper-tiny-french-cased")
expected = whisper.transcribe(model, audio_file)
model = whisper.load_model("tmp_model")
output = whisper.transcribe(model, audio_file)
assert expected == output
The second loading of the model fails (whisper.load_model("tmp_model")
).
It happens because the model is sharded. I will investigate that.
@LaurinmyReha Can you just check that your folder "your_model_dir" contains something similar to this:
config.json generation_config.json pytorch_model-00001-of-00003.bin pytorch_model-00002-of-00003.bin pytorch_model-00003-of-00003.bin pytorch_model.bin.index.json
from whisper-timestamped.
exactly. Increase the max_shard_size to something larger and you should be fine i think. 10GB as in the example will definately be enough :)
model.save_pretrained("your_model_dir", safe_serialization=False, max_shard_size= '10GB')
from whisper-timestamped.
Can you give some code to reproduce?
Hi !
Here is my code:
import whisper_timestamped as whisper
audio = whisper.load_audio("4d8b691a-7529-47d1-a3e3-00ce32f430c2.wav")
model = whisper.load_model("BlahBlah314/whisper_LargeV3FR_ft-V1", device="cuda")
result = whisper.transcribe(model, audio, language="fr")
I must specify that I don't have a .bin model, but a safetensor. Therefore, maybe I need to convert somehow my safetensors model ? Or a config to specify on whisper timestamped ?
from whisper-timestamped.
from transformers import WhisperForConditionalGeneration
model=WhisperForConditionalGeneration.from_pretrained("BlahBlah314/whisper_LargeV3FR_ft-V1")
model.save_pretrained("path_to_where_you_want_to_safe_your_bin", safe_serialization=False, max_shard_size= '10GB')
exactly. I think this should do the conversion.... there is probably a better way but this seemed easiest to me. After that add the .bin file to where your safetensor file resides ( in your case the huggingface hub) and you should be able to load the model :)
from whisper-timestamped.
Thank you ! I'll try that way
from whisper-timestamped.
very cool!! thanks for the quick update!
from whisper-timestamped.
Related Issues (20)
- How to add new _ALIGNMENT_HEADS? HOT 4
- whisper_timestamped doesn't work from an URL in CLI
- whisper_timestamped blocks from an URL in CLI into subprocess module HOT 6
- Error with Whisper v3 HOT 2
- everytime I update this, it bricks my python install HOT 1
- Beam Search Decoding How to Get Beam of Tokens as Output HOT 3
- Error when using -vad_v3.1 HOT 1
- Consider using whisper-distilled HOT 2
- Publication on Pypi failing HOT 7
- Is there a way to use it with whisper.cpp HOT 2
- Cannot find audio file HOT 3
- Only part of audio transcribed HOT 4
- Trouble transcribing list of files HOT 2
- torch hub path is not properly set HOT 1
- Broken link for plotting word alignment section HOT 7
- How to activate flash attention? HOT 2
- Could it be possible to apply the same technique to the whisper API? HOT 6
- ctranslate2 support HOT 1
- CPU only light install links are broken? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from whisper-timestamped.