Comments (4)
For sure! Thanks for reporting it! I'll have a look to see if there's a way we can avoid this in the future (potentially being more explicit in the text?)
from community-events.
Hey @BirgerMoell, sorry to hear that you lost some files 😥
This behaviour looks to be a combination of a couple of things:
- The
output_dir
is set to./
(i.e. the directory in which the scriptrun.sh
is saved). The README instructions suggest that you run the script from within a new model repository cloned from the Hugging Face Hub (i.e. set your current working dir to a cloned model repo). This means that the only files that can be touched by the script are those within the model repository, and any files outside the repo are safe! The Trainer is sandbox in this regard, it only has access to the files you specify in theoutput_dir
. If you don't want Trainer to touch files in your current working dir, you need to run the script from a different dir or setoutput_dir
accordingly. - This is the desired behaviour by setting
overwrite_output_dir
(see docs). I'm afraid we can't change this as it's required for over-riding checkpoints and flushing dirs.
So what you can do is run the script from within a new model repo or set --overwrite_output_dir=False \
in the arguments to run.sh
. We're trying to give instructions that can be used by as many participants as possible. They've been stress tested when all of the instructions are run sequentially, but the behaviour will no doubt be different when different steps are run. Sorry to hear that you lost files. If this was a model repo cloned from the HF Hub could you revert to the last commit?
from community-events.
I got most of the stuff back through git but I seemed a bit unexpected so I just wanted to report it to make sure it behaved as it was intendent.
from community-events.
(It will only do a full wipe if the output dir is not a HF repo:
https://github.com/huggingface/transformers/blob/6a707cf586f37865d1a9fac02c56c43cf8dcf979/src/transformers/trainer.py#L3326)
from community-events.
Related Issues (20)
- How to use the resulting whisper checkpoints when finetuning HOT 9
- Fine-tuned Whisper models perform worse than OpenAI HOT 8
- Super large number of epoch HOT 1
- Using finetuned whisper checkpoints for inference HOT 1
- Whisper parameters HOT 1
- Add Keras Dreambooth notebook to this repo
- Lambda platform doesn't support Tensorflow-Gpu HOT 1
- Visualize ControlNet results using Weights & Biases Tables HOT 11
- WhisperPositionalEmbedding HOT 1
- this is more question than an issue HOT 2
- Padding conflict in loss computation HOT 2
- Whisper finetune HOT 1
- common_voice map error in the notebook of fine-tune-whisper-non-streaming HOT 1
- huggan.pytorch.lightweight_gan.lightweight_gan.LightweightGAN _from_pretrained requires use_auth_token but this is not passed by the from_pretrained method inherited from ModelHubMixin HOT 5
- Increasing WER & Validation Loss During Whisper Fine-Tuning HOT 1
- Poor Real-Time Performance of Whisper Models Fine-Tuned on Synthetic Data HOT 1
- Colab runtime crash HOT 2
- How to prepare audio dataset for whisper fine-tuning with timestamps?
- The fine tuning script run_speech_recognition_seq2seq_streaming.py use interleave_datasets which will truncate the train split HOT 2
- Using rename_column and remove_column method for a IterableDataset object leads to its feature property become None --- in the Whisper Fine-Tuning Event HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from community-events.