BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
To install the dependencies, run
pip install -r requirements.txt
In addition, install torch torchaudio
and
!git clone https://github.com/WangHelin1997/s3prl
!cd s3prl
!pip install -e "[.all]"
- Prepare Hugginface-like audio dataset.
- In configs/pretrain.yaml, set up to your own path.
- Train example:
sh pretrain.sh
- Train example:
sh train_caption.sh
- Train example:
sh train_retrieval.sh