liuziyang365 / transformercap

A vanilla transformer model for image captioning

Readme

---

Prepare data(only COCO)

• Text: (Download）
• Image
  • Raw images(Download)
  • Buttom-up Attention(Download)
• Tools
  • Coco eval coco-caption(pull it in this repository)

---

Process data

• For text

  python scripts/prepro_text.py

  you will obtain four files, train.json, val.json, test.json, vacal.pkl

  train.json is a dictionary with sentence id as key

  val.json and test.json both are dictionaries with image id as key

  vocal.pkl is an example of vocabulary class

• For Buttom-up Attention

  python scripts/prepro_bu.py --num_box fixed(Only use fixed 36 features)

---

Training

python train.py --id baseline

---

Evaluation

python eval.py --id baseline

---

Acknowledgenments

Thanks to a captioning base code repository self-critical.pytorch and a lot of codes refer to it.