kennyhuangml100 / scenimefy

[ICCV 2023] Scenimefy: Learning to Craft Anime Scene via Semi-Supervised Image-to-Image Translation

Scenimefy: Learning to Craft Anime Scene via Semi-Supervised Image-to-Image Translation

Yuxin Jiang^*, Liming Jiang^*, Shuai Yang, Chen Change Loy

MMLab@NTU affiliated with S-Lab, Nanyang Technological University

In ICCV 2023.

📃Paper | 🌐Project Page | 📂Anime Scene Dataset | 🤗Demo

---

Updates

• [08/2023] Integrated to Hugging Face. Enjoy the web demo!
• [08/2023] Inference code and Dataset is released.
• [08/2023] Project page is built.
• [07/2023] The paper is accepted to ICCV 2023!

---

🔧 Installation

1. Clone this repo:

   git clone https://github.com/Yuxinn-J/Scenimefy.git
   cd Scenimefy

2. Install dependent packages: after installing Anaconda, you can create a new Conda environment using conda env create -f Semi_translation/environment.yml.

⚡ Quick Inference

1. Python script 2. Gradio demo

Python script

• Download pre-trained models: Shinkai_net_G.pth

  wget https://github.com/Yuxinn-J/Scenimefy/releases/download/v0.1.0/Shinkai_net_G.pth -P Semi_translation/pretrained_models/shinkai-test/

• Inference! Simply run the following command, or refer the ./Semi_translation/script/test.sh for detailed usage:

  cd Semi_translation
  
  python test.py --name shinkai-test --CUT_mode CUT  --model cut --phase test --epoch Shinkai --preprocess none

  • Results are in the ./Semi_translation/results/shinkai-test/ by default.
  • To prepare your own test images, you can refer to the data folder structure in ./Semi_translation/datasets/Sample, and place your test images in testA.

Gradio demo

• We provide a UI for testing Scenimefy, which is built with gradio. To launch the demo, simply execute the following command in your terminal:

  git clone https://huggingface.co/spaces/YuxinJ/Scenimefy
  pip install -r requirements.txt
  pip install gradio
  python app.py
  

• This demo is also hosted on Hugging Face🤗.

📂 Anime Scene Dataset

It is a high-quality anime scene dataset comprising 5,958 images with the following features:

• High-resolution (1080×1080)
• Shinkai-style (from 9 Mokota Shinkai films)
• Pure anime scene: manual dataset curation by eliminating irrelevant and low-quality images

In compliance with copyright regulations, we cannot directly release the anime images. However, you can conveniently prepare the dataset following instructions here.

🤟 Citation

If you find this work useful for your research, please consider citing our paper:

@inproceedings{jiang2023scenimefy,
  title={Scenimefy: Learning to Craft Anime Scene via Semi-Supervised Image-to-Image Translation},
  author={Jiang, Yuxin and Jiang, Liming and Yang, Shuai and Loy, Chen Change},
  booktitle={ICCV},
  year={2023}
}

🤗 Acknowledgments

Our code is mainly developed based on Cartoon-StyleGAN and Hneg_SRC. We thank facebook for their contribution of Mask2Former.

🗞️ License

Distributed under the S-Lab License. See LICENSE.md for more information.