tachibanayoshino / animeganv3

Use AnimeGANv3 to make your own animation works, including turning photos or videos into anime.

AnimeGANv3

Paper Title: A Novel Double-Tail Generative Adversarial Network for Fast Photo Animation.

Let's use AnimeGANv3 to produce our own animation.

📢 Updates

• 2023-12-10 Added a new AnimeGANv3 model for Portrait to Oil-painting style. Its onnx is available here.
• 2023-11-23 The code and the manuscript are released. 🦃
• 2023-10-31 Added three new styles of AnimeGANv3: Portrait to Cute, 8bit and Sketch-0 style. 👻
• 2023-09-18 Added a new AnimeGANv3 model for Face to Kpop style.
• 2023-01-16 Added a new AnimeGANv3-photo.exe for the inference of AnimeGANv3's onnx model.
• 2023-01-13 Added a new AnimeGANv3 model for Face to comic style.
• 2022-12-25 Added the tiny model (2.4MB) of Nordic myth style and USA style 2.0. It can go upto 50 FPS on iphone14 with 512*512 input. 🎅
• 2022-11-24 Added a new AnimeGANv3 model for Face to Nordic myth style. 🦃
• 2022-11-06 Added a new AnimeGANv3 model for Face to Disney style V1.0.
• 2022-10-31 Added a new AnimeGANv3 model for Face to USA cartoon and Disney style V1.0. 🎃
• 2022-10-07 The USA cartoon Style of AnimeGANv3 is integrated to ProfileProfile with Core ML. Install it by the Apple Store and have a try.
• 2022-09-26 Official online demo is integrated to Huggingface Spaces with Gradio.
• 2022-09-24 Added a new great AnimeGANv3 model for Face to USA cartoon Style.
• 2022-09-18 Update a new AnimeGANv3 model for Photo to Hayao Style.
• 2022-08-01 Added a new AnimeGANv3 onnx model (Colab) for Face to Arcane style.
• 2022-07-13 Added a new AnimeGANv3 onnx model (Colab) for Face to portrait sketch.
• 2021-12-25 The paper of AnimeGANv3 will be released in 2022. 🎄

---

🎮 Usage

• Official online demo is released in .

• Download this repository and use AnimeGANv3's UI tool and pre-trained *.onnx to turn your photos or videos into anime. 😊

• 🛠️ Installation

  1. Clone repo

     git clone https://github.com/TachibanaYoshino/AnimeGANv3.git
     cd AnimeGANv3   

  2. Install dependent packages

     pip install -r requirements.txt  

  3. Inference with *.onnx

     python deploy/test_by_onnx.py -i inputs/imgs/ -o output/results -m deploy/AnimeGANv3_Hayao_36.onnx  

  4. video to anime with *.onnx

     python tools/video2anime.py -i inputs/vid/1.mp4 -o output/results -m deploy/AnimeGANv3_Hayao_36.onnx  

🚀 Landscape Demos

🔥 Video to anime (Hayao Style)

---

🎨 Photo to Hayao Style

more surprise 👈

---

🎨 Photo to Shinkai Style

more surprise 👈

---

🚀 Portrait Style Demos

The paper has been completed in 2022. The study of portrait stylization is an extension of the paper.

Some exhibits  👈

🎨 Face to USA cartoon style

---

🎨 Face to Disney cartoon style

---

🎨 Face to USA cartoon + Disney style

more surprise 👈

---

🎨 Face to Arcane style

---

🎨 Portrait to comic style

---

🎨 Face to Kpop style

---

🎨 Portrait to Oil-painting style

more surprise 👈

---

🎨 Portrait to Cute style

---

🎨 Portrait to 8bit style

---

🎨 Portrait to Sketch-0 style

---

🎨 Face to portrait sketch

input	Face	panoramic image

more surprise 👈

🔨 Train

1. Download dataset and pretrained vgg19

1. vgg19
2. Hayao dataset
3. Shinkai dataset
4. photo dataset

2. Do edge_smooth

    cd tools && python edge_smooth.py --dataset Hayao --img_size 256

3. Do superPixel

    cd tools && python visual_superPixel_seg_image.py

4. Train

    python train.py --style_dataset Hayao --init_G_epoch 5 --epoch 100

✒️ Citation

Consider citing as below if you find this repository helpful to your project:

@article{Liu2024dtgan,
  title={A Novel Double-Tail Generative Adversarial Network for Fast Photo Animation},
  author={Gang LIU and Xin CHEN and Zhixiang GAO},
  journal={IEICE Transactions on Information and Systems},
  volume={E107.D},
  number={1},
  pages={72-82},
  year={2024},
  doi={10.1587/transinf.2023EDP7061}
}

📜 License

This repo is made freely available to academic and non-academic entities for non-commercial purposes such as academic research, teaching, scientific publications. Permission is granted to use the AnimeGANv3 given that you agree to my license terms. Regarding the request for commercial use, please contact us via email to help you obtain the authorization letter.

📧 Author

Asher Chan [email protected]

animeganv3's Issues

Where to get onnx file for AnimeGANv3_USA model?

I was trying to locate the onnx file for AnimeGANv3_USA model but could not locate it in the repo.

How many data need to train one style?

Hi, Thanks for your amazing work, I'm wondering how much data you usually need to train a style model?

Is there an onxx for paprika and shinkai style?

Im new to this and wondered if tehre was an onxx for pap and shin? Do i have to make one or something for those 2? and if so how do i go about making one?

Increase number of images processed when converting folder of images

Is it possible to increase the number of images processed?

49 is very limiting.

Why is there even a limit?

自训练模型，最后在Ckpt2pb里无法找到input:AnimeGANv3_input, 后改为 val_input完成pb2onnx, 请问这样对不?

谢谢大大的分享。

我按照readme的指引， 自己训练了模型， 最后用animegan_Ckpt2pb.py将checkpoint转换为onnx 。
脚本里input: AnimeGANv3_input始终无法找到， 后来查看所有op名字找到val_input, 用这个作为input完成了转换。
就是不知道这个input是不是正确的?

use request

I am a graduate student of Beijing University of Information Technology, I am particularly interested in your AnimeGANv3, can you provide me with your unzip password? I would be grateful!

Is the code available for research purpose?

possible to give me a license for video animation for this code?

thanks

support windows10 and amd cpu?

Hi ,
Support windows10 and amd cpu?

why USA cartoon onnx is not included?

the offered 1-3 onnx are not good

Releases?

When will, usa style onnx available?
Also where can i use new comic styles demo?

linux version

this is a feature request for linux version

thanks for your time!

[Need Info] Train ai?

Hi, so I found this awesome repo and decided to try it. But I'm currently using linux and there isn't and documentation about training the ai?. Is it possible for the collaborators to add and documentation about training?. thanks

onnxruntime failed to load library

Thanks for your great work!
There is onnxruntime error when I running the exe. Even though I can process the images, but the results seem to be not as good as hope.
Then I run "pip install onnxruntime" in the cmd window with conda base virtual environment.
But the error still exists. It's as the following:

2022-05-18 16:19:56.9664601 [E:onnxruntime:Default, provider_bridge_ort.cc:901 onnxruntime::ProviderSharedLibrary::Ensure] Failed to load library, error code: 126
2022-05-18 16:19:56.9666273 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:1624 onnxruntime::python::pybind11_init_onnxruntime_pybind11_state] Init provider bridge failed.

AnimeGANv3-photo.exe能将整个图片都转成漫画风格吗？

AnimeGANv3-photo.exe能将整个图片都转成漫画风格吗？
目前看到生成的尺寸只有头像。

How to run code with onnxruntime-gpu？

Hi @TachibanaYoshino,

Thank you for your work.
For AnimeGANv3 demo.I want to use onnxruntime-gpu to run the code, how to modify the configuration?
My email is [email protected]

Thank you @TachibanaYoshino!

请问hayao的解压 密码是多少

AttributeError: 'NoneType' object has no attribute 'shape'

When I try to make photo2anime I am getting this error. How can I solve this?

Have you tried running your code on an Ubuntu system? There are too many errors, and training is not possible.

AnimeGANv3_Trump v1.5

can anyone share AnimeGANv3_Trump v1.5 onnx cause i tested some other which are not correct animation

Could you give me demo code?

I try to use this model in browser but I do not know the tensor size and what I should do when I got a picture. I need your python demo code, I will transform to onnxjs or tensorflowjs.

can you share your pwd?

关于预训练模型

作者你好，请问可以再次发布预训练模型吗，您提供的链接已经失效，万分感谢！

Could I release the generator model reproduced from your onnx file?

I reproduced the network arch of animeganv3's generator, could I release it in my repo?
by the way, I notice that you modified the AdaIN with self-adaptive affine transform directly from 3x3_conv output features to stabilize the cartoon-style formation?

How can i get Trump style plz?

I think the Trump style is best,How can i get it or othor,plz

解压密码是什么谢谢？

解压密码是什么谢谢？

Dependencies Conflict

Which python version has you used ?? Dependencies conflict is encountered again and again. The conflict between libraries is: onnx==1.10.1
onnxruntime>=1.16.6
tensorflow-gpu==2.10.0

when will the source code release?

when will the source code release?

How to convert from anime to photo

If there is a method to convert from anime(Hayao or Shinkai) to photo , could you tell me.

how can i convert more than 49 images in single time in animeGan v3 exe

49 is the maximum number of photos that can be converted each time.
this GUI programme runs in single thread o cpu not supporting multi thread on gpu

how can i convert more than 49 images to cartoon in single time

Portrait-Style-model training dataset

Hey, great work on AnimeGANv3!
Can you briefly introduce how your Portrait-Style-model is trained? How the training dataset is generated?

When the Nordic myth style can be integrated into Huggingface Spaces

Could you please tell me when the Nordic myth style can be integrated into Huggingface Spaces, this style is really awesome, I am looking forward to it very much.

Using the photo in the example also results in the following error

GPU与onnxruntime报错问题

哥，真神人也，小弟拜服！！

有2个问题请帮忙解惑

• 不能够使用GPU。处理时候用的是cpu，这个怎么解决。
• 这个报错怎么处理。。。
  2023-04-03 11:29:55.0674790 [E:onnxruntime:Default, provider_bridge_ort.cc:901 onnxruntime::ProviderSharedLibrary::Ensure] Failed to load library, error code: 126 2023-04-03 11:29:55.0675858 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:1624 onnxruntime::python::pybind11_init_onnxruntime_pybind11_state] Init provider bridge failed. Making: videoplayback_AnimeGANv3.mp4: 100%|███| 158/158 [03:00<00:00, 1.14s/it] Total time : 181.23930430412292 s

About the .zip archive

I have tried several times for decomplie the models but unfortunately failed. I have realized that the magic head in the file "struct" is different from the modules you mentioned ending with with test_by_onnxZIP in HEX-editor, although I have fixed them, it still comes
bugs.

@TachibanaYoshino

I need AnimeGANv3 Disney Model code implementation

ONNX model files

Hi @TachibanaYoshino,

Thank you for your work.
For AnimeGANv3 demo. I do not have Windows computer to run the test. Could you share your Onnx model files. So I can test them.
My email is [email protected]

Thank you @TachibanaYoshino!

Could the images or videos generated by this software be used in commercial purpose?

as the title suggests, can I use the images or videos in commerical purpose? the original picture is produced by our team.

arcane style model

Hi, could you please share the exe file and model of arcane style? Thanks! @TachibanaYoshino

Where can I buy Disney models?

Where can I buy Disney models?

Dataset Release

Thanks for the work. Will you releasing the dataset used to train the models ?

train Code?

can you share the train code ?

sample Colab file for Video to Anime using AnimeGANv3

Hi
I cannot find Colab for Video to Anime using AnimeGANv3.
Is An inference notebook (Colab) for Video to Anime using AnimeGANv3?

请问一下，转换器能在手机上用吗？

why not using git push your code?

release via zip file is not a wise solution.

Did paper release?

Did you release the paper?
Looking forward to your reply.

Inquiry About [Portrait to Cute style] Implementation

Hello,

Firstly, I'd like to express my appreciation for the effort you've put into AnimeGANv3.

While exploring the project, I became particularly interested in how [Portrait to Cute style] is implemented.(https://github.com/TachibanaYoshino/AnimeGANv3?tab=readme-ov-file#art-portrait-to-cute-style).

I've gone through the documentation and the code but I was wondering if you could provide a bit more insight or point me to the relevant parts of the documentation/code that would help me understand the process better.

Thank you for your time and effort. Looking forward to your response!

Best regards,