The possibility of using small models about real-esrgan

[xinntao]

We have not tried a smaller network. You may have a try, for example, with fewer RRDB blocks.

I think the performance may have a drop, but I cannot tell how large the drop is.

[xiong-jie-y]

Hi

I also trained a SRResNet model in the Real ESRGAN framework.
It's fast (around 50fps when upscaling 256x256 image by 4x) and the appearance of the upscaled image is not bad, though it has some weakness.
Please give it a try if someone needs lightweight model.

Model
https://huggingface.co/xiongjie/realtime-SRGAN-for-anime
Example
https://github.com/xiong-jie-y/ml-examples/tree/master/realtime_srgan_anime
Hugging Face Space (Running time in hugging face space is slow)
https://huggingface.co/spaces/xiongjie/realtime-SRGAN-for-anime-example

This is the model paramater for someone who want to train the same model.

network_g:
  type: MSRResNet
  num_in_ch: 3
  num_out_ch: 3
  num_feat: 32
  num_block: 6

[xiong-jie-y]

@jorjiang
Hi

Thank you!

do you only train the models on anime images, is it possible to achieve good results with the real photo when I train on real photos

I trained the models on only anime images. I tried training another models with real images for 12 hours, but it seems more difficult than anime images because output is really smooth. 😢

How exactly did you manage to shrink the model, do you just use a smaller model and train it from scratch or did you used some techniques such as knowledge distillation

I just used a smaller model for anime model and followed a training procedure in Raal ESRGAN repository.

[Ibrahimmughal47]

I am not satisfied 😏😏😏

[xinntao]

For the anime model, we use a much smaller model, which has about 4,467,779 parameters (~25%). It can achieve similar performance. https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth

For general images, we have also tried a smaller network and has a slight performance drop.

We are also developing much smaller models with comparable performance. Will release when they are ready :-)

[zdyshine]

Thanks for the reply, I am also trying to train a small model. The data degradation of the scene I faced is not very serious, so I am trying to weaken the degradation process to train a small model. However, a suitable parameter has not yet been adjusted. Do you have any suggestions for degraded parameter modification?

[xinntao]

You can tune the hyper-parameters by looking at the generated degraded images.

I think the generated images should look like the real images as much as possible, especially the degradations.

[xiong-jie-y]

Hi

I trained a smaller RRDBNet model with these hyper parameters.

network_g:
  type: RRDBNet
  num_in_ch: 3
  num_out_ch: 3
  num_feat: 32
  num_block: 3
  num_grow_ch: 32

If someone needs it, please give it a try :D
I'm considering to improve this model more in near future.

[jorjiang]

Hi

I also trained a SRResNet model in the Real ESRGAN framework. It's fast (around 50fps when upscaling 256x256 image by 4x) and the appearance of the upscaled image is not bad, though it has some weakness. Please give it a try if someone needs lightweight model.

Model https://huggingface.co/xiongjie/realtime-SRGAN-for-anime Example https://github.com/xiong-jie-y/ml-examples/tree/master/realtime_srgan_anime Hugging Face Space (Running time in hugging face space is slow) https://huggingface.co/spaces/xiongjie/realtime-SRGAN-for-anime-example

This is the model paramater for someone who want to train the same model.

network_g:
  type: MSRResNet
  num_in_ch: 3
  num_out_ch: 3
  num_feat: 32
  num_block: 6

very cool, but I feel the result works very good on anime but overtly smooth on real photos

may I ask a few questions:

1. do you only train the models on anime images, is it possible to achieve good results with the real photo when I train on real photos
2. How exactly did you manage to shrink the model, do you just use a smaller model and train it from scratch or did you used some techniques such as knowledge distillation

[sunyclj]

Hi

I also trained a SRResNet model in the Real ESRGAN framework. It's fast (around 50fps when upscaling 256x256 image by 4x) and the appearance of the upscaled image is not bad, though it has some weakness. Please give it a try if someone needs lightweight model.

Model https://huggingface.co/xiongjie/realtime-SRGAN-for-anime Example https://github.com/xiong-jie-y/ml-examples/tree/master/realtime_srgan_anime Hugging Face Space (Running time in hugging face space is slow) https://huggingface.co/spaces/xiongjie/realtime-SRGAN-for-anime-example

This is the model paramater for someone who want to train the same model.

network_g:
  type: MSRResNet
  num_in_ch: 3
  num_out_ch: 3
  num_feat: 32
  num_block: 6

I use smaller models trained on real data, and as the number of iterations increases, the value of PSNR gets smaller and smaller.The results of the tests were also poor.