cszn / bsrgan

Designing a Practical Degradation Model for Deep Blind Image Super-Resolution (ICCV, 2021) (PyTorch) - We released the training code!

Kai Zhang

[Homepage] [Google Scholar] [ResearchGate] [知乎]

I am currently a postdoctoral researcher at Computer Vision Lab, ETH Zurich, Switzerland.

⚡ News:

Our new work SCUNet for practical image denoising.

⚡ News

We released the testing code of BSRGAN!

🌱 My Repositories

SCUNet	BSRGAN	KAIR	USRNet	DPIR	IRCNN	SRMD	DPSR	DnCNN	FFDNet

The pytorch training and testing codes of SRMD, DnCNN, FFDNet can be found in KAIR.

bsrgan's Issues

关于模型训练的一些疑问

大佬，有几个问题想请教下：

1. 如果从头训练，需要准备多少张高清训练样本？（数据主要为电商平台的商品主图）
2. BSRNET和BSRGAN分别需要训练多少个epoch/iter，训练时长大概是多久？
3. 为什么要先训练BSRNET再训练BSRGAN？训练BSRGAN的时候只需要导入训好的BSRNET的XXX_G.pth吗？
   麻烦大佬得空的时候解答下，万分感谢！

when use your degrade algorithm ，I get some images like this ，I wanna konw if it is correct？

Paper

would you mind sharing the paper?

Replication of BSRGAN results

Dear authors,

I am more than interested in your work and would very much appreciate if you answer the following questions:

• Have you used the blindsr.degradation_bsrgan script to generate test dataset? I inferenced BSRNet and RRDB from type-I to type-3 (of proposed DIV2K4D) and achieved almost the same results as reported in the paper. However, results on type-4 (generated by abovementioned script) led to significantly worse results: BSRNet 24.40 +- 0.14 dB (over 10 realizations) and RRDB: 22.50 +- 0.14 dB. Metrics are calculated in Y channel.
• Do you use the same script to generate a training dataset? Also, are the training parameters (learning rate, patch shape (72x72), num. of iterations, etc.) of BSRNet the same as BSRGAN?

Sincerely,
Magauiya

Model for x2 BSRNet

Thank you for sharing the code and model.

Could you please share the trained model of x2 BSRNet? Thanks

简化模型

您好，您的网络模型很好，可以直接使用，想请问您，，是否有量化或者简化后的模型，可以在一般pc机上运行？谢谢

图片加jpeg_noise后输出是全黑的

导入add_jpeg_noise函数后，输出的图片是全黑的，怎么回事？

Your Results in New Super-Resolution Benchmarks

Hello,

MSU Graphics & Media Lab Video Group has recently launched two new Super-Resolution Benchmarks.

• Video Upscalers Benchmark: Quality Enhancement determines the best upscaling methods for increasing video resolution and improving visual quality.
• Super-Resolution for Video Compression benchmark aims to test Super-Resolution methods on compressed videos and select the best model for each video codec standard.

Your method achieved 3rd place in Video Upscalers Benchmark: Quality Enhancement in 'Camera 4x' category. We congratulate you on your result and look forward to your future work!

We would be grateful for your feedback on our work.

when you training, if the patch_size 72*72 in the paper means randomcrop 72*72 from lr_image

the patch_size 7272 in the paper means randomcrop 7272 from lr_image ,and make it as input of generator ，and hr_img should be 288 *288 corrspond to randomcrop's lr_img , am I right?

训练参数iter设置问题

请问您训练的x2超分模型硬件算力是怎样的？iter为多少停止训练？

Denoising performance of the model

The model can be image denoised while SR, but in some scenarios, details are lost,has this happened during your test?for example
original image

SR image

Tile texture features missing.

mistake solved

when will you release data processing code?

Hi, You did a good job！
I want to study the specific process, please tell me when you will release data processing code.
Thank you.

How to fix CUDA out of memory error?

Is there any way to define patch size in main_test_bsrgan.py?

请问LPIPS指标如何计算？

在测试代码以及训练代码KAIA中，暂时没有找到该指标的计算方式，请问您是如何计算的？代码中有计算LPIPS的方法吗？

when the train code release?

the test result is really good , when the train code release?

Trained BSRGAN and got blank images

I followed the instructions in https://github.com/cszn/BSRGAN#training and trained DIV2K dataset.
However after a day of training all test images were just uniform patches

Output:

Anyone successfully trained BSRGAN using the code in this repo?
Thanks

Dataset and Degradation Model

Great work! Is there any plan releasing the DIV2k3D dataset or degradation model in the paper?
Tks!

Any code for testing?

Hi Cszn! Any code for testing?

Run on mobile devices

Thanks for the interesting work. I like to run it on a mobile device in android. Is it possible to do that? I guess we need a .tflite model file?

pretrained_netE 和pretrained_netD的关系

请问pretrained_netE 和pretrained_netD是同一个东西吗？ 我填写如下 是否有问题
"root": "superresolution" // "denoising" | "superresolution"
, "pretrained_netG": "bsrgan_x4_gan/models/BSRNet.pth" // path of pretrained model
, "pretrained_netD": "bsrgan_x4_gan/models/vgg19-dcbb9e9d.pth" // path of pretrained model
, "pretrained_netE": "bsrgan_x4_gan/models/vgg19-dcbb9e9d.pth" // path of pretrained model

What does 'axis length' mean?

As section 3.1 of your paper said, the length of each axis for scale factors 2 and 4 uniformly from [0.5, 6] and [0.5, 8], respectively. However, I am not sure what does that mean. In my understanding, gaussian kernel needs only 1. kernel size, 2. eigen-vals, and 3. rotation angle.

I will appreciate it if you can reply at any time you are available.
Thank you!

What do I have to type in in order to test my own images?

Can somebody give me the command?

Can you share 2x model in another form?

Is it possible for you to post the 2x pretrained model in the architecture of your 4x model? Other than that thanks for this great work. Are you thinking about improving the face restoration quality in later projects?

Also I was wondering if you have any experimental models you would like to share for bsrgan without any denoising but with the ability of face restoration,motion blur removal etc? I mean bsrgan model but just without denoise

i can't find path

Hi. I want to train this model with my own data. In one of the steps, it says, "Put BSRNet model (e.g., '400000_G.pth') into superresolution/bsrgan_x4_gan/models." Where are BSRNet model and superresolution/bsrgan_x4_gan located? and How can I do this step?

How can I get ISP model?

genius work！By the way,how can i get ISP model for your degrade model?

Possible bug in degradation_bsrgan

It is possible that dimensions are permutated in the cropping operation in degradation_bsrgan_plus, degradation_bsrgan functions in 541, 450 lines.

It should be
img = img.copy()[:h1 - h1 % sf, :w1 - w1 % sf, ...] # mod crop
instead of
img = img.copy()[:w1 - w1 % sf, :h1 - h1 % sf, ...] # mod crop

ISP degradation code

Thank you release the degradation model code, will ISP degradation code be published in the future?

请问下，训练BSRGAN的时候，是需要先训练BSRNet，然后再训练BSRGAN吗？

Question about Gaussain Noise

Hi, thank you for your work.
I have a question about the degradation. For the Gaussian noise, in the general case, why you only set L=noise_level2/255.0 (https://github.com/cszn/BSRGAN/blob/main/utils/utils_blindsr.py#L371) instead of generating a random value between noise_level1 and noise_level2, like for the grayscale and color Gaussian noises?

Thanks!

如何制作自己的训练集

from utils import utils_blindsr as blindsr
img_lq, img_hq = blindsr.degradation_bsrgan(img, sf=4, lq_patchsize=72)

但是报错，这这这咋办
h1, w1 = img.shape[:2]
AttributeError: 'str' object has no attribute 'shape'

3x3 and 5x5 Gaussian kernel sizes?

Hi @cszn,

Great work on the paper, the results are really fascinating.

Is there any reason why you haven't included 3x3 and 5x5 Gaussian blur kernels? They seem to be pretty standard so I was wondering why they aren't included.

My line of reasoning is that including them could expand the augmentation space and thus help with generalization.

License Information

Hi. Could you add license details

0.2 multiplier in RRDB output

I only wanted to know why do you multiply times 0.2 the output of the redo's single blocks and entire block.

What is the reasoning? the advantages? How did you arrived at this solution?

大佬，求教 训练代码

大佬，请问啥时候才会看源训练代码，感觉测试效果非常棒。

How can I get ISP model for your degrade model?

UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`. Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)

python main_train_psnr.py --opt options/train_bsrgan_x4_psnr.json 运行至model.update_learning_rate(current_step)报上面错误；pytorch版本为1.7.1

[Feature Request] Any planning/progress on publishing QualityScaler for Linux?

I like this tool from the looks and general preview a lot and could not find a similar tool for Linux on itch.io so I wanted to ask if there is a progress or starting for Linux? I just own Linux Arch and don't use windows anymore sadly and I don't know how I could upscale images but I want to, I'd like to support the project at patreon or through another subscription because I think there should be more tools like this out there.
Not really an issue but a feature request or question, delete if unwanted

About PSNR value in papers

Hi,
Great works! Is the PSNR from table 1 in papers calculated in RGB space or on the luminance channel in YCbCr color space?

Training details

Hi, great work the results look really good. I had few questions regarding training

1. How long did you train the BSRNet?
2. Did you apply the proposed data degradation pipeline on the fly as a data-augmentation technique or you applied it on images beforehand and stored them?

训练过程的问题

您好，请问为什么先训练BSRNet，再训练BSRGAN呢？请问您的训练数据量、参数以及算力资源是怎样的？训练多久？

import ISP model in the degradation model

Hi Kai,
thank you for your great work.
And I want to know how to import the ISP model in the function "degradation_bsrgan_plus(img, sf=4, shuffle_prob=0.5, use_sharp=True, lq_patchsize=64, isp_model=None)" to process the RGB images.
Thank you again for your time and attention.

Question about the utils_blindsr

why did you write the "util.imresize_np" function? Is there a difference between cv2.resize and util.imresize_np?

.....
if sf == 4 and random.random() < scale2_prob: # downsample1
if np.random.rand() < 0.5:
img = cv2.resize(img, (int(1/2img.shape[1]), int(1/2img.shape[0])), interpolation=random.choice([1,2,3]))
else:
img = util.imresize_np(img, 1/2, True)
img = np.clip(img, 0.0, 1.0)
sf = 2
......