ysnower / bicubic-plusplus

An unofficial bicubic++ repo

Bicubic++: Slim, Slimmer, Slimmest - Designing an Industry-Grade Super-Resolution Network (🚀 Winner of NTIRE RTSR Challange Track 2 (x3 SR) @ CVPR 2023)

##note:This is an unofficial code repo,some code is inherited from bicubic-plusplus,this method adds ~1dB on Bicubic upscaling PSNR for all tested SR datasets and runs with ~0.5ms(2000FPS) per image on RTX4090,only inference time is included, not pre-processing and post-processing time.

Installation

git clone https://github.com/Ysnower/bicubic-plusplus.git

2.Install the dependencies. Python 3.7 PyTorch 1.13.1 opencv-python 4.5.1.48 onnxruntime 1.10.0 onnxruntime-gpu 1.10.0

my gpu is RTX4090 with CUDA 12.1

Data Preparation

only train on DIV2K dataset,you can use your own data, the number of hr images must be equal to lr images. Download DIV2K data password:xb5r and unzip the DIV2K.zip to the datasets. I used 860 images for training and 40 images for validation.

Train

1.Set validation & training dataset paths in configs/conf.yaml (data.val.lr_path, data.val.hr_path, data.train.lr_path, data.train.hr_path). Set loader.train.batch_size and loader.val.batch_size according to your dataset.

note: if use degradation(blur is True or img_noise is True), training will slow down,you can use degradation offline to generate lr data to the lr_path.

The scale/sr_rate must be divisible by patch_cropsize.

2.Run train code.

python3 train.py

pytorch inference

python3 inference.py

You can change the img_path in the inference.py file to your image location.

onnx inference

1.pytorch to onnx model

python3 torch2onnx.py

You can change the model_path to your pytorch model location and change the export_onnx_file to your onnx model name.

2.onnxruntime inference

python3 onnx_inference.py

reference

1.bicubic-plusplus

2.pytorch-static-quant

If this repository helps you，please star it. Thanks.

bicubic-plusplus's Issues

I got a bad result.

Hi! Thank you for generously sharing your code.
However, I encountered some problems when trying it out. I used my own images to generate the x3 SR images with both your PTH files and the paper author’s PTH file. However, I got a lower score than the bicubic interpolation approach with the PSNR and SSIM evaluation index. Could you please help me with this issue? Thank you! 😊

IndexError: list index out of range

Hi! Thanks for your coke.
But I meet some problems as follow:

(bicubic++2) chenz@chenz-System-Product-Name:~/bicubic-plusplus$ python train.py
/home/chenz/anaconda3/envs/bicubic++2/lib/python3.7/site-packages/torch/utils/data/dataloader.py:557: UserWarning: This DataLoader will create 24 worker processes in total. Our suggested max number of worker in current system is 8, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
cpuset_checked))
Traceback (most recent call last):
File "train.py", line 119, in
for k, return_dict in enumerate(train_data):
File "/home/chenz/anaconda3/envs/bicubic++2/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 628, in next
data = self._next_data()
File "/home/chenz/anaconda3/envs/bicubic++2/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1333, in _next_data
return self._process_data(data)
File "/home/chenz/anaconda3/envs/bicubic++2/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1359, in _process_data
data.reraise()
File "/home/chenz/anaconda3/envs/bicubic++2/lib/python3.7/site-packages/torch/_utils.py", line 543, in reraise
raise exception
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/chenz/anaconda3/envs/bicubic++2/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
data = fetcher.fetch(index)
File "/home/chenz/anaconda3/envs/bicubic++2/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 58, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/chenz/anaconda3/envs/bicubic++2/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 58, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/chenz/bicubic-plusplus/training/dataset.py", line 66, in getitem
img_hr, img_lr = self.imread_modcrop(self.hr_images[item], self.lr_images[item], is_train=self.is_train)
IndexError: list index out of range

I got a not so good result

Code execution is slow.

Thank you for sharing your great work.
Loading initial model parameters takes approximately 10 minutes.
Execution speed also takes about 2.5 seconds for a 0004x3.png (680x448) image.

GPU A6000, torch 1.13.1 opencv-python 4.5.1.48 onnxruntime 1.10.0 onnxruntime-gpu 1.10.0
I ran it in the above environment.
Are there any possible causes?

使用CUDA调用GPU时第一次推理速度慢

设置

1. 平台：Windows 11
2. 设备：Intel i5-13600KF + RTX 4090 + cuda 11.7
3. 环境 onnxruntime=1.10.0、onnxruntime-gpu=1.10.0，opencv-python=4.5.1.48，torch =1.13.1+cu117，torchaudio =0.13.1+cu117，torchvision=0.14.1+cu117

描述

直接运行一次inference.py，使用cuda，仓库自带图片，耗时1023ms，

修改代码运行一次inference.py，使用cpu，仓库自带图片，耗时35ms，

使用for循环运行inference.py，将output = network(img)部分循环100次记录时间，耗时达到ReadME中的描述，并且甚至出现了0.0ms

问题

相同的情况出现在了我的另一台2060设备上，查阅资料确实有CPU任务比GPU任务快的情况，比如模型太小或者数据量太小，把东西从CPU加载到GPU的时间比整个任务都长，或者是任务存在cache导致后续的任务并不是完整的，导致出现了0.0ms的速度。我是一个超分辨率入门学习者，希望作者能解答我的疑惑，这到底是单纯的第一张慢后续的正常，还是因为cache，其实都很慢？

NoneType' object has no attribute 'astype'

使用FFHQ数据进行训练，但是总会遇到读取不到数据的情况
NoneType' object has no attribute 'astype'