swz30 / mprnet Goto Github PK

WOW! Thanks for your great work, and when will you make the codes open source.

Hi Syed,
could you update the repo with the license for this MPRNet implementation ?

thanks !

While trying/testing a pretrained model with the cmd:

python demo.py --task Deblurring --input_dir path_to_images --result_dir save_images_here

I am getting some runtime error.
So basically, the terminal shows this

RuntimeError: CUDA out of memory. Tried to allocate 2.15 GiB (GPU 0; 1.96 GiB total capacity; 420.11 MiB already allocated; 400.00 MiB free; 233.89 MiB cached)

I am guessing this is most likely because my GPU's memory is occupied and is not sufficient for some allocation that the program tries during runtime. Since you can see that the allocation size if already greater than the total capacity, so could you suggest or recommend some methods or change in the code for reducing this allocation size.
Thanks

The code reports the error as shown. How should I correct it?

Hello, thank you for your patience!
Your suggestion is very effective for me. After changing the learning rate to 1.5e-4, the loss （debluring）was about 100 in the first epoch and dropped to 70 in the 69th epoch, but it suddenly became 800,000 in the 70th epoch! I don't know what happened…

Thank you very much for sharing, this is a very interesting paper. I face some issues when I try to train the denoise and Derain model. when I run the Derain the train.py and download the dataset.i face the
"RuntimeError: stack expects each tensor to be equal size, but got [3, 229, 256] at entry 0 and [3, 256, 171] at entry 1". should I to pretrain the data in the same size before I run the train.py of the Derain the model? could you give me a hand for this problem? thanks very much

in_c and out_c arguments are un-used, and a default value of 3 is used in the code. Should the constructor instead be;

class MPRNet(nn.Module):
    def __init__(self, in_c=3, out_c=3, n_feat=80, scale_unetfeats=48, scale_orsnetfeats=32, num_cab=8, kernel_size=3,
                 reduction=4, bias=False):
        super(MPRNet, self).__init__()

        act = nn.PReLU()
        self.shallow_feat1 = nn.Sequential(conv(in_c, n_feat, kernel_size, bias=bias),
                                           CAB(n_feat, kernel_size, reduction, bias=bias, act=act))
        self.shallow_feat2 = nn.Sequential(conv(in_c, n_feat, kernel_size, bias=bias),
                                           CAB(n_feat, kernel_size, reduction, bias=bias, act=act))
        self.shallow_feat3 = nn.Sequential(conv(in_c, n_feat, kernel_size, bias=bias),
                                           CAB(n_feat, kernel_size, reduction, bias=bias, act=act))

        # Cross Stage Feature Fusion (CSFF)
        self.stage1_encoder = Encoder(n_feat, kernel_size, reduction, act, bias, scale_unetfeats, csff=False)
        self.stage1_decoder = Decoder(n_feat, kernel_size, reduction, act, bias, scale_unetfeats)

        self.stage2_encoder = Encoder(n_feat, kernel_size, reduction, act, bias, scale_unetfeats, csff=True)
        self.stage2_decoder = Decoder(n_feat, kernel_size, reduction, act, bias, scale_unetfeats)

        self.stage3_orsnet = ORSNet(n_feat, scale_orsnetfeats, kernel_size, reduction, act, bias, scale_unetfeats,
                                    num_cab)

        self.sam12 = SAM(n_feat, kernel_size=1, bias=bias)
        self.sam23 = SAM(n_feat, kernel_size=1, bias=bias)

        self.concat12 = conv(n_feat * 2, n_feat, kernel_size, bias=bias)
        self.concat23 = conv(n_feat * 2, n_feat + scale_orsnetfeats, kernel_size, bias=bias)
        self.tail = conv(n_feat + scale_orsnetfeats, out_c, kernel_size, bias=bias)

if i just have blur data , how can i train？

Hello, thank you for your contribution.
I want to know have you tried to use dual pixel images to train your network?
for example dual_pixels_dataset_canon, which only contain 500 pairs dual pixel labels.
if yes, can you share the codes or ideas about how use it on that projects?

hello! when I use four GPU TiTanXP to train,it shows that CUDA out of memory.
Could you tell me how many GPUs you use,please.
Thanks very much.

A CUDA Out of Memory error occurred when I was testing the DeBlurring model on the GTX 1080Ti in my datasets

RuntimeError: CUDA out of memory. Tried to allocate 636.00 MiB (GPU 0; 10.92 GiB total capacity; 8.34 GiB already allocated; 21.38 MiB free; 10.34 GiB reserved in total by PyTorch)

Can you help me solve this problem？

I downloaded the Medium dataset, but it is in Mat format. Do I have to convert it to PNG format first. Or do you have the code to convert the format? Thanks!

Hi, the performance of MPRNet is impressive!
I am wondering if you could also test MPRNet on our newly released real blur dataset BSD (more blur intensity and motion pattern settings and better design of beam-splitter acquisition system).
The link of the dataset is as follows:
https://github.com/zzh-tech/ESTRNN
Thank you very much :)

Hello author, can i run the test with cpu or amd gpu? because i tested on my PC it shows error about found no nvidia driver.

Hello, thank you for your patience!
Your suggestion is very effective for me. After changing the learning rate to 1.5e-4, the loss （debluring）was about 100 in the first epoch and dropped to 70 in the 69th epoch, but it suddenly became 800,000 in the 70th epoch! I don't know what happened…

when I run deblurring task python train.py , I get the error "AttributeError: 'CosineAnnealingLR' object has no attribute 'get_last_lr'" when the epoch is 3, how can i solve the problem

Great Job！How about use this network for the task of dehazing？

RuntimeError: CUDA out of memory. Tried to allocate 1.18 GiB (GPU 0; 24.00 GiB total capacity; 18.28 GiB already allocated; 549.31 MiB free; 21.26 GiB reserved in total by PyTorch)

This occurred when trying running demo.py using a 3296x2400 png.

How can you reduce the batch size?

or

How would you clear the cached memory using 'torch.cuda.empty_cache()'

P.S Nothing memory intensive was running in the background and I have already tried shutting down or restarting my PC. This works for smaller images, however, I am looking at if possible for higher dimensions.

Old photo has some characteristic like noise, blur.

Hello, my picture contains a fingerprint (red) and a signature (black). I want to remove the signature and repair the part of the fingerprint obscured by the signature. Can I use Deraining to transform it successfully?

Great work! I'm trying to rebuild the network with tensorflow framework. However, all blocks in MRPNet have no normalization layer, neither BatchNormalization nor GroupNormalization. As the network architecture is pretty huge, I just wonder whether the network exists the gradient vanishing/exploding issue in training. Especially for the ORS-Net, it contains 8 CAB blocks.
I have roughly built the MPRNet, but the training is not stable. The loss is easily becoming inf or nan.

Hi, thank you for your very cool work for image resotration.

I tried to evaluate Hide dataset.

I downloaded the dataset using link you provide.

But input folder have 1,079 images and target folder have only 942 images.

Could you describe how to evaluate model for Hide dataset?

Did you evaluate on only 942 images?

Thank you

it's wrong when i train for my custom data . anyone now ,thanks!!!

first
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 3 and 4 in dimension 1 at /pytorch/aten/src/TH/generic/THTensor.cpp:689
i add .convert('RGB') after Image.open(inp_path)
then
RuntimeError: Calculated padded input size per channel: (2 x 130). Kernel size: (3 x 3). Kernel size can't be greater than actual input size

Test100

In paper, table 2, we can see Test100 contains 100 image pairs.

And ... there are 2801 images in Test2800-input, one image named 926_10(1).jpg lol.

first RuntimeError: Given groups=1, weight of size 96 3 3 3, expected input[1, 4, 225, 149] to have 3 channels, but got 4 channels instead
I change inp = Image.open(path_inp) to inp = Image.open(path_inp).convert('RGB')
but than RuntimeError: The size of tensor a (296) must match the size of tensor b (298) at non-singleton dimension 3

thanks

===>Testing using weights:  ./pretrained_models/model_denoising.pth
  0%|          | 0/1 [00:00<?, ?it/s]D:\WorkTools\ProgramData\Anaconda3\envs\py36\lib\site-packages\torch\nn\functional.py:3103: UserWarning: The default behavior for interpolate/upsample with float scale_factor changed in 1.6.0 to align with other frameworks/libraries, and now uses scale_factor directly, instead of relying on the computed output size. If you wish to restore the old behavior, please set recompute_scale_factor=True. See the documentation of nn.Upsample for details. 
  warnings.warn("The default behavior for interpolate/upsample with float scale_factor changed "
  0%|          | 0/1 [00:16<?, ?it/s]
Traceback (most recent call last):
  File "E:/gitrestorys/denoising/MPRNet/Denoising/test_SIDD.py", line 65, in <module>
    restored_patch = model_restoration(noisy_patch)
  File "D:\WorkTools\ProgramData\Anaconda3\envs\py36\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "D:\WorkTools\ProgramData\Anaconda3\envs\py36\lib\site-packages\torch\nn\parallel\data_parallel.py", line 149, in forward
    return self.module(*inputs, **kwargs)
  File "D:\WorkTools\ProgramData\Anaconda3\envs\py36\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "E:\gitrestorys\denoising\MPRNet\Denoising\MPRNet.py", line 300, in forward
    res1_top = self.stage1_decoder(feat1_top)
  File "D:\WorkTools\ProgramData\Anaconda3\envs\py36\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "E:\gitrestorys\denoising\MPRNet\Denoising\MPRNet.py", line 145, in forward
    x = self.up32(dec3, self.skip_attn2(enc2))
  File "D:\WorkTools\ProgramData\Anaconda3\envs\py36\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "E:\gitrestorys\denoising\MPRNet\Denoising\MPRNet.py", line 183, in forward
    x = x + y
RuntimeError: The size of tensor a (66) must match the size of tensor b (67) at non-singleton dimension 2

Process finished with exit code -1

how to fix it

1 when I train debluring model，patch_size=512,batch_size=1, rtx2080ti cuda out of memory，I wander know how many
gpu memory should i use in this situation ,because I think path_size =256 a little bit small for image

2 Have you ever try add gan loss to your model for ablation experiment?

Using 4 GPUs of 2080Ti to train the MPRNet deblurring model, I find the maximum batchsize is 4 while the default batchsize is 16. So, other default parameters such as learning rate need to be tuned accordingly ?

Excuse me, what is the subdirectory' input' under' train' of deblur data set? Have you processed the gopro data set yourself? Because the subdirectory under the original gopro is ’gopro _ large/train/**（e.g gopr0884 _ 11 _ 00 )/sharp’
I don't know how to modify the path of dataset……
thank you very much!

Hi, your MPRNet is excellent. I also work on image deblurring tasks, but have been struggling to design a good loss function. I'm wondering that could you write some example of train.py, and I'm also curious about how do implement proposed loss function.
Thanks.

I am a new bird in image denoising but have some experiences in image restoration.
After reading github and paper , I don't know image denoising TrainDatasets and TestDatasets when SWZ30 are experimenting.
Can someone directy provide me Image Denoising TrainDatasets and TestDatasets by google driver ?
Thanks a lot ！！

Hi, I test the denoise model by using skimage.measure.compare_ssim for computing ssim value of sidd dataset, but I couldn't get the value as your paper shows. Could you release the code of ssim value computing?
Thank you in advance.

Hi, Thanks for the great works. I have one question about perf and #param mentioned in the paper(Table 7). I use torchsummary to generate the GMACS and #param for MPRNET(2-stage), it turns out that #param is 53.66MB(>>11.3MB in paper), from GMACS side, it is about 10x more than DMPHN(1-2-4). Although torchsummary isn't perfect and there exists a delta between real performance and GMACS, I wonder any hint/idea to explain the gap ?

Hello, I want to run the model. The shape of my torch is (3,4,224,224). But when I wanted to tun it, the following Error occurred. Can you please help me? Thank you!

Hi, Thanks for your greate work and the open source code. Can you give the link of the datsets used in the paper on Baidu Drive?

Hi! I try to use the small model of MPRNet.
Could you share a checkpoint for the small model of MPRNet mentioned here?
Thanks

Hello, thank you for your contribution.
I changed batchsize to 1 (in training.yml), and used two GPUs to run, but the loss was about 1200 in the first two epochs, and turned into INF in the third epoch. Could you please tell me why? (I am running debluring)

I want to ask, is your image denoiseing result on RGB channel or YCbCr channel？
Thank you

I am facing difficulty while running the code on windows.
It shows the following error on running train.py:
RuntimeError:
Attempt to start a new process before the current process
has finished its bootstrapping phase.
This probably means that you are on Windows and you have
forgotten to use the proper idiom in the main module:
if name == 'main':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce a Windows executable.
Can anyone please help out with this ?

Hi @swz30 ,
I was interested in exploring this a bit further and was wondering if you had any plans to release the training scripts?

I'm trying to deblur my image with the size around 100x100 pixels^2. And 2.97% of the outputs of the deblur have some distortion effect.

Question;

1. What causes the problem?
2. Any method to fix it?

Hi, thanks for the great work.

Could you tell me which GPU you use for training, how long it takes for training, and the version of PyTorch?

thank you so much!!!

thanks!

Hi,
I'd like to know if you have pre-processed the data of RealBlur datasethttp://cg.postech.ac.kr/research/realblur/.
Since I directly use those data for training, the loss becomes inf.
The learning rate and batch size are as default settings.
I'd like to know how you select the data for training.Thanks!

Hi authors,
Thank you for the good work.
Can you please optimize and run your code in google Colab?

Edit: I found the issue: CUDA 9.0. It doesn't support NVidia 2xxxx series (Turing). Can you change that to something that would work with newer GPU cards?

EDIT2: Found a solution: use conda install pytorch==1.1.0 torchvision==0.3.0 cudatoolkit=10.0 -c pytorch

Hello,
I'm trying to run this and I did everything as in instruction, but I get this error (RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED) when I try to run pretrained model (which I downloaded).
I'm using in on Windows.

What could be a problem here?

python demo.py --task Denoising --input_dir ./export --result_dir ./output/

Full traceback

Traceback (most recent call last):
  File "demo.py", line 74, in <module>
    restored = model(input_)
  File "D:\Users\user_name\miniconda3\envs\pytorch1\lib\site-packages\torch\nn\modules\module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "Denoising\MPRNet.py", line 284, in forward
    x1ltop = self.shallow_feat1(x1ltop_img)
  File "D:\Users\user_name\miniconda3\envs\pytorch1\lib\site-packages\torch\nn\modules\module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "D:\Users\user_name\miniconda3\envs\pytorch1\lib\site-packages\torch\nn\modules\container.py", line 92, in forward
    input = module(input)
  File "D:\Users\user_name\miniconda3\envs\pytorch1\lib\site-packages\torch\nn\modules\module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "D:\Users\user_name\miniconda3\envs\pytorch1\lib\site-packages\torch\nn\modules\conv.py", line 338, in forward
    self.padding, self.dilation, self.groups)
RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

Hello, I am using your published network structure (MPRNet) to retrain a new denoising dataset. Why doesn't the loss function drop? Are there other settings?