goutamgmb / deep-burst-sr Goto Github PK

Hi @goutamgmb I can't download the pretrained pwc-net using the install.sh and the 'gdown' command, and I also can't open the google drive link you provide. Could you please give some advice?

Hi, thank you for your great work. Is it possible that you could also share the code of pre-processing the BurstSR dataset, which roughly align the crops from the original images? I am doing similar research which requires taking advantage of image pairs of different views and it would be really helpful if you could share the code. Thank you.

Hello, is your local.py file missing?
Can you provide a new file？

hi thanks for your nice work, can you please help me in knowing as to which part of code to be used to view generated synthetic raw images and how to view it

Hi,
I can't find the test dataset in the zip from https://data.vision.ee.ethz.ch/bhatg/zurich-raw-to-rgb.zip, there is train dataset only, so how can I get the test dataset?

Hi, thanks for the excellent work. When I tried to test your pretrained model on my captured RAW bursts, I prepared my testing datasets as your evaluation datasets —— RAW bursts captured from Samsung Galaxy S8, 4 channels 'png' format and uint16 type. However, I found that there exists checkerboard artifacts in the output super resolution image. How to avoid these checkerboard artifacts?

The input sample and the output image are shown in the https://drive.google.com/drive/folders/15ya_CuvVZ65gjBwXqfjTd1pgqi3da8cO?usp=sharing

Has the PWC-Net been fine-tuned for DBSR in "pwcnet-network-default.pth"?

能再提供一下预训练模型吗

Thanks for your wonderful work. I have some problems when I try to fine-tune DBSR on the BurstSR dataset for a burst size of 14.

I firstly train it on the synthetic dataset and it finally achieves 40.95 dB on the synthetic validation dataset. Then I try to fine-tune it on the BurstSR dataset. While the validation result only grows at the beginning but drops gradually after it achieves 47.86 dB. It seems that the training set of BurstSR is overfitted.

Have you met the same problem? Is it because the training set is too small or there is a domain gap between the training and validation set? I have been confused by this problem for months. I would appreciate it a lot if you could help me figure this out.

Thanks.

Is it possible to release DBSR retrained model got 40.76db results in synthetic dataset in your ICCV 2021 work? Thanks for your help.

gdown https://drive.google.com/uc\?id\=1s11Ud1UMipk2AbZZAypLPRpnXOS9Y1KO -O pretrained_networks/pwcnet-network-default.pth

The URL given above cannot be accessed directly, and the complete trained .pth model cannot be downloaded using the 'gdown' command. did i wrote wrong code?

Update environment settings
The environment setting file admin/local.py contains the paths for pre-trained networks, datasets etc. Update the paths in local.py according to your local environment.

The admin/local.py file is missing. Is this file still required?

Hi, @goutamgmb ! Thanks a lot for your amazing work in DBSR.

Now I want to train a Burst-SR Model which could adapt real-world image data with larger size, like 512512, instead of 160160. So how can I crop a BurSR Dataset with any size from BurstSR dataset (full).

If possible, could you please share your codes used to obtain pre-processed version of the BurstSR dataset that contains roughly aligned crops from the original images. If not, could you share more details about cropping and alignment to help me reproduce this part of work.

Thank you again!

After downloading the test dataset from the official website, there is actually 22G of the data.
Is the 1204 images selected for the validation set specific? Is that randomly chosen or others?

If there is no test dataset for val, here should have some changes:
from:

    if split in ['train', 'test']:
          self.img_pth = os.path.join(root, split, 'canon')

to:

        if split in ['train', 'test']:
            self.img_pth = os.path.join(root, 'train', 'canon')

Am I right?

Hi @goutamgmb when training the model on BurstSR dataset by using the command 'python run_training.py dbsr default_realworld', the errors are as belows. I guess it's caused by PWCNet module. And I have checked the original PWCNet git and the Pytorch-PWC git, but I didn't find similar problems. Could you please provide some advice?

Traceback (most recent call last):
File "/media/data4/syj/DBSR/deep-burst-sr/trainers/base_trainer.py", line 69, in train
self.train_epoch()
File "/media/data4/syj/DBSR/deep-burst-sr/trainers/simple_trainer.py", line 95, in train_epoch
self.cycle_dataset(loader)
File "/media/data4/syj/DBSR/deep-burst-sr/trainers/simple_trainer.py", line 75, in cycle_dataset
loss, stats = self.actor(data)
File "/media/data4/syj/DBSR/deep-burst-sr/actors/dbsr_actors.py", line 72, in call
pred, aux_dict = self.net(burst)
File "/media/data4/syj/media/data4/syj/envs/torch_burstsr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/media/data4/syj/DBSR/deep-burst-sr/models/dbsr/dbsrnet.py", line 34, in forward
out_enc = self.encoder(im)
File "/media/data4/syj/media/data4/syj/envs/torch_burstsr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/media/data4/syj/DBSR/deep-burst-sr/models/dbsr/encoders.py", line 61, in forward
offsets = self.alignment_net(x_oth.view(-1, *x_oth.shape[-3:]), x_ref.view(-1, *x_ref.shape[-3:]))
File "/media/data4/syj/media/data4/syj/envs/torch_burstsr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/media/data4/syj/DBSR/deep-burst-sr/models/alignment/pwcnet.py", line 273, in forward
flow = self.net(target_img_re, source_img_re)
File "/media/data4/syj/media/data4/syj/envs/torch_burstsr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/media/data4/syj/DBSR/deep-burst-sr/models/alignment/pwcnet.py", line 225, in forward
objEstimate = self.netSix(tenFirst[-1], tenSecond[-1], None)
File "/media/data4/syj/media/data4/syj/envs/torch_burstsr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/media/data4/syj/DBSR/deep-burst-sr/models/alignment/pwcnet.py", line 161, in forward
tenVolume = torch.nn.functional.leaky_relu(input=correlation.FunctionCorrelation(tenFirst=tenFirst, tenSecond=tenSecond), negative_slope=0.1, inplace=False)
File "/media/data4/syj/DBSR/deep-burst-sr/external/pwcnet/correlation/correlation.py", line 386, in FunctionCorrelation
return _FunctionCorrelation.apply(tenFirst, tenSecond)
File "/media/data4/syj/DBSR/deep-burst-sr/external/pwcnet/correlation/correlation.py", line 325, in forward
raise NotImplementedError()
NotImplementedError

This looks like a powerful group of functions. Do you have a quick start tutorial or code example where someone can supply a set of images and see the results? I am having a hard time grasping where to start will all this great information here.

I am not able to locate the BurstSR dataset path, the function for the download requires a path.

I got the x.dim()=4 in model/encoder.py, forward()
But the asked dim is 5
But I didn't change any code, except the input data numbers.

Hi, @goutamgmb ! Thanks a lot for your wonderful work in DBSR. In the BurstSR Dataset, you mention that the cropped images in training dataset are generated from the burst images. I guess it means that you use the original raw image to crop into the resized training raw images. So my question is, how to crop the RAW image?

Again, thanks a lot for your excellent job! Look forward to your reply :)

Hi,

when I train the dbsr model on the default_realworld setting I get the following error during training:

Error in magma getdevice arch: MAGMA not initialized (call magma init() first) or bad device
First, I thought this was due to multiprocessing start method, but it is already set to spawn.

Any Ideas?

Best,
Matthias

Hi!

I use the crops from the BurstSR dataset to some task related to computational photography and I realized that some HR images from the training samples where flat, and thus totally different from the companion LR frames. The IDs of these samples are

• 0004_0006
• 0026_0002
• 0130_0041
• 0168_0018

For reproducing the SRx4 results it seems OK to keep these four samples (out of 5405!), but for other tasks like mine it is a great deal to get rid of these.

Hope it may help!

Hello

An example on what's the proper way to create a script that takes as input: str:'path_to_img.jpg'
and returns as output: folder with dng files

thanks

Can you please tell me, how have you taken the training crops of size 448x448? Have you used cv2.resize to resize the images, as some of the images seem to be blurred to some extent?
Where in the code, have you implemented this part of cropping?

Hi, @goutamgmb I train the model using the burstsr dataset, and then I use the provided compute_score.py to compute the psnr between net_pred_warped_m and gt. The computed PSNR is around 46dB.

And then I use the provided visualize_result.py to see the SR result. But I find the SR results seem to be kind of blurry. So I calculate the PSNR result between the pred_proc_np and frame_gt_np. And the PSNR decreases to around 29 dB.

My guess is the Canon.generate_process_image operation somehow influences the visulization. So I wonder could you please give me some advice?

When I load canon bayer data(im_raw.png), it has size 640x640x3
If I want to transfer it to bayer size (EX: NxNx1), should I just sample the data point in 640x640x3?

It seems that you have saved the structure of this directory into the checkpoints. However, the directory has been changed. The "local" module in the admin directory has been removed. Consequently, the checkpoints can not be properly loaded. Would you mind providing new checkpoints?

Hi @goutamgmb,

Is there any possibility to add or guide me how to add simililar method to visualize_synburst_val but for bursts from BurstSR dataset?
I have tried to do it following previous and visualize_results, but the results were not satisfying.
The steps I followed:

1. load SamsungRAWImage
2. convert RGGB to RGB
3. use generate_processed_image method

I very much appreciate any help as I am not an expert in photography technology.

Regards