goutamgmb / deep-burst-sr Goto Github PK
View Code? Open in Web Editor NEWOfficial implementation of Deep Burst Super-Resolution
License: Other
Official implementation of Deep Burst Super-Resolution
License: Other
Hi @goutamgmb I can't download the pretrained pwc-net using the install.sh and the 'gdown' command, and I also can't open the google drive link you provide. Could you please give some advice?
Hi, thank you for your great work. Is it possible that you could also share the code of pre-processing the BurstSR dataset, which roughly align the crops from the original images? I am doing similar research which requires taking advantage of image pairs of different views and it would be really helpful if you could share the code. Thank you.
Hello, is your local.py file missing?
Can you provide a new file?
hi thanks for your nice work, can you please help me in knowing as to which part of code to be used to view generated synthetic raw images and how to view it
Hi,
I can't find the test dataset in the zip from https://data.vision.ee.ethz.ch/bhatg/zurich-raw-to-rgb.zip, there is train dataset only, so how can I get the test dataset?
Hi, thanks for the excellent work. When I tried to test your pretrained model on my captured RAW bursts, I prepared my testing datasets as your evaluation datasets —— RAW bursts captured from Samsung Galaxy S8, 4 channels 'png' format and uint16 type. However, I found that there exists checkerboard artifacts in the output super resolution image. How to avoid these checkerboard artifacts?
The input sample and the output image are shown in the https://drive.google.com/drive/folders/15ya_CuvVZ65gjBwXqfjTd1pgqi3da8cO?usp=sharing
Has the PWC-Net been fine-tuned for DBSR in "pwcnet-network-default.pth"?
能再提供一下预训练模型吗
Thanks for your wonderful work. I have some problems when I try to fine-tune DBSR on the BurstSR dataset for a burst size of 14.
I firstly train it on the synthetic dataset and it finally achieves 40.95 dB on the synthetic validation dataset. Then I try to fine-tune it on the BurstSR dataset. While the validation result only grows at the beginning but drops gradually after it achieves 47.86 dB. It seems that the training set of BurstSR is overfitted.
Have you met the same problem? Is it because the training set is too small or there is a domain gap between the training and validation set? I have been confused by this problem for months. I would appreciate it a lot if you could help me figure this out.
Thanks.
Is it possible to release DBSR retrained model got 40.76db results in synthetic dataset in your ICCV 2021 work? Thanks for your help.
gdown https://drive.google.com/uc\?id\=1s11Ud1UMipk2AbZZAypLPRpnXOS9Y1KO -O pretrained_networks/pwcnet-network-default.pth
The URL given above cannot be accessed directly, and the complete trained .pth model cannot be downloaded using the 'gdown' command. did i wrote wrong code?
Update environment settings
The environment setting file admin/local.py contains the paths for pre-trained networks, datasets etc. Update the paths in local.py according to your local environment.
The admin/local.py file is missing. Is this file still required?
Hi, @goutamgmb ! Thanks a lot for your amazing work in DBSR.
Now I want to train a Burst-SR Model which could adapt real-world image data with larger size, like 512512, instead of 160160. So how can I crop a BurSR Dataset with any size from BurstSR dataset (full).
If possible, could you please share your codes used to obtain pre-processed version of the BurstSR dataset that contains roughly aligned crops from the original images. If not, could you share more details about cropping and alignment to help me reproduce this part of work.
Thank you again!
After downloading the test dataset from the official website, there is actually 22G of the data.
Is the 1204 images selected for the validation set specific? Is that randomly chosen or others?
If there is no test dataset for val, here should have some changes:
from:
if split in ['train', 'test']:
self.img_pth = os.path.join(root, split, 'canon')
to:
if split in ['train', 'test']:
self.img_pth = os.path.join(root, 'train', 'canon')
Am I right?
Hi @goutamgmb when training the model on BurstSR dataset by using the command 'python run_training.py dbsr default_realworld', the errors are as belows. I guess it's caused by PWCNet module. And I have checked the original PWCNet git and the Pytorch-PWC git, but I didn't find similar problems. Could you please provide some advice?
Traceback (most recent call last):
File "/media/data4/syj/DBSR/deep-burst-sr/trainers/base_trainer.py", line 69, in train
self.train_epoch()
File "/media/data4/syj/DBSR/deep-burst-sr/trainers/simple_trainer.py", line 95, in train_epoch
self.cycle_dataset(loader)
File "/media/data4/syj/DBSR/deep-burst-sr/trainers/simple_trainer.py", line 75, in cycle_dataset
loss, stats = self.actor(data)
File "/media/data4/syj/DBSR/deep-burst-sr/actors/dbsr_actors.py", line 72, in call
pred, aux_dict = self.net(burst)
File "/media/data4/syj/media/data4/syj/envs/torch_burstsr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/media/data4/syj/DBSR/deep-burst-sr/models/dbsr/dbsrnet.py", line 34, in forward
out_enc = self.encoder(im)
File "/media/data4/syj/media/data4/syj/envs/torch_burstsr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/media/data4/syj/DBSR/deep-burst-sr/models/dbsr/encoders.py", line 61, in forward
offsets = self.alignment_net(x_oth.view(-1, *x_oth.shape[-3:]), x_ref.view(-1, *x_ref.shape[-3:]))
File "/media/data4/syj/media/data4/syj/envs/torch_burstsr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/media/data4/syj/DBSR/deep-burst-sr/models/alignment/pwcnet.py", line 273, in forward
flow = self.net(target_img_re, source_img_re)
File "/media/data4/syj/media/data4/syj/envs/torch_burstsr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/media/data4/syj/DBSR/deep-burst-sr/models/alignment/pwcnet.py", line 225, in forward
objEstimate = self.netSix(tenFirst[-1], tenSecond[-1], None)
File "/media/data4/syj/media/data4/syj/envs/torch_burstsr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/media/data4/syj/DBSR/deep-burst-sr/models/alignment/pwcnet.py", line 161, in forward
tenVolume = torch.nn.functional.leaky_relu(input=correlation.FunctionCorrelation(tenFirst=tenFirst, tenSecond=tenSecond), negative_slope=0.1, inplace=False)
File "/media/data4/syj/DBSR/deep-burst-sr/external/pwcnet/correlation/correlation.py", line 386, in FunctionCorrelation
return _FunctionCorrelation.apply(tenFirst, tenSecond)
File "/media/data4/syj/DBSR/deep-burst-sr/external/pwcnet/correlation/correlation.py", line 325, in forward
raise NotImplementedError()
NotImplementedError
This looks like a powerful group of functions. Do you have a quick start tutorial or code example where someone can supply a set of images and see the results? I am having a hard time grasping where to start will all this great information here.
I am not able to locate the BurstSR dataset path, the function for the download requires a path.
I got the x.dim()=4 in model/encoder.py, forward()
But the asked dim is 5
But I didn't change any code, except the input data numbers.
Hi, @goutamgmb ! Thanks a lot for your wonderful work in DBSR. In the BurstSR Dataset, you mention that the cropped images in training dataset are generated from the burst images. I guess it means that you use the original raw image to crop into the resized training raw images. So my question is, how to crop the RAW image?
Again, thanks a lot for your excellent job! Look forward to your reply :)
Hi,
when I train the dbsr model on the default_realworld setting I get the following error during training:
Error in magma getdevice arch: MAGMA not initialized (call magma init() first) or bad device
First, I thought this was due to multiprocessing start method, but it is already set to spawn.
Any Ideas?
Best,
Matthias
Hi!
I use the crops from the BurstSR dataset to some task related to computational photography and I realized that some HR images from the training samples where flat, and thus totally different from the companion LR frames. The IDs of these samples are
For reproducing the SRx4 results it seems OK to keep these four samples (out of 5405!), but for other tasks like mine it is a great deal to get rid of these.
Hope it may help!
Hello
An example on what's the proper way to create a script that takes as input: str:'path_to_img.jpg'
and returns as output: folder with dng files
thanks
Can you please tell me, how have you taken the training crops of size 448x448? Have you used cv2.resize to resize the images, as some of the images seem to be blurred to some extent?
Where in the code, have you implemented this part of cropping?
Hi, @goutamgmb I train the model using the burstsr dataset, and then I use the provided compute_score.py to compute the psnr between net_pred_warped_m and gt. The computed PSNR is around 46dB.
deep-burst-sr/evaluation/burstsr/compute_score.py
Lines 126 to 128 in 73d4f34
And then I use the provided visualize_result.py to see the SR result. But I find the SR results seem to be kind of blurry. So I calculate the PSNR result between the pred_proc_np and frame_gt_np. And the PSNR decreases to around 29 dB.
deep-burst-sr/evaluation/burstsr/visualize_results.py
Lines 71 to 75 in 73d4f34
My guess is the Canon.generate_process_image operation somehow influences the visulization. So I wonder could you please give me some advice?
When I load canon bayer data(im_raw.png), it has size 640x640x3
If I want to transfer it to bayer size (EX: NxNx1), should I just sample the data point in 640x640x3?
It seems that you have saved the structure of this directory into the checkpoints. However, the directory has been changed. The "local" module in the admin directory has been removed. Consequently, the checkpoints can not be properly loaded. Would you mind providing new checkpoints?
Hi @goutamgmb,
Is there any possibility to add or guide me how to add simililar method to visualize_synburst_val but for bursts from BurstSR dataset?
I have tried to do it following previous and visualize_results, but the results were not satisfying.
The steps I followed:
I very much appreciate any help as I am not an expert in photography technology.
Regards
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.