Homepage: https://csbhr.github.io
GoogleScholar: https://scholar.google.com/citations?user=SBYJ6YoAAAAJ
The repository is an official implementation of our CVPR2020 paper : Cascaded Deep Video Deblurring Using Temporal Sharpness Prior
License: MIT License
Homepage: https://csbhr.github.io
GoogleScholar: https://scholar.google.com/citations?user=SBYJ6YoAAAAJ
Your work looks amazing! I want to try to run this on a dataset of my own to improve the quality of a video. As this video is blurry of it’s own I do not have a non-blurry ground truth available. Is there any way to run the model without?
Hi! Could you share information about training time for DVD dataset? On my GPU one epoch takes more than one hour and I wonder if it is correct.
Hi, dear authors, can you share your training logs on Gopro dataset?
" input=correlation.FunctionCorrelation(tensorFirst=tensorFirst, tensorSecond=tensorSecond),
AttributeError: module 'model.correlation' has no attribute 'FunctionCorrelation'"
Any ideas how to fix this?
I try to run the code and I have the following error:
Inference - 2021-05-17 03:42:15
save_image: True
border: False
model_path: model/CDVD_TSP_DVD_Paper.pt
data_path: data/dataset/
result_path: ../infer_results
n_seq: 5
size_must_mode: 4
device: cuda
Creating CDVD-TSP Net
Is meanfilter image when process mask: True
Select mask mode: concat, num_mask=1
Creating Flow PWC
Creating Recons-Video Net
SRN Video Net extra in channels: 1
The input of SRN is image
Loading model from model/CDVD_TSP_DVD_Paper.pt
Traceback (most recent call last):
File "inference.py", line 243, in <module>
Infer.infer()
File "inference.py", line 123, in infer
self.logger.write_log("# Total AVG-PSNR={:.5}, AVG-SSIM={:.4}".format(sum_psnr / n_img, sum_ssim / n_img))
ZeroDivisionError: float division by zero
What can be the reason?
Hello. I'm Kang.
I run your uploaded training code on GoPro dataset.
However, the PSNR of result ranges about 23~24 dB.
I just put my dataset path on your code and run the code by your guidance.
Could you give me some solutions for training?
Thank you.
Loading model from ../pretrain_models/CDVD_TSP_DVD_Convergent.pt
Traceback (most recent call last):
File "inference.py", line 243, in
Infer.infer()
File "inference.py", line 123, in infer
self.logger.write_log("# Total AVG-PSNR={:.5}, AVG-SSIM={:.4}".format(sum_psnr / n_img, sum_ssim / n_img))
ZeroDivisionError: float division by zero
When running inference.py
:
Traceback (most recent call last):
File "/home/ubuntu/github/CDVD-TSP/code/model/flow_pwc.py", line 10, in <module>
from model import correlation
File "/home/ubuntu/github/CDVD-TSP/code/model/correlation.py", line 282, in <module>
@cupy.util.memoize(for_each_device=True)
File "/home/ubuntu/anaconda3/envs/deblur37/lib/python3.7/site-packages/cupy/__init__.py", line 875, in __getattr__
f"module 'cupy' has no attribute {name!r}")
AttributeError: module 'cupy' has no attribute 'util'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "inference.py", line 10, in <module>
from model.cdvd_tsp import CDVD_TSP
File "/home/ubuntu/github/CDVD-TSP/code/model/cdvd_tsp.py", line 4, in <module>
from model import flow_pwc
File "/home/ubuntu/github/CDVD-TSP/code/model/flow_pwc.py", line 13, in <module>
import correlation # you should consider upgrading python
ModuleNotFoundError: No module named 'correlation'
what is gt mean?
Hi, your work is really great. It helps me a lot in my research domain. However, there is a question that can we visualize the output results of TSP module and hard example mining mask?
请问能把预训练模型放到百度网盘吗,国内上不了google网盘
when i run inference.py i get
ModuleNotFoundError: No module named 'correlation'
CDVD-TSP/code/option/template.py
Line 6 in a9bfcae
What does this superparameter do, I feel that the number of image frames in each category is not exactly equal to this value
I suppose, the default main.py
training will start training from scratch, how can use one of your pretrained weights and start training with that?
Dear!
I am trying to train CDVD-TSP on my own database, but i don't know where i can download the pretrained PWC-Net model.
I found that downloaded from https://github.com/NVlabs/PWC-Net/tree/master/PyTorch is corrupted.
Thank you in advance.
What is class Refiner and class Network in your project?
您好!想请问一下您,每个video内的帧数最多只能是100吗?这个限制应该在哪个文件里面修改呢?我在一个video文件里装了5k张图片,但是程序显示只读取了100张图片。谢谢您的解答!
Hello! Could you please tell me that the maximum number of frames in each video is 100? In which file should this restriction be changed? I have packed 5K pictures in a video file, but the program shows that only 100 pictures have been read. Thank you for your answer!
hi, your work is great!But I wonder why there is a gap between inference results (about 28.4db on GOPRO) using the provided pretrained model and the reported reuslts (31.67db) in paper?
Thanks for sharing this fantastic work.
Would you consider make a Google Collab version available for less-techinically inclined folks like myself to try it out? Thanks in advance.
How many epochs has the network learned to achieve the result shown in the article?
I'd like to know how fast the model converges
How should I change class Decoder if I want to learn Network with additional:
torch.nn.Conv2d(in_channels=16, out_channels=16, kernel_size=3, stride=1, padding=1),
torch.nn.LeakyReLU(inplace=False, negative_slope=0.1)
in each module in Extractor.
Your pretrained models do a decent job but I was hoping to add in the REDS data to get even better results. However I can't seem to get the training to work due to the following errors. I figured out how to resolve the two warnings but not sure about the actual error. I'm using pytorch 1.81 with cuda 10.2.
C:\ProgramData\Miniconda3\lib\site-packages\torch\optim\lr_scheduler.py:129: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`. Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
warnings.warn("Detected call of `lr_scheduler.step()` before `optimizer.step()`. "
C:\ProgramData\Miniconda3\lib\site-packages\torch\optim\lr_scheduler.py:369: UserWarning: To get the last learning rate computed by the scheduler, please use `get_last_lr()`.
warnings.warn("To get the last learning rate computed by the scheduler, "
Epoch 2 with Lr 1.00e-4
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\ProgramData\Miniconda3\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\ProgramData\Miniconda3\lib\multiprocessing\spawn.py", line 125, in _main
prepare(preparation_data)
File "C:\ProgramData\Miniconda3\lib\multiprocessing\spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\ProgramData\Miniconda3\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "C:\ProgramData\Miniconda3\lib\runpy.py", line 265, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\ProgramData\Miniconda3\lib\runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:\ProgramData\Miniconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "F:\workspaces\CDVD-TSP\code\main.py", line 21, in <module>
t.train()
File "F:\workspaces\CDVD-TSP\code\trainer\trainer_cdvd_tsp.py", line 34, in train
for batch, (input, gt, _) in enumerate(self.loader_train):
File "C:\ProgramData\Miniconda3\lib\site-packages\torch\utils\data\dataloader.py", line 355, in __iter__
return self._get_iterator()
File "C:\ProgramData\Miniconda3\lib\site-packages\torch\utils\data\dataloader.py", line 301, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File "C:\ProgramData\Miniconda3\lib\site-packages\torch\utils\data\dataloader.py", line 914, in __init__
w.start()
File "C:\ProgramData\Miniconda3\lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
File "C:\ProgramData\Miniconda3\lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\ProgramData\Miniconda3\lib\multiprocessing\context.py", line 327, in _Popen
return Popen(process_obj)
File "C:\ProgramData\Miniconda3\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
prep_data = spawn.get_preparation_data(process_obj._name)
File "C:\ProgramData\Miniconda3\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
_check_not_importing_main()
File "C:\ProgramData\Miniconda3\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
thanks for your great project and wonderful temporal prior!
When I retrain on your code (there is no change except removing the test() func, the training time is very long . I wonder if you have this problem during training?)
In addition, in order to speed up the training, I removed a part of the prequel in the optical flow network. I wonder if this part is useful?
change
warped01, _, _, flow_mask01 = self.flow_net(frame_list[1], frame_list[0])
into (code/model/cdvd_tsp.py)
warped01, flow_mask01 = self.flow_net(frame_list[1], frame_list[0])
change
# warp frame_2_grad_warp, _ = self.warp(frame_2_grad, flow.detach()) frame_2_warp, mask = self.warp(frame_2, flow) return frame_2_warp, flow, frame_2_grad_warp, mask
into(code/model/flow_pwc.py line 95)
# warp frame_2_warp, mask = self.warp(frame_2, flow) return frame_2_warp, mask
Thanks in advance!
Hi,I managed to run the project but I cann't run it now after reinstalling OS.I installed all the dependency but always get error like no module name torch.. Someone provides a requirement.txt, plz.
I'm trying to train your model on a very small frames (96 by 96). I'm getting following error while training:
(cdvd-tsp) vmuser@a87bb9983200:~/Data/denoising/CDVD-TSP/code$ python main.py --save ../lp_results --dir_data ../lp_train --dir_data_test ../lp_test --epochs 500 --batch_size 5
Save Path : ../experiment/../lp_results
Selected task: VideoDeblur
Making model...
Creating CDVD-TSP Net
Is meanfilter image when process mask: True
Select mask mode: concat, num_mask=1
Creating Flow PWC
Loading Flow PWC pretrain model from ../pretrain_models/network-default.pytorch
Creating Recons-Video Net
SRN Video Net extra in channels: 1
The input of SRN is image
Preparing loss function:
1.000 * L1
2.000 * HEM
n_seq: 5
n_frames_per_video: 100
Loading train => DVD DataSet
DataSet gt path: ../lp_train/gt
DataSet blur path: ../lp_train/blur
Number of videos to load: 12
Number of frames to load: 303
Dataset repeat: 16
n_seq: 5
n_frames_per_video: 100
Loading test => DVD DataSet
DataSet gt path: ../lp_test/gt
DataSet blur path: ../lp_test/blur
Number of videos to load: 3
Number of frames to load: 18
Using Trainer-CDVD-TSP
Now training
Epoch 1 with Lr 1.00e-4
Traceback (most recent call last):
File "main.py", line 21, in <module>
t.train()
File "/home/vmuser/Data/denoising/CDVD-TSP/code/trainer/trainer_cdvd_tsp.py", line 34, in train
for batch, (input, gt, _) in enumerate(self.loader_train):
File "/home/vmuser/anaconda3/envs/cdvd-tsp/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 336, in __next__
return self._process_next_batch(batch)
File "/home/vmuser/anaconda3/envs/cdvd-tsp/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 357, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
ValueError: Traceback (most recent call last):
File "/home/vmuser/anaconda3/envs/cdvd-tsp/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 106, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/vmuser/anaconda3/envs/cdvd-tsp/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 106, in <listcomp>
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/vmuser/Data/denoising/CDVD-TSP/code/data/videodata.py", line 95, in __getitem__
inputs_concat, gts_concat = self.get_patch(inputs_concat, gts_concat, self.args.size_must_mode)
File "/home/vmuser/Data/denoising/CDVD-TSP/code/data/videodata.py", line 153, in get_patch
input, gt = utils.get_patch(input, gt, patch_size=self.args.patch_size)
File "/home/vmuser/Data/denoising/CDVD-TSP/code/utils/utils.py", line 17, in get_patch
ix = random.randrange(0, iw - ip + 1)
File "/home/vmuser/anaconda3/envs/cdvd-tsp/lib/python3.6/random.py", line 199, in randrange
raise ValueError("empty range for randrange() (%d,%d, %d)" % (istart, istop, width))
ValueError: empty range for randrange() (0,-159, -159)
I am trying to learn the network on gpu (RTX 2060) with batch = 1 but the computer starts humming and turns off. On which machine the network was learned? Can I change some parameter to the training on my computer starts correctly?
hi,
your work is great.
I just wonder can this project work for single image deblurring?
thank you
There are 3 video deblurring datasets. DVD(Deep video deblurring for hand-held cameras), GOPRO (from Deep multi-scale convolutional neural network for dynamic scene deblurring) and REDS (NTIRE 2019)
What datasets did the released pretrained model use for training?
Also what datasets did the EDVR (Quantitative evaluations) train on?
Did you test CDVD-TSP result on REDS dataset?
Hi , your work is very good. But I do not know how to download the PWC-Net pre-trained model when I try to train your network. Can you tell me ? Thanks a lot!
dear author:
I have a question that training on gopro, have 2 blur folder, that blur and blur_gamma, I want to know you used is blur or blur_gamma? thankyou
Hello,
In your paper you compare your model with EDVR.
Did you train EDVR on your dataset or only tested the pretrained model? In EDVR paper they use REDS dataset. For the sake of fare comparison could you please train your model with REDS dataset? It would be really good to see the CDVD-TSP and EDVR trained on same data.
I think everyone would highly appreciate it.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.