zhangandin / edvr

Winning Solution in NTIRE19 Challenges on Video Restoration and Enhancement (CVPR19 Workshops) - Video Restoration with Enhanced Deformable Convolutional Networks

EDVR [BasicSR] [DNI]

Paper | Project Page | Open VideoRestoration Doc (under construction)

Video Restoration with Enhanced Deformable Convolutional Networks

By Xintao Wang, Kelvin C.K. Chan, Ke Yu, Chao Dong, Chen Change Loy

EDVR won all four tracks in NTIRE 2019 Challenges on Video Restoration and Enhancement (CVPR19 Workshops).

Highlights

• A unified framework suitable for various video restoration tasks, e.g., super-resolution, deblurring, denoising, etc
• State of the art: Winners in NTIRE 2019 Challenges on Video Restoration and Enhancement
• Multi-GPU (distributed) training

---

• We are going to have a presentation in NTIRE 2019 Workshop (Monday, June 17, 2019 - room 104A Long Beach Convention Center, posters: Pacific Arena Ballroom). Welcome to our poster!

• Check out our CVPR19 work: DNI - Deep Network Interpolation for Continuous Imagery Effect Transition

---

Updates

[2019-06-12] Add training codes.
[2019-06-11] Add data preparation in wiki.
[2019-06-07] Support DUF testing (converted from officially released models).
[2019-05-28] Release testing codes.

Dependencies and Installation

• Python 3 (Recommend to use Anaconda)
• PyTorch >= 1.0
• NVIDIA GPU + CUDA
• Deformable Convolution. We use Charles Shang's DCNv2 implementation. Please first compile it.

  cd ./codes/models/modules/DCNv2
  bash make.sh
  

• Python packages: pip install numpy opencv-python lmdb pyyaml
• TensorBoard:
  • PyTorch >= 1.1: pip install tb-nightly future
  • PyTorch == 1.0: pip install tensorboardX

Dataset Preparation

We use datasets in LDMB format for faster IO speed. Please refer to wiki for more details.

Get Started

Please see wiki for the basic usage, i.e., training and testing.

Model Zoo and Baselines

Results and pre-trained models are available in the wiki-Model zoo.

Contributing

We appreciate all contributions. Please refer to mmdetection for contributing guideline.

Python code style
We adopt PEP8 as the preferred code style. We use flake8 as the linter and yapf as the formatter. Please upgrade to the latest yapf (>=0.27.0) and refer to the yapf configuration and flake8 configuration.

Before you create a PR, make sure that your code lints and is formatted by yapf.

Citation

@InProceedings{wang2019edvr,
  author    = {Wang, Xintao and Chan, Kelvin C.K. and Yu, Ke and Dong, Chao and Loy, Chen Change},
  title     = {EDVR: Video restoration with enhanced deformable convolutional networks},
  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)},
  month     = {June},
  year      = {2019},
}
@Article{tian2018tdan,
  author    = {Tian, Yapeng and Zhang, Yulun and Fu, Yun and Xu, Chenliang},
  title     = {TDAN: Temporally deformable alignment network for video super-resolution},
  journal   = {arXiv preprint arXiv:1812.02898},
  year      = {2018},
}