peterzhousz / errnet

Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements (CVPR 2019)

ERRNet

The implementation of CVPR 2019 paper "Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements"

News (30/06/2019): Our suppl. material, poster, pretrained model and collected unaligned dataset are now available at Google Drive.

Highlights

• Our network can extract the background image layer devoid of reflection artifacts, as in the example:

• We captured a new dataset containing 450 unaligned image pairs that are considerably easier to collect. Image samples from our unaligned dataset are shown below:

• We introduce a simple but powerful alignment-invariant loss function to facilitate exploiting misaligned real-world training data. Finetuning on unaligned image pairs with our loss leads to sharp and reflection-free results, in contrast to the blurry ones when using a conventional pixel-wise loss (L1, L2, e.t.c.). The resulting images finetuned by different losses are shown below: (Left: Pixel-wise loss; Right: Ours)

Prerequisites

• Python >=3.5, PyTorch >= 0.4.1
• Requirements: opencv-python, tensorboardX, visdom
• Platforms: Ubuntu 16.04, cuda-8.0

Quick Start

1. Preparing your training/testing datasets

Training dataset

• 7,643 cropped images with size 224 × 224 from Pascal VOC dataset (image ids are provided in VOC2012_224_train_png.txt, you should crop the center region with size 224 x 224 to reproduce our result).

• 90 real-world training images from Berkeley real dataset

Testing dataset

• 100 synthetic testing images from CEILNet dataset (testdata_reflection_synthetic_table2)
• 20 real testing images from Berkeley real dataset.
• Three sub-datasets, namely ‘Objects’, ‘Postcard’, ‘Wild’ from SIR^2 dataset

Once the data are downloaded, you must organize the dataset according to our code implementation (see the source code of datasets.CEILDataset, e.t.c.)

2. Playing with aligned data

Testing

• Download our pretrained model from Google Drive and move errnet_060_00463920.pt to checkpoints/errnet/.
• Evaluate the model performance by python test_errnet.py --name errnet -r --icnn_path checkpoints/errnet/errnet_060_00463920.pt --hyper

Training

• Reproduce our results by python train_errnet.py --name errnet --hyper
• Check options/errnet/train_options.py to see more training options.

3. (TODO) Playing with unaligned data

Citation

If you find our code helpful in your research or work please cite our paper.

 @inproceedings{wei2019single,
   title={Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements},
   author={Wei, Kaixuan and Yang, Jiaolong and Fu, Ying and David, Wipf and Huang, Hua},
   booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
   year={2019},
 }

Contact

If you find any problem, please feel free to contact me (kaixuan_wei at bit.edu.cn). A brief self-introduction is required, if you would like to get an in-depth help from me.

Acknowledgments

• Our code architecture is inspired by CycleGAN and EDSR.

• Special thanks to @fqnchina and @ceciliavision for some discussions of this work.