likesum / deepfnf

Code for "Deep Denoising of Flash and No-Flash Pairs for Photography in Low-Light Environments".

Deep Denoising of Flash and No-Flash Pairs for Photography in Low-Light Environments

Project Page | Video | Paper

A Tensorflow implementation of our CVPR 2021 work on deep flash denoising.

Deep Denoising of Flash and No-Flash Pairs for Photography in Low-Light Environments (CVPR 2021)
Zhihao Xia¹, Michael Gharbi², Federico Perazzi³, Kalyan Sunkavalli², Ayan Chakrabarti^1
1WUSTL, ²Adobe Research, ³Facebook

Dependencies

Python 3 + Tensorflow-1.14

Download our testing benchmark

We generate our test set with 128 images (filenames provided in data/test.txt) from the raw images of the Flash and Ambient Illuminations Dataset. The noisy and clean flash/no-flash pairs can be found here. You can also download the test set by running

bash ./scripts/download_testset.sh

Download pre-trained model

Our pre-trained model for flash denoising can be found here. You can run

bash ./scripts/download_models.sh

to download it.

Test with pre-trained models

Run

python test.py [--wts path_to_model]

to test flash denoising with the pre-trained model on our test set.

Training your own models

Our model is trained on the raw images of the Flash and Ambient Illuminations Dataset. To train your own model, download the dataset and update data/train.txt and data/val.txt with path to each image. Note that you need to exclude images that are used in our test set (filenames provided in data/test.txt) from the training or val set.

The raw images of the Flash and Ambient Illuminations Dataset are in 16-bit PNG files. Exif information including the color matrix and calibration illuminant necessary for the color mapping are attached to the PNGs. To save them in pickle files for training and testing later, run

python dump_exif.py

After that, run

python gen_valset.py

to generate a validation dataset.

Finally, run

python train.py

to train the model. You can press ctrl-c at any time to stop the training and save the checkpoints (model weights and optimizer states). The training script will resume from the latest checkpoint (if any) in the model directory and continue training.

Citation

If you find the code useful for your research, we request that you cite the paper. Please contact [email protected] with any questions.

@InProceedings{deepfnf2021,
    author={Zhihao Xia and Micha{\"e}l Gharbi and Federico Perazzi and Kalyan Sunkavalli and Ayan Chakrabarti}
    title     = {Deep Denoising of Flash and No-Flash Pairs for Photography in Low-Light Environments},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {2063-2072}
}

Acknowledgments

This work was supported by the National Science Foundation under award no. IIS-1820693. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors, and do not necessarily reflect the views of the National Science Foundation.

License

This implementation is licensed under the MIT License.

deepfnf's Issues

How to test my own images?

Thanks for your works! How to test my own images?

Excellent work and a question

I tried to retrain your code, but I couldn't reach the performance of the pre-trained model. Is there anything to watch out for during training?

sorry for wrong issue

Sorry for Wrong issue

Excellent work and a questions?

The pre-training model of this project has achieved good results, but the model trained by ourselves have not achieved ideal results. May I ask how many interations you have trained to get the pre-training model? Have the hyperparameters used been changed?

excellent work and several questions

Excellent work and a question

Is validating the model necessary? For I don't understand the effect of the function swap_val and swap_train.

Excellent work and a question about OOM

When I am training with 6g gpu memory, my tensorflow keeps prompting OOM for gpu-training , how can I solve it？I have decreased bsz ( I think it's batch_size ) and nch(number of channel), but still cannot fix it.