jackzhousz / bipnet

[CVPR 2022--Oral, Best paper Finalist] Burst Image Restoration and Enhancement. SOTA for Burst Super-resolution, Low-light Burst Image Enhancement, Burst Image De-noising

Burst Image Restoration and Enhancement (CVPR 2022 -- Oral)

Akshay Dudhane, Syed Waqas Zamir, Salman Khan, Fahad Shahbaz Khan, and Ming-Hsuan Yang

News

• June 22, 2022: BIPNet oral presentation is available here
• June 21, 2022: BIPNet was best paper finalist at CVPR 2022 🔥
• April 3, 2022: Training and Testing codes with pre-trained models are released 🔥
• March 29, 2022: BIPNet is selected for an ORAL presentation at CVPR 2022 💫
• March 3, 2022: Paper accepted at CVPR 2022 🎉

---

Abstract: Modern handheld devices can acquire burst image sequence in a quick succession. However, the individual acquired frames suffer from multiple degradations and are misaligned due to camera shake and object motions. The goal of Burst Image Restoration is to effectively combine complimentary cues across multiple burst frames to generate high-quality outputs. Towards this goal, we develop a novel approach by solely focusing on the effective information exchange between burst frames, such that the degradations get filtered out while the actual scene details are preserved and enhanced. Our central idea is to create a set of pseudo-burst features that combine complimentary information from all the input burst frames to seamlessly exchange information. However, the pseudo burst cannot be successfully created unless the individual burst frames are properly aligned to discount interframe movements. Therefore, our approach initially extracts pre-processed features from each burst frame and matches them using an edge-boosting burst alignment module. The pseudo-burst features are then created and enriched using multi-scale contextual information. Our final step is to adaptively aggregate information from the pseudo-burst features to progressively increase resolution in multiple stages while merging the pseudo-burst features. In comparison to existing works that usually follow a late fusion scheme with single-stage upsampling, our approach performs favorably, delivering state-of-the-art performance on burst super-resolution, burst low-light image enhancement and burst denoising tasks.

---

Network Architecture

Installation

See install.yml for the installation of dependencies required to run Restormer.

conda env create -f install.yml

Citation

If you use BIPNet, please consider citing:

@inproceedings{dudhane2021burst,
        title={Burst Image Restoration and Enhancement},
        author={Akshay Dudhane and Syed Waqas Zamir and Salman Khan and Fahad Shahbaz Khan and Ming-Hsuan Yang},
        booktitle={CVPR},
        year={2022}
      }

Contact

Should you have any question, please contact [email protected]

Acknowledgment: This code is based on the NTIRE21_BURSTSR toolbox.