o0t1ng0o / depth-aware-endoscopy-sr

Dynamic Depth-Aware Network for Endoscopy Super-Resolution (JBHI 2022)

Depth-Aware-Endoscopy-SR

This repository is an official PyTorch implementation of the paper "Dynamic Depth-Aware Network for Endoscopy Super-Resolution"[paper] from JBHI 2022

Environment

Please follow the requirements.txt

Dataset

In this work, we use the Kvasir and the EndoScene datasets.

1. We provide the HR and LR images (factor=8) for the Kvasir dataset, which can be downloaded from google drive. This includes the GT depth map of LR imaiges (LR_depth.targ.gz). For the factor = 2 or 4, please manually downscale the HR images according to the target factor.
   
2. To download EndoScene dataset, please see here. The corresponding depth maps for LR images can be obtained through the following depth estimation part to predict depth maps.
   

Training & Testing Model

Here, we give an example to train the SR model for x8 Kvasir dataset.

1. Configuration:
   Please modify the data path dataroot_GT, dataroot_LQ, dataroot_depthMap in codes/options/train/train_depthNet_SEAN_depthMask_x8.yml
   
2. Training:
   

sh ./launch/train.sh

3. Testing:
   Please modify the model and data path pretrain_model_G, dataroot_GT, dataroot_LQ, dataroot_depthMap in codes/options/test/test_depthNet.yml
   

sh ./launch/test.sh

Depth Estimation

Here, we pre-trained a depth estimator based on monodepth2 and use this model to generate the depth map as ground-truth depth map for our proposed method.
Please download the pre-trained model, unzip weights_19.tar.gz and place it to ./codes/depth_estimation/pretrained_models/weights_19

cd ./codes/depth_estimation

Please provide --image_path data_root/img_path in ./launch/test.sh
Run the following command to obtain the depth maps:

sh ./launch/test.sh

Cite

If you find our work useful in your research or publication, please cite our work:

@article{chen2022dynamic,
  title={Dynamic depth-aware network for endoscopy super-resolution},
  author={Chen, Wenting and Liu, Yifan and Hu, Jiancong and Yuan, Yixuan},
  journal={IEEE Journal of Biomedical and Health Informatics},
  volume={26},
  number={10},
  pages={5189--5200},
  year={2022},
  publisher={IEEE}
}

Contact

Please contact us here if you have any question.