matrixgame2018 / monocular-depth-estimation-toolbox

Monocular Depth Estimation Toolbox based on MMSegmentation.

Monocular-Depth-Estimation-Toolbox

Introduction

Monocular-Depth-Estimation-Toolbox is an open source monocular depth estimation toolbox based on PyTorch and MMSegmentation v0.16.0.

It aims to benchmark MonoDepth methods and provides effective supports for evaluating and visualizing results.

Major features

• Unified benchmark

  Provide a unified benchmark toolbox for various depth estimation methods.

• Modular design

  Depth estimation frameworks are decomposed into different components. One can easily construct a customized framework by combining different modules.

• Support of multiple methods out of box

  I would like to reproduce some of the most excellent depth estimation methods based on this toolbox.

• High efficiency

  It seems that there are few depth estimation benchmarks, so I start this project and hope it is helpful for research.

Thanks to MMSeg, we own these major features. 😊

Benchmark and model zoo

Results and models are available in the model zoo.

Supported backbones (partially release):

• ResNet (CVPR'2016)
• EfficientNet (ICML'2019)
• Vision Transformer (ICLR'2021)
• Swin Transformer (ICCV'2021)
• I recommend cross-package import in config, so that you can utilize other backbone in MMcls, MMseg, etc. Refer to introduction. I will add more backbones in the future.

Supported methods:

• BTS (Arxiv'2019)
• Adabins (CVPR'2021)
• SimIPU (AAAI'2022)
• DPT (ICCV'2021) (TBD)
• NeWCRFs (CVPR'2022) (TBD)
• DepthFormer (Arxiv'2022)
• BinsFormer (Arxiv'2022) (Early release. It may contain bugs or potential performance drop)

Supported datasets:

• KITTI
• NYU-Depth-V2
• SUN RGB-D (Eval)
• Cityscapes

Installation

Please refer to get_started.md for installation and dataset_prepare.md for dataset preparation.

Get Started

We provide train.md and inference.md for the usage of this toolbox.

License

This project is released under the Apache 2.0 license.

Acknowledgement

This repo benefits from awesome works of mmsegmentation, Adabins, BTS. Please also consider citing them.

Cite

@misc{lidepthtoolbox2022,
  title={Monocular Depth Estimation Toolbox},
  author={Zhenyu Li},
  howpublished = {\url{https://github.com/zhyever/Monocular-Depth-Estimation-Toolbox}},
  year={2022}
}

Changelog

• Nov. 29, 2022: Release the config for BinsFormer on KITTI. Release configs for cross-dataset inference (nyu2sunrgbd). Fix NYUv2 download bugs as discussed in Issue #66 and #36. I add several codes for NeWCRFs but have no plan to conduct detailed experiments for it due to the current limited resource and time.
• Jul. 23, 2022: Release codes for BinsFormer. Potential bugs or performance drops can exist, waiting for completion.
• Jun. 5, 2022: Add support for custom dataset training. Add a warmup interface for sigloss to help convergence as discussed in Issue #20. Enhance the DPT support and fix bugs in provided pre-trained models as reported in Issue #23.
• Apr. 16, 2022: Finish most of docs and provide all pre-trained parameters. Release codes about BTS, Adabins, DPT, SimIPU, and DepthFormer. Support KITTI, NYU-v2, SUN RGB-D(eval), and CityScapes.

TODO

• I will complete the release of BinsFormer models on KITTI dataset (currently limited by the GPU resource).
• I would like to include self-supervised depth estimation methods, such as MonoDepth2.