liviniuk / dorn_depth_estimation_pytorch

Pytorch implementation of "Deep Ordinal Regression Network for Monocular Depth Estimation" paper by Fu et. al.

DORN_depth_estimation_Pytorch

This is an unoficial Pytorch implementation of Deep Ordinal Regression Network for Monocular Depth Estimation paper by Fu et. al.

Table. Performance on NYU V2.

Source	δ1	δ2	δ3	rel	log10	rms
Original paper*	0.828	0.965	0.992	0.115	0.051	0.509
This repo*	0.806	0.957	0.989	0.151	0.062	0.586

*Note, that the data splits are different (see Known Differences below for details). The worse performance might be due to the smaller training set (795 vs about 120K images).

How to use

These steps show how to run the code on the official split of the NYU V2 depth dataset.

To prepare data:

• Download nyu_depth_v2_labeled.mat (data) and splits.mat.
• Edit create_nyu_h5.py to add data_path (folder with the .mat files from previous step) and output_path.
• Run:

python create_nyu_h5.py

For start training on NYU V2 run:

train.py [-h] [--dataset DATASET] [--data-path DATA_PATH]
              [--pretrained] [--epochs EPOCHS] [--bs BS] [--bs-test BS_TEST]
              [--lr LR] [--gpu GPU]

Or simply:

python train.py --data-path DATA_PATH --pretrained

(where DATA_PATH is same as output_path used during preparing data).

For more info on arguments run:

python train.py --help

To train on a different dataset, implementation of the DataLoader is required.

To monitor training, use Tensorboard:

tensorboard --logdir ./logs/

Known Differences

The implementation closely follows the paper and the official repo with some exceptions. The list of known differences:

• Only training on the labeled part of NYU V2 is currently implemented (not on all the raw data).
• ColorJitter is used instead of the color transformation from the Eigen's paper.
• Feature extractor is pretrained on a different dataset.

Pretrained feature extractor

DORN uses a modified version of ResNet-101 as a feature extractor (with dilations and three 3x3 convolutional layers in the begining instead of one 7x7 layer). If you select pretrained=True, weights pretrained on MIT ADE20K dataset will be loaded from this project. This is different from the paper (the authors suggest pretraining on ImageNet). That is the only suitable pretrained model on the Web that I am aware of.

Requirements

• Python 3
• Pytorch (version 1.3 tested)
• Torchvision
• Tensorboard

Acknowledgements

The code is based on this implementation.

dorn_depth_estimation_pytorch's Issues

someting wrong

Traceback (most recent call last):
File "", line 1, in
Traceback (most recent call last):
File "train.py", line 56, in
File "D:\Users\Administrator\Anaconda3\envs\VNL\lib\multiprocessing\spawn.py", line 105, in spawn_main
for i, (input, target) in enumerate(train_loader):
File "D:\Users\Administrator\Anaconda3\envs\VNL\lib\site-packages\torch\utils\data\dataloader.py", line 278, in iter
exitcode = _main(fd)
File "D:\Users\Administrator\Anaconda3\envs\VNL\lib\multiprocessing\spawn.py", line 114, in _main
return _MultiProcessingDataLoaderIter(self)
File "D:\Users\Administrator\Anaconda3\envs\VNL\lib\site-packages\torch\utils\data\dataloader.py", line 682, in init
prepare(preparation_data)
File "D:\Users\Administrator\Anaconda3\envs\VNL\lib\multiprocessing\spawn.py", line 225, in prepare
_fixup_main_from_path(data['init_main_from_path'])
w.start() File "D:\Users\Administrator\Anaconda3\envs\VNL\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path

run_name="__mp_main__")

File "D:\Users\Administrator\Anaconda3\envs\VNL\lib\multiprocessing\process.py", line 112, in start
File "D:\Users\Administrator\Anaconda3\envs\VNL\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
self._popen = self._Popen(self) File "D:\Users\Administrator\Anaconda3\envs\VNL\lib\runpy.py", line 96, in _run_module_code

mod_name, mod_spec, pkg_name, script_name)

File "D:\Users\Administrator\Anaconda3\envs\VNL\lib\multiprocessing\context.py", line 223, in _Popen
File "D:\Users\Administrator\Anaconda3\envs\VNL\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
return _default_context.get_context().Process._Popen(process_obj) File "C:\DORN_depth_estimation_Pytorch-master\train.py", line 18, in

parser = argparse.ArgumentParser(description='DORN ')

File "D:\Users\Administrator\Anaconda3\envs\VNL\lib\multiprocessing\context.py", line 322, in _Popen
NameError: name 'argparse' is not defined
return Popen(process_obj)
File "D:\Users\Administrator\Anaconda3\envs\VNL\lib\multiprocessing\popen_spawn_win32.py", line 89, in init
reduction.dump(process_obj, to_child)
File "D:\Users\Administrator\Anaconda3\envs\VNL\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
BrokenPipeError: [Errno 32] Broken pipe

The question of Subjective effect

Hello，your code help me a lot,thank u very much
I have a question about subjective effect.The RMSE is great,but the prediect depth has a block effect.Do you konw how to solve this effect?

The Peformance on KITTI

Hi,

Thank you for providing this PyTorch implementation.

Have you tested the accuracy performance of this code on KITTI?

Many thanks

Depth results on NYUv2

First of all, Thank you very much for sharing this code. It is an improved version of the former implementation.

I have tested the NYUv2 values, and I got the results you publish but for TRAINING, not for testing. Is that right? In the DORN paper, these values refer to the TESTING set.

Is it that correct?
As you mention the amount of images can be a reason of the difference.

D1, D2, D3 gradually become smaller

Thank you for your code, it's very useful to me. But in the training process, D1, D2, D3 gradually become smaller. What's the problem?

Out of memory exception.

Hi, I found really usefull for implementaion and i am facing an out of memory exception when i learn in a bigger Dataset. I did not change the model implementation and i tracked the memory over the batches.

Since you are more familiar with the code, where do u think that a memory leak could happen? I will investigate this if i find something I will post it here.