subfocal's Introduction

(TCI 2023) Learning Sub-Pixel Disparity Distribution for Light Field Depth Estimation

Network Architecture

SOTA on 4D Light Field Benchmark

Our method ranks the first place on the HCI 4D LF Benchmark in terms of all the five accuracy metrics (i.e., BadPix0.01, BadPix0.03, BadPix0.07, MSE and Q25).

For more detail comparison, please use the link below.
Benchmark link

Environment

Ubuntu            16.04
Python            3.8.10
Tensorflow-gpu    2.5.0
CUDA              11.2

Two-Stage Train SubFocal

Download HCI Light field dataset from http://hci-lightfield.iwr.uni-heidelberg.de/.
Unzip the LF dataset and move 'additional/, training/, test/, stratified/ ' into the 'hci_dataset/'.
Stage 1: Run python train_sub.py

Checkpoint files will be saved in 'LF_checkpoints/XXX_ckp/iterXXXX_valmseXXXX_bpXXX.hdf5'.
Training process will be saved in
- 'LF_output/XXX_ckp/train_iterXXXXX.jpg'
- 'LF_output/XXX_ckp/val_iterXXXXX.jpg'.

Stage 2: Run python train_sub_js.py

Satge 1 model as pretrained, finetune load_weight_is=True
path_weight='LF_checkpoint/SubFocal_sub_0.5_ckp/iter0049_valmse0.845_bp2.04.hdf5'

Evaluate SubFocal

Run python evaluation.py
- path_weight='LF_checkpoint/SubFocal_sub_0.5_js_0.1_ckp/iter0010_valmse0.768_bp1.93.hdf5'

Submit SubFocal

Run python submission.py
- path_weight='LF_checkpoint/SubFocal_sub_0.5_js_0.1_ckp/iter0010_valmse0.768_bp1.93.hdf5'

Citation

@ARTICLE{chao2023learning,
  author={Chao, Wentao and Wang, Xuechun and Wang, Yingqian and Wang, Guanghui and Duan, Fuqing},
  journal={IEEE Transactions on Computational Imaging}, 
  title={Learning Sub-Pixel Disparity Distribution for Light Field Depth Estimation}, 
  year={2023},
  volume={},
  number={},
  pages={1-12},
  doi={10.1109/TCI.2023.3336184}}

Last modified data: 2022/08/18.

The code is modified and heavily borrowed from LFattNet: https://github.com/LIAGM/LFattNet

The code they provided is greatly appreciated.

subfocal's People

Contributors

Stargazers

Watchers

subfocal's Issues

关于4D LF benchmark数据集的问题

你好，师兄。能问您一个问题吗？关于4D LF benchmark数据集的其中的test文件下的bedroom,bicycle,herbs,origami四个场景gt的pfm文件怎么获得呢？如果没获取的话，那些bp007和mse指标怎么计算得到的呢？

Hi, I use your checkpoint 'LF_checkpoint/SubFocal_sub_0.5_js_0.1_ckp/iter0010_valmse0.768_bp1.93.hdf5' to evaluate, but I cannot reach the accuracy in the paper, here is my result(badpixel 0.07):
backgammon:96.947
dots: 97.4974
pyramids: 97.7432
stripes: 97.5043
boxes:97.0235
cotton:98.0419
dino: 97.4948
sideboard: 98.3075

Recommend Projects

chaowentao / subfocal Goto Github PK