gabrielmajeri / nyuv2-python-toolbox

Python toolbox for the NYU Depth Dataset V2

Python Toolbox for the NYU Depth Dataset V2

Description

Python support library for using the NYU Depth Dataset V2.

The official toolbox for processing the raw dataset is written in MATLAB. This repository contains a Python port of the toolbox.

Project status

The port of the toolbox is not complete. I'm no longer actively adding new features on this project, but if you find this library useful, feel free to fork it, reuse it and/or contribute back to it.

Features

This library currently contains code for:

• Higher-level interface to the labeled subset
• Raw dataset extraction and preprocessing
• Performing data augmentation

License

The original toolbox was written by the authors of the dataset: Nathan Silberman, Pushmeet Kohli, Derek Hoiem, Rob Fergus.

This code is licensed under the MIT license. See the LICENSE file for the full license text.

nyuv2-python-toolbox's Issues

generate RGB images

Hi! Thanks for your contributions!
Would you please tell me how to generate synchronized RGB and Depth frames by this toolbox, corresponding to the get_synched_frames.m from the original one?

ModuleNotFoundError: No module named 'nyuv2.raw.project'

when I run the test file I get this error,
ModuleNotFoundError: No module named 'nyuv2.raw.project'

what should project file contain. I feel lost without documentation. it would help me if you give me some guidance in this.
Thank you

I got the following error when running the test

C:\Users\ZAK\Pytorch working folder\MY_THESIS_DEPTH_ESTIMATION\nyuv2-python-toolbox\nyuv2\labeled.py:17: H5pyDeprecationWarning: The default file mode will change to 'r' (read-only) in h5py 3.0. To suppress this warning, pass the mode you need to h5py.File(), or set the global default h5.get_config().default_file_mode, or set the environment variable H5PY_DEFAULT_READONLY=1. Available modes are: 'r', 'r+', 'w', 'w-'/'x', 'a'. See the docs for details.
self.depth_maps = getDepthImages(self.file)

KeyError Traceback (most recent call last)
in
----> 1 test_labeled_dataset()
2 #test_raw_dataset()

in test_labeled_dataset()
14
15 def test_labeled_dataset():
---> 16 labeled = LabeledDataset(DATASET_DIR / 'nyu_depth_v2_labeled')
17
18 color, depth = labeled[42]

~\Pytorch working folder\MY_THESIS_DEPTH_ESTIMATION\nyuv2-python-toolbox\nyuv2\labeled.py in init(self, path)
16 """Opens the labeled dataset file at the given path."""
17 self.file = h5py.File(path)
---> 18 self.color_maps = self.file['images']
19 self.depth_maps = self.file['depths']
20

h5py_objects.pyx in h5py._objects.with_phil.wrapper()

h5py_objects.pyx in h5py._objects.with_phil.wrapper()

C:\Anaconda3\Anaconda3\lib\site-packages\h5py_hl\group.py in getitem(self, name)
262 raise ValueError("Invalid HDF5 object reference")
263 else:
--> 264 oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
265
266 otype = h5i.get_type(oid)

h5py_objects.pyx in h5py._objects.with_phil.wrapper()

h5py_objects.pyx in h5py._objects.with_phil.wrapper()

h5py\h5o.pyx in h5py.h5o.open()

KeyError: "Unable to open object (object 'images' doesn't exist)"

Project depth onto RGB

Hello, your python port has been very useful so far. I was wondering if you ever implemented the part that processes the raw data, specifically projecting the raw depth values onto the rgb image. Basically what is left out here:

nyuv2-python-toolbox/test.py

Line 62 in 1ad502c

# TODO: project depth and RGB image

Is there any progress on this? How did you overcome this?
Thanks in advance.

RGB and Depth is not aligned

I think current codebase doesn't align depth maps with RGB images. In Matlab code, there are functions to align the image and depth map, is there any plan to implement them in python?

Incorrect frame sync

The frame synchronization method is missing a very important line, you need to add:

diff = new_diff

somewhere after the if, around here:

nyuv2-python-toolbox/nyuv2/raw/extract.py

Line 87 in 1ad502c

Otherwise, the timedelta never gets updated.