masataka46 / multimodaldl

This model implementation of 'Multimodal Deep Learning for Robust RGB-D Object Recognition'

Multimodal Deep Learning for Robust RGB-D Object Recognition

Requirements

• Pillow (Pillow requires an external library that corresponds to the image format)

Description

This is an implementation of 'Multimodal Deep Learning for Robust RGB-D Object Recognition'. It requires the training and validation dataset of following format:

• Each line contains one training example.
• Each line consists of two elements separated by space(s).
• The first element is a path to 256x256 RGB image.
• The second element is its groundtruth label from 0 to arbitrary.

The text format is equivalent to what Caffe uses for ImageDataLayer.

This example requires "mean file" which is computed by compute_mean.py.

This example also requires CaffeNet model 'bvlc_reference_faffenet.caffemodel' sited at http://dl.caffe.berkeleyvision.org/

So, you must to download its model before implement training.

The process to train is follow:

1. command 'python train_rgb_d.py' with color datas.
2. command 'python train_rgb_d.py' with depth datas.
3. command 'python train_full.py' with color datas and depth datas.

multimodaldl's Issues

Some question about the code of MultimodalDL

Hello！ I am a graduate student of UESTC in China, I haved dowmloaded your code about the MultimodalDL, and want to run it.But I meet a trouble, when I run the file "train_rgbd.py", it need load the caffemodel, and you use the function "serializers.load_npz",in the "load_npz" function , it try to load the caffemodel by "numpy.load", and the "numpy.load" can't be used to load the caffemodel, it just can load the file about "npz" 、"npy" format and so on. So when I run the code ,it throw the error "Failed to interpret file 'bvlc_reference_caffenet.caffemodel' as a pickle". I just want to ask you, how can you do to change the caffemodel to the "npz" or "npy" format? or maybe I forget do something for this code ?
Hope you can anwser my quetion , Sincerely thanks!