pondruska / deeptracking

Source code of DeepTracking research project

DeepTracking: Seeing Beyond Seeing Using Recurrent Neural Networks

This is an official Torch 7 implementation of the method for the end-to-end object tracking from occluded sensor measurements using neural network presented in the academic paper:

P. Ondruska and I. Posner, "Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks", in The Thirtieth AAAI Conference on Artificial Intelligence (AAAI), Phoenix, Arizona USA, 2016.

• author: Peter Ondruska, Mobile Robotics Group, University of Oxford
• email: ondruska(at)robots.ox.ac.uk
• paper: http://www.robots.ox.ac.uk/~mobile/Papers/2016AAAI_ondruska.pdf
• webpage: http://mrg.robots.ox.ac.uk/

For any questions about the code or the method please contact the author.

Installation

Install Torch 7 and the following dependencies (using luarocks install [package]):

• nngraph
• image
• cunn (optional for training on a GPU)

Data

Download and unzip the training data for the simulated moving balls scenario:

http://mrg.robots.ox.ac.uk:8080/MRGData/deeptracking/DeepTracking_1_1.t7.zip

This is a native Torch 7 file format.

Training

To train the model run:

th train.lua

Training of the neural network using provided data takes about 12 hours on Nvidia Titan X. Every 1000 iterations the training error is logged to log_model.txt, network weights are saved to weights_model and the visualisation of its performance is stored to video_model.

Optional parameters

Flag	Description
-gpu [id]	use GPU [id] (0 to use CPU)
-model [file]	neural network model
-data [file]	data for training
-iter [number]	the number of training iterations
-N [number]	the length of training sequences
-learningRate [number]	learning rate
-initweights [file]	initial weights
-grid_[minX/maxX/minY/maxY/step] [number]	2D occupancy grid parameters
-sensor_[start/step]	1D depth sensor parameters

License

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.

Release Notes

Version 1.0

• Original version from the academic paper.

Version 1.1

• Native decoding of raw 1D depth data into 2D input.
• Larger NN network.

deeptracking's Issues

Deep Tracking on the Move

I have read through your paper "Deep Tracking on the Move: Learning to Track the World from a
Moving Vehicle using Recurrent Neural Networks" https://arxiv.org/pdf/1609.09365.pdf and want to apply it for a project of mine.
I understand that the basic Spatial Convolution Layers have been replaced by Dilated Convolutions and instead of using Vanilla RNNs, GRUs are being used.
What I am confused about is the how the Spatial Transformation Module is being integrated into the network.
It would be really helpful if you could give me some insight on the same or if there is an open source implementation of yours that I can have a look at.

Is there pytorch version?

Sorry. Is there pytorch version? torch 7 came to its end of the life. pytorch is more popular now.

Tracking

1). I am unable to get how are you tracking an object? I can't get the tracking part.
2). Please describe WeightedBCECriterion.lua file as I am unable to get the function of this.

What are the next steps after training

Hi, @pondruska ,

After the training (almost it spent 13 hours in my system), what are the next steps to use the trained model?

Thanks~
Milton

Question the section "Unsupervised Training"

I have read your paper "Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks". but I don't understand the approach called “unsupervised training”. In the section “Unsupervised Training”, you said "we propose to train the network not to predict the current state , but a state in the future" , can you tell me the difference between them ? Thank you.
And is the Equation 7 used in the unsupervised training ?
Thank you.

Testing model use our own dataset

I want to ask how to use the weights to be applied to our own dataset? Thank you