Datasets from mono and stereo cameras about ossdc-visionbasedacc

[mslavescu]

We'll need some algorithms to align and synchronize the data (image/sensors) to maximize the accuracy of depth map and Visual SLAM (with IMU).

If you like to help please try to create a Google Colaboratory version also.

The dataset was collected between 17:36 and 18:06 EST, so it was almost dark and also was snowing lightly.

Lets see how robust are the stereo depth estimation, ego motion/speed estimation, and object detection/instance segmentation algorithms on this fairly difficult dataset

Would be interesting to use instance segmentation and then calculate disparity at contour level.

The images have noise (it was fairly dark), and the snow/wiper gets in the way too, so lets see if we could improve the accuracy of instance segmentation in these situations.

The main target would be to test #1-mvp-vbacc and create a 3D map (also correlating compass​, IMU, OBD II speed) to help #2-mvp-lkas (the lane markings are covered by snow).

[mslavescu]

For stereo (multi) camera depth map estimation, check this excellent thesis to also see how #fpga can help creating #3-mvp-smart-camera from monocular to multi camera setups, and what configurations are good for certain scenarios (number of cameras, resolution, fps) :

https://twitter.com/GTARobotics/status/875932414996094976

[ppirrip]

I dug a bit deeper on the topic / rosmsg type expected from the imu:

/Imu
std_msgs/Header header
uint32 seq
time stamp
string frame_id
geometry_msgs/Quaternion orientation
float64 x
float64 y
float64 z
float64 w
float64[9] orientation_covariance
geometry_msgs/Vector3 angular_velocity
float64 x
float64 y
float64 z
float64[9] angular_velocity_covariance
geometry_msgs/Vector3 linear_acceleration
float64 x
float64 y
float64 z
float64[9] linear_acceleration_covariance

It take a quaternion for orientation.

For fun I took a look at the node graph also:

The author recommends Kalibr for stereo calibration (another ros program expects a bag input

[mslavescu]

I'll try to build a camera like this and use it for datasets collection also:

https://github.com/harshatech2012/dual-raspi-stereo-vision

[mslavescu]

Lets try to run Kalibr in Colaboratory.

We just need to build a publisher with images/IMU, from the dataset, that is expected by these ROS calibration/disparity estimation tools.

[mslavescu]

We should try this also, from Skydio team:

End-to-End Learning of Geometry and Context for Deep Stereo Regression
https://arxiv.org/abs/1703.04309

[mslavescu]

We should try and test it with this new approach also:

Dataset and Methods for Deep inertial Odometry
http://deepio.cs.ox.ac.uk/