Using sparse depth maps about lessmore

[ebrach]

Hey,

the constD trick is a very crude tool that's only justified if you have no information about the correct depth. In your case, there is quite a lot of information available (even if it is sparse). I would try to create dense maps from your sparse maps using interpolation / hole filling methods, and train with those dense maps. Even if they are not perfect, training stage two (optimizing reprojection error) will probably fix errors in many places.

As a first quick and dirty check, you could just do nearest neighbor interpolation of your sparse depth (assign the depth of the nearest pixel with a value). Maybe do early stopping in training stage one to avoid overfitting to these interpolated depth maps.

Kind regards,
Eric