Error messages and tuning when transplanting R-VIO to non-ROS setup about r-vio

[huaizheng]

Those messages are particular for informing, so if you keep seeing them then that means something out of this codebase may be wrong, as you said the extrinsic parameters (including the temporal offset). While the title of your issue @jeffcyu may confuse the other users because actually this is caused by your transplant attempt, not an issue about the R-VIO itself.

[jeffcyu]

Thanks for the very quick reply!

I apologize for the title, hopefully this new title is more reasonable.

So just to confirm what you're saying: the error messages that are showing are probably due to some incorrect parameter like the extrinsics or the temporal offset, or from some bug introduced when transplanting the code; is this correct?

Additionally, would you happen to have some intuition on which parameters tend to be the most sensitive? For example, do you find that the extrinsics and temporal offset the greatest culprits in most cases, or is it difficult to generalize?

Lastly, I was curious if extrinsics play a role in the feature tracking? I noticed that there is a rotation matrix that is created with them and then used for ransac. Does this mean that bad extrinsics would also result in a lot of features being tossed during ransac, meaning a symptom of bad extrinsics could be the need to constantly supplement the feature list?

Thanks again for your time,
~Jeff

[huaizheng]

Thanks @jeffcyu, I saw it. The importance of extrinsic parameters is that they connect the visual and inertial information in both front end (tracking) and back end (update). As you got negative depth estimate of feature, I may think this was caused by extrinsic calibration because the information for doing this feature recovery includes the relative IMU poses, feature tracking results, and extrinsic parameters. You need good extrinsic parameters to convert relative IMU poses to relative camera poses, or it would not be surprise to get a landmark being initialized behind the camera and to have those messages come out. Also, my 2-point ransac needs relative camera rotation information, while I can first get relative IMU rotation and then convert it to the camera frame using extrinsic parameters. So if those parameters are not reasonable enough, it may cause big problem, e.g., rejecting good tracks instead. Kalibr is the tool I used for calibration, which can give you both spatial and temporal parameters, while you need to know something about your sensor, such as IMU noise parameters, in order to get good calibration result. Also, please note that it is better to start R-VIO from stationary in order to avoid possible drift caused by my current initialization method.

[jeffcyu]

Hi @huaizheng, thanks for your good suggestions!

I ran my system through Kalibr and generated the intrinsics for the camera as well as the spatial and temporal extrinsics. Now, I'm noticing that the results of the tracking are quite good, but the feature depth parameter (rho) is still frequently becoming negative after the LM solve when performing updates.

I've also noticed that the timestamps from my camera are a bit inconsistent, and images aren't necessarily coming in at a fixed rate; could this be the root of the issue?

Lastly, regarding the Mahalanobis test, what would be the effects of relaxing the chi-squared threshold to something lower?

Thanks again for your time,
~Jeff

[huaizheng]

@jeffcyu, it is normal to have a triangulation result with negative depth. For example, a feature may be tracked from an edge, but this could be either the closest or the farthest edge of a plane depending on the view point, so the depth ambiguity would cause the above result, but it is not used by estimator.

Unless you use some very cheap camera which has very low frame rate, the inconsistency of timestamps may not become a big issue.

The mahalanobis test plays a role in outlier rejection, in addition to the ransac in tracker. Its value reflects the consistency of feature measurements as I show in the paper, and this is why the current value is the degrees of freedom of all the measurements of a single feature.