Question about surfel position optimization about badslam

[puzzlepaint]

Hi, the computation of the Jacobian wrt. t is stated below. It leads to the result in the code, so I believe it is correct. If not, please point out the error.

The variable naming could definitely be improved however. I believe that the geometric residual was first called 'depth residual' and this has only partially been replaced in the code.

Derivative computation
The "depth" / geometric residual is, taken from ComputeRawDepthResidual() in cost_function.cuh:

raw_residual_inv_stddev_estimate * Dot(surfel_local_normal, *local_unproj - surfel_local_position)

If we insert the update of surfel_local_position with t there, we get:

raw_residual_inv_stddev_estimate * Dot(surfel_local_normal, *local_unproj - (surfel_local_position + t * surfel_local_normal))

To take the derivative without knowing about vector calculus, let's reformulate this with scalars only, and abbreviate the terms:

inv_stddev = raw_residual_inv_stddev_estimate
(nx, ny, nz)^T = surfel_local_normal
(ux, uy, uz)^T = *local_unproj
(sx, sy, sz)^T = surfel_local_position

This yields:

inv_stddev * (nx * (ux - (sx + t * nx)) + ny * (uy - (sy + t * ny)) + nz * (uz - (sz + t * nz)))

Then drop all constant terms that are not relevant for the derivative wrt. t:

inv_stddev * (-t * nx * nx - t * ny * ny - t * nz * nz)

Take the derivative:

inv_stddev * -1 * (nx * nx + ny * ny + nz * nz)

Since the normal is supposed to have length 1, this can be simplified to:

-inv_stddev

[puzzlepaint]

In principle yes, since surfels are always compared to the pixel that they project to, and this pixel might change. However, as long as a surfel projects to the same pixel, the corresponding unprojected point and the related standard deviation stays constant in the formulation that the system uses. Since the changes between different pixels only happen step-wise and we could not compute any derivative for those steps anyway since the function is discontinuous there, the derivative computation assumes that the surfel position is not at such a discontinuous point, and then the pixel remains constant for the purpose of local derivative computation.

I guess it would be nicer to treat the depth map as a continuous surface in places where it actually is continuous, to prevent potential issues at discontinuities (such as oscillation of Gauss-Newton). However, it is always hard to decide what is a discontinuity and what is not.

[wlsh24]

Thank you so much for the detailed answer, I really appreciate that.

I agree with your derivative computation but there is something that is making me confused.

In your computation, you assume that the terms local_unproj and inv_stddev are not related to t.
Aren't they also depending on the surfel_local_position and thus on the parameter t ?

[wlsh24]

I see, thank you for the clarification :)

[wlsh24]

Hello, I am still trying to understand the geometry optimization part and there is something troubling me during the accumulation of H when both the position and the descriptors are optimized.

It looks like H(0,0) is equal to the sum of the squared value of the jacobian of the depth (L154), position1, and position2 (L216).
I feel it should be the squared value of the sum instead since the jacobian of the cost function with respect to t yields in a sum. Or am I missing something ?

[puzzlepaint]

Note that all of the three Jacobians that are summed here are taken wrt. t, but for three different residuals: the geometric residual, and the two components of the photometric term (the latter differs from the paper, where the photometric residual only has one component). Again, the variable naming in the code is not optimal.

As for why the different Jacobians are squared and then summed, please refer to any derivation of the Gauss-Newton method, for example that of g2o.

[wlsh24]

Thank you for the reference, I understood my mistake...