Depth Value Discrepancies about pyk4a

[JJLimmm]

Issue thread regarding this question.

[nliedmey]

Hey @JJLimmm,
have you found a valid solution for your problem?

I am currently working on a comparable project and struggle to get valid depth vales for my 2D-RGB Keypoints as well.

The first try I did was taking the transformed_depth_image and just look up the depth value at [x,y]-coordinate of the RGB pose keypoint. At first it looked good, but some further investigations showed off, that this process has a major issue:

The depth value, that is returned by the transformed_depth_image, is not the true depth (euclidean distance) from the object to the sensor, but it already is the Z-coordinate from one of the both 3D-coordinate systems. Some tests showed off this issue, even though on of the Azure Kinect developers wrote it differently: (Github Issue).

When now converting [x,y] (2D image plane) and Z (3D coordinate system) to a set of 3D coordinates, by hand or with the help of calibration.convert_2d_to_3d() function, the resulting coordinates do not match the reality due to the miss interpretation of the given depth value. In my case, the hip of the 3D person is tilted forward towards the camera because it is placed approximately on the Z-coordinate line.

Have you encountered similar anomalies and how did you solved your problems? :)

[JJLimmm]

Hey @nliedmey , yes i noticed this discrepancy as well between using the calibration from Azure SDK and also our own conversion. Currently i cannot find a way to create a custom calibration to obtain the depth at the 2D keypoint coordinate that i want, it will take up a lot of time and i dont have the luxury of that right now.

For my project, i'm just taking the obtained depth value in the 2D RGB space at face value, and if it shows an invalid depth, i will skip that frame and obtain the next.