Get Z coordinate from body joint x and y about pykinect2

[TheSOMGuy]

The solution for the "the function doesn't exist anymore"-Question is quite simple: The pip repository is out of date.
There is a function body_joints_to_depth_space in the actual git branch.
Use it this way:

depth_points = self._kinect.body_joints_to_depth_space(joints)
x = depth_points[PyKinectV2.JointType_Head].x
y = depth_points[PyKinectV2.JointType_Head].y

and it works. But i don't get the z-value.

If i try

if self._kinect.has_new_depth_frame():
self._depth = self._kinect.get_last_depth_frame()
z = self._depth[x,y]

i get a "too many indices for array" exception.

If i try
z = self._depth[ y * self._frame_surface.get_width() + x ]

i get an index out of range exception.
So, what's the correct way to adress the z value in the depth data?

Thanks again for a reply.

[wxy656]

@TheSOMGuy
z = self._depth[ y * 512 + x ] the depth frame is 512*424
But I have another question: is the x,y value color_frame's pixel or a point in coordinate system? and is x,y value's unit consistent with z value's (all meter？)

[TheSOMGuy]

@wxy656

It depends on the function call.
The call depth_points = self._kinect.body_joints_to_depth_space(joints) should always return values in the world coordinate system.
But if i need the z coordinate i would use joint_points = self._kinect.body_joints_to_color_space(joints) for x and y to get it from the image matrix by adressing it via x and y.
Otherwise i obviously can't address the position of the z-value in the image matrix.
That was the intention of my original question.
If the world coordinates don't contain the z value how can i access the real world z-coordinate?
There is no function like z = joint_points[PyKinectV2.JointType_Head].z or z = depth_points[PyKinectV2.JointType_Head].z.

But yes, they should be consistent.

[dhecloud]

@TheSOMGuy Hey, I been reading through your comments, and i figured i would have a better chance asking you since the repo seems to be abandoned.

I am interested in getting the depth images from a kinectv2.

What is the difference between self._kinect.body_joints_to_depth_space and frame = self._kinect.get_last_depth_frame()? Is there a need to use the former if the latter already returns the depth image?
Also, do you have any tips on how to extract only the hand depth image from the full image? Right now i am setting depth values > and < than a threshold. it looks like this
. Not very sure if this is the best method

[nitzel]

@dhecloud If you know a point that definitely belongs to the hand you can start region growing from there, adding nearby pixels that aren't too far away. That way you wouldn't have the two other artefacts that happen to be within your distance-range. You could also configure region growing to have a maximum depth-distance of say 10cm to avoid adding the arm into the region.

[mouarcher]

            if self._bodies is not None:
                if self._kinect.has_new_depth_frame: 
                    for i in range(0, self._kinect.max_body_count):
                        body = self._bodies.bodies[i]
                        if not body.is_tracked: 
                            continue 
                        
                        joints = body.joints 
                        # convert joint coordinates to color space 
                        joint_points = self._kinect.body_joints_to_color_space(joints)
                        self.draw_body(joints, joint_points, SKELETON_COLORS[i])
                        # get the skeleton joint x y z
                        depth_points = self._kinect.body_joints_to_depth_space(joints)
                        x = int(depth_points[PyKinectV2.JointType_ElbowLeft].x)
                        y = int(depth_points[PyKinectV2.JointType_ElbowLeft].y)
                        _depth = self._kinect.get_last_depth_frame()    
                        z = int(_depth[y* 512 + x] )
                        array_x.append(x)
                        array_y.append(y)
                        array_z.append(z)

i want to get the z coordinate, but there is an error (TypeError: 'NoneType' object is not subscriptable ), can anyone helps me~ thanks~

[juanmaro97]

@TheSOMGuy
z = self._depth[ y * 512 + x ] the depth frame is 512*424
But I have another question: is the x,y value color_frame's pixel or a point in coordinate system? and is x,y value's unit consistent with z value's (all meter？)

I do not understand the equation, may you explain me, please? Why 512?

[nitzel]

@juanmaro97
The Kinect camera is taking three different kind of images. One of them is a colour image as you would get from your smartphone (with a resolution of 1920x1080 IIRC), an infrared image (can't remember the resolution) and a depth image with a resolution of 512x424 pixels.

The depth image is a single array of 512*424 numbers, one after another. The first pixel in that array is the pixel in the top left corner. The next pixel in the array is the pixel right next to it.
So if you wanted to iterate over the first row from the image, you'd do for index in range(0, 512): self._depth[index].

Now if you would like to get a specific pixel where you know the row(=y) and column(=x) number of, then you can calculate its index via index = row * 512 + column since you need to sum up the number of pixels in all the rows above and then add the number of pixels to the left of the pixel you want.