How to correct the distortion in D405, I found that when the object is too large or not in the center, the x,y coordinates of the detected point are not quite correct about librealsense

[MartyG-RealSense]

Hi @ybefe The D405 camera model is designed for close-range depth sensing, with an ideal depth range of 7 cm to 50 cm. The D405 therefore has a different default depth scale value (0.01) to the rest of the models in the RealSense 400 Series range (which have a default depth scale of 0.001).

If you are using the camera at a distance from the object then you could therefore try using the rs.option.depth_units instruction to change the depth scale to 0.001 to see whether it makes a difference.

depth_sensor.set_option(rs.option.depth_units, 0.001)

The accuracy of the x and y coordinates can also drift when moving away from the center if the get_distance() instruction is used in the program script.

[ybefe]

I did use get_distance() to get the 3D coordinates of the point, so how do I fix this? @MartyG-RealSense

[MartyG-RealSense]

Are you performing depth to color alignment before calling the get_distance() instruction?

[ybefe]

align = rs.align(rs.stream.color)
frames = pipeline.wait_for_frames()
aligned_frames = align.process(frames)
aligned_color_frame = aligned_frames.get_color_frame()
aligned_depth_frame = aligned_frames.get_depth_frame() I think I'm supposed to be aligned

[MartyG-RealSense]

Yes, it is correct to use depth to color alignment. Either the aligned intrinsics or the color intrinsics should be used, as using the depth intrinsics can cause the inaccuracy at the outer edges of the image that you experienced. This is because the origin point of depth changes to the color stream's origin point when depth to color alignment is performed.

[ybefe]

So how to use the aligned intrinsics or the color intrinsics

[MartyG-RealSense]

A Python get_distance() script at the link below that uses color intrinsics should be a helpful reference.

https://support.intelrealsense.com/hc/en-us/community/posts/23541262437011/comments/23566066750355

color_intrin = aligned_color_frame.profile.as_video_stream_profile().intrinsics

[ybefe]

align = rs.align(rs.stream.color)->align = rs.align(rs.stream.depth)
dis, camera_coordinate = get_3d_camera_coordinate(depth_pixel, aligned_depth_frame, depth_intrin)->
dis, camera_coordinate = get_3d_camera_coordinate(depth_pixel, aligned_depth_frame, color_intrin)
It's probably just using color alignment, and then using the camera parameters to find the 3D coordinates,right?

[MartyG-RealSense]

When using align_to(), the alignment calculations are handled by a software routine in the SDK called the align processing block that performs automated adjustments, such as adjusting for differences in resolution between the streams or resizing the depth's field of view size to match the field of view size of color.

Things are a little more complicated on the D405 model, as it does not have a dedicated RGB sensor and its color is provided by the depth sensor and passed through an Image Signal Processor (ISP) chip to produce the final color image. The color intrinsics should still be different from the depth intrinsics though.

[ybefe]

So I should use align = rs.align(rs.stream.color) and color_intrin ,right?

[MartyG-RealSense]

Yes, align to color and use color intrinsics.

[ybefe]

Thank you very much!