Questions about the code in dex_dataset.py. about unidexgrasp

[mzhmxzh]

To generate random scenes with an object on top of a table, we drop every object onto the table 100 times randomly. This will generate 100 affine transform matrices for each objects, which are recorded in poses.npy. When you apply a matrix to some object, the object will be rotated and translated so that it lies stably on the z=0 plane. pcs_table.npy records the 100 table-top scenes in the form of point clouds, with the first 3000 points sampled from the object, and the last 1000 points from the table using furthest point sampling. You can visualize the point cloud in this script, change --num to an integer in [0, 100) to visualize different poses in our object dataset.

However, when we generate the table-top grasp dataset, we only use the table plane rather than the entire transform matrix. That is to say, for each grasp, we first randomly sample a transform matrix from poses.npy, then extract the table plane Ax + By + Cz + D = 0 in the object reference frame. You can verify that (A, B, C) == pose_matrix[2, :3] and D == pose_matrix[2, 3] * object_scale. As a result, in order to retrieve the correspondence between a grasp and a pose matrix, we need to find the matrix with the smallest |(A, B, C) - pose_matrix[2, :3]| in poses.npy. Use this script to visualize a grasp.

In retrospect, there could be better ways to do this, but the code was running fine so we didn't find it necessary to change it...

[wyl2077]

Thank you very much. So, the 'plane' in grasp file (e.g. "pose/core/bottle-asdasfja12jaios9012/00000.npz") is the table plane of the correspondence object pose. Is that correct?

[mzhmxzh]

Yes.