questions about dataset format about omni3d

[garrickbrazil]

Hello!

1. The bbox2D_trunc field is the minimum enclosed projected 3D bounding box which is truncated to the image bounds when necessary. We tend to prefer this for training instead of bbox2D_proj which can result in very large boxes as z3d approaches 0.

2. The R_cam matrix is a per object annotation which rotates an object in the camera space around its own origin (hence before any 3D translation to the cuboid, seen here). I'm not sure if the concept of world to camera is as applicable here but I see how the name could imply that. Hope that answers your question or feel free to ask more!

3. To get the form from 8 projected vertices (8 x 3 Tensor), the general process is to take the min/max for each dim similar to:

x1 = vertices[:, 0].min(1)
y1 = vertices[:, 1].min(1)
x2 = vertices[:, 0].max(1)
y2 = vertices[:, 1].max(1)

[SyanneL]

Thanks so much for your reply! I am also wondering how could I get the truncation?

[WangYueFt]

Hey @garrickbrazil , I have a follow up question regarding bbox2D_trunc and bbox2D_proj. Is bbox2D_trunc truncated to image bound from bbox2D_proj? How do you get the minimal enclosed projected 3D bounding box? Thanks!

[gkioxari]

Hi @WangYueFt

To get the minimal enclosed 2D bounding box we use the help of rendering.
We represent the 3D bounding box as a mesh: 8 vertices and 12 triangular faces. Then we render the 3D box mesh with Py3D which renders the silhouette of the mesh and take the minimum enclosed bounding box of the rendered silhouette. Note that via rendering, the part of the box that is outside of the image plane is not rendered thus the resulting box is truncated, as intended.