Prediction of 3D boxes format about omni3d

[longbowzhang]

Actually I have a similar question as (3) in yours. Then I dive a little bit deep in the codes and find the following

omni3d/cubercnn/modeling/roi_heads/roi_heads.py

Lines 115 to 118 in 47580fd

	if self.dims_priors_enabled and priors is not None:
	self.priors_dims_per_cat = nn.Parameter(torch.FloatTensor(priors['priors_dims_per_cat']).unsqueeze(0))
	else:
	self.priors_dims_per_cat = nn.Parameter(torch.ones(1, self.num_classes, 2, 3))

It looks like those per-category means are learned.

[ofir1080]

Thanks
however, in the paper:

How do you interpret that?

[gkioxari]

Hi all!

Let me answer your questions, and expand a bit for pedagogical reasons!

Per-category $(w_0, h_0, l_0)$ means.

As you have noticed we have support for a few settings. We treat the $(w_0, h_0, l_0)$ as parameters, initialized fro scratch or from training ground truth information, which can be finetuned or frozen. In our default setting, the means $(w_0, h_0, l_0)$ are pre-computed from the training ground truth (see here) and are not further learned (aka frozen) (see here). Interestingly, we found that learning them from scratch was just as good but would slow down convergence and model training, as you'd expect!

Is $(u,v)$ in pixel space?

Yes. $(u, v)$ is the projected 3D object center on the image plane. So if $(X,Y,Z)$ is the object's 3D center, $(u,v)$ is its projection on the image plane using the camera matrix $K$.

Virtual depth

The focal length for each image is given. The focal length is a camera parameter that provides information about scale. The focal length $f$ varies per image of course. So what virtual depth does, is that it projects the real (world) space to a scale-invariant space which is much friendlier for neural nets to cast predictions in.

Allocentric vs Egocentric

That's right. Poses are stored in egocentric format and can be converted in allocentric ones (see our appendix). We predict allocentric poses.

[Sondosmohamed1]

@gkioxari Thank you very much for your work and clarification, i have question regarding the camera rotation and location is the model is affected with camera rotation and location (external matrix ), because i generated custom dataset from different cameras ,and i rotated the cameras and put them in different locations , then for every camera and object i generated the R_cam from roll , pitch ,yaw matrices with respect to camera ( the difference between the camera rotation and the object rotation) in every axis , my question is what i did is right , or this rotation of the camera i made will affect the predication