Usage of the rotation and translation vectors in the Registration call and the DRR class about diffdrr

[eigenvivek]

Hi @charlesalec, thanks for bringing this to my attention. This is due to an issue with diffdrr.pose.RigidTransform.convert.

diffdrr.drr.DRR and diffdrr.registration.Registration assume that, given a rotation R and a translation t, the camera pose is pose = t.compose(R). Written as a matrix, this looks like [R | Rt]. If you look at a CV textbook like Hartley and Zisserman, they explain that this represents a camera center Rt and a camera orientation R.

In contrast, diffdrr.pose.RigidTransform.convert assumes that the camera pose is pose = [R | t]. Therefore, when it converts a pose into its underlying parameters, it turns your R into Euler angles and returns your t without modification.

[R | t] is the old convention in DiffDRR and I am pretty sure I will make [R | Rt] the new convention going forward (since it directly models how C-arms behave). Thanks for catching this piece of legacy code - will add it to the list of fixes for v0.4.0.

For now, here's a hack that should get you what you need:

# pose : diffdrr.pose.RigidTransform that we want to convert into `R` and `t`
rot, rotxyz = pose.convert("euler_angles", "ZXY")
R = convert(rot, torch.zeros_like(rot), parameterization="euler_angles", convention="ZXY")
xyz = R.inverse()(rotxyz.unsqueeze(0))[0]

reg = Registration(
    drr,
    rot,
    xyz,
    parameterization="euler_angles",
    convention="ZXY", 
)

[charlesalec]

Aha yes, I see it now. You're right, that did indeed fix it. Thanks for the explanation, that wasn't obvious to me at all that they behaved differently.

Much appreciated!

[eigenvivek]

No problem! I'll keep this open until a fix is merged.