About the tp1 in replay about rvt

[imankgoyal]

Hi @TeleeMa,

Thanks for your kind words.

I am not sure if I undertand your question correctly.

tp1 (defined here: https://github.com/NVlabs/RVT/blob/master/rvt/utils/dataset.py#L242) stores the gripper location in the next key frame. It is used to extact the action for supervision (https://github.com/NVlabs/RVT/blob/master/rvt/utils/dataset.py#L169-L184).

Hope it helps.

Best,
Ankit

[TeleeMa]

Hi @imankgoyal,

Thank you for your reply. I mean in the process of training, each batch from dataloader contains the keys of "tp1" as the image shows.

And the tp1 keys and values are added into the batch when sampling here https://github.com/NVlabs/YARR/blob/72c37e6d31ff1b1d7ce131be5c8f80f66ee271d8/yarr/replay_buffer/uniform_replay_buffer.py#L821-L832.
But they are not used in the model training. So what these for?

[imankgoyal]

You can safely igore them. These are legacy attribute coming from PerAct -- Arm as our dataloader is exactly the same.