ut-austin-rpl / viola Goto Github PK

How to visualize when running the program viola_bc/final_eval_script.py? Where does the offscreen_visualization function in img = offscreen_visualization(env, use_eye_in_hand=cfg.algo.use_eye_in_hand) come from? I didn't find this function

I try training the model with your generated datasets, and get this error:
Cannot import name 'CropRandomizer' from 'robomimic.models.base_nets'

With suspecting incorrect installation of robomimic, I go to https://github.com/ARISE-Initiative/robomimic/blob/master/robomimic/models/base_nets.py to check. However, I can't find CropRandomizer and Randomizer either.

Could you help me figure out why and how?

I have some questions:
For the question 1 and 2 (line 168 on here)

    transformer_out = transformer_out.reshape(original_shape)
    action_token_out = transformer_out[:, :, 0, :]
    if per_step:
        action_token_out = action_token_out[:, -1:, :]

1. Is there any reason you only use index 0 of transformer output?
2. During inference, you why do you take -1: index? Why do you set different setting for inference?
3. In your paper, you mentioned action token is used for the input, but I cannot find code where you used action token as input. Can you show where the corresponding code exists?
4. Can you explain why TensorUtils.time_distributed is used on this line?
5. During inference, is there a reason why do post-processing for gripper-history?

Thank you in advance!

where np.load(f"scenes/{domain_name}/normal/{eval_run_idx}_{i + rank * 50}.npz")

Do you have the scripts to collect the demos on the real robot?

action_token_out = transformer_out[:, :, 0, :].

Hello, i don't know why directly take the first dimension of the output as the action_token_out. After your grouping, the grouped input should follow this order: spatial_context_feature + region_feature + action_token + other obs feature. Would the dimension be changed when they pass through the transformer_decoder?

In addition, about the image augmentation (padding + random_crop), how many crops did you take? I saw around the code, only take the default value: num_crops=1. Doesn't the global feature really get lost if there is only one? Because i saw your code, the feature map is extracted from the cropped image.

Could you help me figure out why and how? Thanks a lot

what is initial_mjstate = env.sim.get_state().flatten()

Hello! Thanks for releasing this great works!
I am trying to reproduce this model in the real-world, so i use the viola dataset. I was wondering what is the scale of the dataset ? Such as, the action is absolute or delta? The translation is measaured as "m" or "cm" ? and the rotation is recorded as "rad" or "degree". I find the measures are confusing.
Thanks for answering my questions ~

Thanks for releasing your great works!!
Can you share Real-Robot Dataset's experiment config file(maybe xx_viola.yaml..) and checkpoint .pth file?