AssertionError: {'action': array([nan, nan, nan, nan, nan, nan], dtype=float32), 'reset': array(False)} about recall2imagine

[Lr-2002]

hi guys , I'm also reproducing the project, Let's make a telegram channel for immediate communication?

[Lr-2002]

https://t.me/+kFtu63HMopQ3Njg0

[Liuxueyi]

It related with the gpus used with policy_devices and train_devices.

[artemZholus]

Hi , @Liuxueyi

did you resolve the issue? How? We observed this type of error in our experiments but it was extremely rare.

I am hiding the link for the telegram chat as the best way to communicate with us is through isssues here.

[Liuxueyi]

From my point of view, the error occurs when the policy_devices is set to be [0] while the train_devices is set to be [0, 1]. When I make policy_devices and train_devices the same, the error is resolved. This issue may be related to jax.

[Liuxueyi]

By the way, could you provide me with the time consumption on each task? And what gpu devices do you use?

[artemZholus]

@Liuxueyi I will answer the questions above later. Just wanted to mention that you might benefit from the argument --unlocked_sampling True in Memory Maze. However, we DO NOT guarantee exactly the same sample efficiency as reported in our paper when using that argument. It lets the model to sample experinces as fast as possible and train as fast as possible. With that enabled, you should tune your number of workers to reach the training_ratio of 50 (shows up in wandb) which we had when doing unlocked sampling.

[artemZholus]

With this enabled you'll have the highest GPU utilization. For us 400M samples with this argument took 14 days on two A100 gpus. See the last page of the paper for more details.