reinforcement-learning-kr / lets-do-irl Goto Github PK

@dongminlee94 , Great work on creating a simple IRL solution - Is your work licensed under the MIT license?

Hi,
Thanks for you providing so nice code.
But when I try to play with "lets-do-irl/mujoco/vail", It make me very confused that the code in train_model.py:
vdb_loss = criterion(learner, torch.ones((states.shape[0], 1))) + \ criterion(expert, torch.zeros((demonstrations.shape[0], 1))) + \ beta * bottleneck_loss
I think this should be:
vdb_loss = criterion(learner, torch.zeros((states.shape[0], 1))) + \ criterion(expert, torch.ones((demonstrations.shape[0], 1))) + \ beta * bottleneck_loss
. In other words, the learner should be pushed to zeros, and expert should be pushed to ones, isn't it? Or both is fine?

By the way, the code in the same file:
beta = max(0, beta + args.alpha_beta * bottleneck_loss)
If beta equals beta + args.alpha_beta * bottleneck_loss, at the next time backward, there will report a bug about beta, which is modified by a inplace operation.

hi, how am i supposed to save expert demo in ppo main?

Hi sir
I finished training the robot mobile model, need help to understand the testing, i used SAC algorithm
Could we make a deal about that please

state_idx = idx_to_state(env, state)

how can I fix this?

idx_to_state
position_idx = int((state[0] - env_low[0]) / env_distance[0])

TypeError: only size-1 arrays can be converted to Python scalars

Hello,

I am running the VAIL example as explained in the README.

main.py works as expected.

When running test (e.g., as written: python test.py --load_model ckpt_4000_vail.pth.tar), there is an import error.

Specifically:

there is no file running_state. In the main.py the running state was defined from Zfilter.

when i run main.py ,i met some problem:
Traceback (most recent call last):
File "main.py", line 186, in
main()
File "main.py", line 159, in main
expert_acc, learner_acc = train_vdb(vdb, memory, vdb_optim, demonstrations, 0, args)
File "/home/parallels/workspace/lets-do-irl-master/mujoco/vail/train_model.py", line 37, in train_vdb
vdb_loss.backward(retain_graph=True)
File "/home/parallels/anaconda3/envs/mujoco/lib/python3.8/site-packages/torch/_tensor.py", line 487, in backward
torch.autograd.backward(
File "/home/parallels/anaconda3/envs/mujoco/lib/python3.8/site-packages/torch/autograd/init.py", line 197, in backward
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [100, 4]], which is output 0 of AsStridedBackward0, is at version 35; expected version 34 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

The main runs on pytorch 0.4.1 but it fails for pytorch 1.6.0.

Running the main.py for VAIL implementation I get a runtime error about a replacement in place of a variable.

I put here the stack trace.

Error detected in AddmmBackward. Traceback of forward call that caused the error:
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 185, in
main()
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 162, in main
train_actor_critic(actor, critic, memory, actor_optim, critic_optim, args)
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/train_model.py", line 75, in train_actor_critic
values = critic(inputs)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/model.py", line 36, in forward
v = self.fc3(x)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 91, in forward
return F.linear(input, self.weight, self.bias)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/functional.py", line 1674, in linear
ret = torch.addmm(bias, input, weight.t())
(function print_stack)
Traceback (most recent call last):
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 185, in
main()
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 162, in main
train_actor_critic(actor, critic, memory, actor_optim, critic_optim, args)
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/train_model.py", line 101, in train_actor_critic
loss.backward()
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/tensor.py", line 185, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/autograd/init.py", line 127, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [100, 1]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!