RuntimeError while running VAIL

[Idate96]

The main runs on pytorch 0.4.1 but it fails for pytorch 1.6.0.

Running the main.py for VAIL implementation I get a runtime error about a replacement in place of a variable.

I put here the stack trace.

Error detected in AddmmBackward. Traceback of forward call that caused the error:
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 185, in
main()
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 162, in main
train_actor_critic(actor, critic, memory, actor_optim, critic_optim, args)
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/train_model.py", line 75, in train_actor_critic
values = critic(inputs)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/model.py", line 36, in forward
v = self.fc3(x)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 91, in forward
return F.linear(input, self.weight, self.bias)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/functional.py", line 1674, in linear
ret = torch.addmm(bias, input, weight.t())
(function print_stack)
Traceback (most recent call last):
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 185, in
main()
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 162, in main
train_actor_critic(actor, critic, memory, actor_optim, critic_optim, args)
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/train_model.py", line 101, in train_actor_critic
loss.backward()
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/tensor.py", line 185, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/autograd/init.py", line 127, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [100, 1]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

[AvinWangZH]

Has the same error. Is there any solve for this?

[SChrisLin]

You can change the function 'train_actor_critic', like the following:

def train_actor_critic():

    ...

    critic_optim.zero_grad()
    actor_optim.zero_grad()
    
    loss.backward(retain_graph=True) 
    loss.backward()
    
    critic_optim.step()
    actor_optim.step()