RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

[sunkymepro]

Dear author， thanks for your code. But I have some trouble in the procedure of backward.

The error message is:

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [512, 1, 3, 3]] is at version 4; expected version 3 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

The program reported this error on this line:

kernel, kernel_flip = ctx.saved_tensors

this line of code is inside the backward function in class BlurFunction.

I don't know why this error occurs. It seems that the value these two variables has changed and these variables are needed for gradient computation.

[sunkymepro]

I add the code 'kernel, kernel_flip = ctx.saved_tensors' before the code 'ctx.save_for_backward(kernel, kernel_flip)' in forward function of class BlurFunction.
And there occurs a new error:

ValueError: not enough values to unpack (expected 2, got 0)

It seems that the variable kernel and variable kernel_flip aren't saved successfully. I don't know why this problem occurs, is there anyone who can help me?

[rosinality]

Hmm I can't reproduce the error. And I don't have clue as inplace operations is not happened over these two tensors.

[sunkymepro]

Hmm I can't reproduce the error. And I don't have clue as inplace operations is not happened over these two tensors.

It may be because the version of cuda and cudnn are mismatch? I do experiment in a 3090 NVIDIA GPU and use torch 1.8.1 and cuda 11.2.

[rosinality]

Hmm, I use torch 1.8.1 and cuda 10.2. But I don't know cuda version could affect these kind of problems.

[superAa]

I have solved ，You can see that [(https://zhuanlan.zhihu.com/p/409117481)]