I recently study PyTorch and backward()
.
I understood how to use it, but when I try:
x = Variable(2*torch.ones(2, 2), requires_grad=True)
x.backward(x)
print(x.grad)
I expect:
tensor([[1., 1.],
[1., 1.]])
because it is an identity function. However, it returns:
tensor([[2., 2.],
[2., 2.]]).
Why this happens?
Actually, this is what you are looking for:
Case 1: when z = 2*x**3 + x
import torch
from torch.autograd import Variable
x = Variable(2*torch.ones(2, 2), requires_grad=True)
z = x*x*x*2+x
z.backward(torch.ones_like(z))
print(x.grad)
output:
tensor([[25., 25.],
[25., 25.]])
Case 2: when z = x*x
x = Variable(2*torch.ones(2, 2), requires_grad=True)
z = x*x
z.backward(torch.ones_like(z))
print(x.grad)
output:
tensor([[4., 4.],
[4., 4.]])
Case 3: when z = x (your case)
x = Variable(2*torch.ones(2, 2), requires_grad=True)
z = x
z.backward(torch.ones_like(z))
print(x.grad)
output:
tensor([[1., 1.],
[1., 1.]])
To learn more how to calculate gradient in pytorch, check this.