pythonpytorchgradienttensorbackpropagation

How `backward()` works in PyTorch?


I recently study PyTorch and backward(). I understood how to use it, but when I try:

x = Variable(2*torch.ones(2, 2), requires_grad=True)
x.backward(x)
print(x.grad)

I expect:

tensor([[1., 1.],
        [1., 1.]])

because it is an identity function. However, it returns:

tensor([[2., 2.],
        [2., 2.]]).

Why this happens?


Solution

  • Actually, this is what you are looking for:

    Case 1: when z = 2*x**3 + x

    import torch
    from torch.autograd import Variable
    x = Variable(2*torch.ones(2, 2), requires_grad=True)
    z = x*x*x*2+x
    z.backward(torch.ones_like(z))
    print(x.grad)
    

    output:

    tensor([[25., 25.],
            [25., 25.]])
    

    Case 2: when z = x*x

    x = Variable(2*torch.ones(2, 2), requires_grad=True)
    z = x*x
    z.backward(torch.ones_like(z))
    print(x.grad)
    

    output:

    tensor([[4., 4.],
            [4., 4.]])
    

    Case 3: when z = x (your case)

    x = Variable(2*torch.ones(2, 2), requires_grad=True)
    z = x
    z.backward(torch.ones_like(z))
    print(x.grad)
    

    output:

    tensor([[1., 1.],
            [1., 1.]])
    

    To learn more how to calculate gradient in pytorch, check this.