I want to compute the gradient of a function in several points. However, If I use tensors generated with torch.arange the gradient is not computed. Instead, using classical tensors it works. Why?
import torch
from torch import tensor
def l(w_1,w_2):
return w_1*w_2
w_1 = tensor(3., requires_grad=True)
w_2 = tensor(5., requires_grad=True)
l_v = l(w_1, w_2)
l_v.backward()
print(l_v.item(), w_1.grad, w_2.grad) # HERE WORKS OK
#############
for w_1_value in torch.arange(+2,+4,0.1, requires_grad=True):
for w_2_value in torch.arange(-2,+4,0.1, requires_grad=True):
print(w_1_value, w_2_value)
l_value = l(w_1_value, w_2_value)
l_value.backward()
print(l_value.item(), w_1_value.grad, w_2_value.grad) # HERE I GET NONE ON GRAD VALUES
Gradients are computed on leaf tensors, which in this case are the torch.arange tensors themselves. You need to access the grad attribute of those tensors.
When you iterate over the items in the torch.arange tensors, you are getting a view of the tensor, which doesn't have the grad.
This works for what you're trying to do:
w_1_values = torch.arange(+2,+4,0.1, requires_grad=True)
w_2_values = torch.arange(-2,+4,0.1, requires_grad=True)
for w_1_value in w_1_values:
for w_2_value in w_2_values:
l_value = l(w_1_value, w_2_value)
l_value.backward()
print(w_1_values.grad)
print(w_2_values.grad)