I have a simple logit and target tensors and computing the cross entropy using Pytorch's from torch.nn import functional as F
module.
>>> F.cross_entropy(torch.tensor([[0.25, 0.5, 0.25]]), torch.tensor([0]))
tensor(1.1891)
I'm getting 1.1891 as the answer. But according to the equation of cross_entropy loss, shouldn't the answer be
-(1*ln(0.25) + 0*ln(0.5) + 0*ln(0.25) = -ln(0.25) = 1.386
?
What is going on?
Because you are missing how it is calculated. It is mentioned in documentation (related docs)
-torch.log(torch.softmax(a,dim=1))[0][0]
It calculates softmax of the entries first and then calculates cross-entropy.
In short
F.nll_loss(F.log_softmax(a), torch.tensor([0]))
Here is the linked documentation to which this refers to