Input:
x = torch.tensor([3.5])
print(x)
print(x.item())
print(float(x))
Output:
(tensor([3.5000]), 3.5, 3.5)
But if I change torch.tensor parameter to any other value rather than some_value.5, it gives a unprecise output:
Input:
x = torch.tensor([3.1])
print(x)
print(x.item())
print(float(x))
Output:
(tensor([3.1000]), 3.0999999046325684, 3.0999999046325684)
What you experience has not much to do with PyTorch itself, but rather with the representation of real numbers as floating point values on a modern computer. The underlying situation here is the following:
It is true that this appears to vanish once you switch from 32-bit floating point values to 64-bit floating point values, as suggested in Karl's answer, as a 64-bit floating point value has twice as many bits for representing each value. However, this is not because the underlying situation has been resolved, but rather because your Python interpreter now knows, in a way, that it has to hide the situation from you.
Try, for example, printing f"{torch.tensor([3.1], dtype=torch.float64).item():.30f}
or, in fact, f"{3.1:.30f}"
, thereby enforcing to see more digits than Python, by default, wants to show you:
import torch
print(f"{torch.tensor([3.1], dtype=torch.float64).item():.30f}")
# >>> 3.100000000000000088817841970013
print(f"{3.1:.30f}")
# >>> 3.100000000000000088817841970013
You will see that also for the 64-bit floating point case, the floating point representation of the number 3.1 is only an approximation of the true value.
Likewise, you can try the same with 3.5 and you will find that the value is stored exactly:
import torch
print(f"{torch.tensor([3.5], dtype=torch.float64).item():.30f}")
# >>> 3.500000000000000000000000000000
print(f"{3.5:.30f}")
# >>> 3.500000000000000000000000000000
The next is probably a bit of a bold claim and, depending on what data you work with, a huge oversimplification or even plain wrong, but still: In your daily life as a programmer, I would say, you rarely have to deal with this knowledge but simply ignore the fact that floating point values are only approximations in most cases. There are also situations where precise floating point arithmetic matters, however, and solutions such as the decimal
module and its Decimal
class have been designed for that. In some other situations, such as the comparison of floating point numbers, it may be appropriate to allow for some tolerance, e.g. by using PyTorch's allclose()
function instead of an equality comparison (torch.allclose(t1, t2)
instead of (t1 == t2).all()
).
For further general information on the properties of floating-point arithmetic, you might want to have a look at Floating-Point Arithmetic: Issues and Limitations from the Python documentation, or at the somewhat standard article on the subject, What Every Computer Scientist Should Know About Floating-Point Arithmetic.