rubyfloating-point-precisionfixnum

I need help understanding ruby's floating point precision


I came across this the other day and sure its not causing me any trouble but i'm just curious as to why it happens?

1.9.2p320 :001 > 0.39-0.09
 => 0.30000000000000004

Solution

  • This is because Ruby by default uses Double-precision floating-point format. You can read about issues related to it here. However here's a short and crisp answer:

    Because internally, computers use a format (binary floating-point) that cannot accurately represent a number like 0.1, 0.2 or 0.3 at all.

    When the code is compiled or interpreted, your “0.1” is already rounded to the nearest number in that format, which results in a small rounding error even before the calculation happens.

    Source: http://floating-point-gui.de/