Is this statement correct? :
Computations with large numbers is less accurate due to the logarithmic distribution of floating point numbers on the computer.
So that means computing with values around 1 is more accurate (because of rounding errors) then the same computations where each number has been scaled with 1e20 for example?
Yes the statement is correct, larger floating point numbers are less precise than smaller ones.
Floating point numbers have a fixed number of bits assigned to the mantissa. If the number that is being represented requires more bits than are in the mantissa then it will be rounded. So a smaller number can be represented more precisely.
To make this more concrete I wrote the following program that adds progressively smaller values to a large floating point number and a small one. Also to show the difference I included a double precision floating point that does not have rounding. But the double would experience the same problem if the mantissa were even larger.
#include <stdio.h>
int main() {
float large_float, small_float, epsilon;
double large_double, small_double;
large_float = 1 << 20;
small_float = 1;
epsilon = 0.1;
large_double = large_float;
small_double = small_float;
printf("large_float\t large_double\t small_float\t small_double\t epsilon\n");
for(int i = 0; i < 10; i++) {
printf("%f\t %f\t %f\t %f\t %f\n", large_float, large_double,small_float, small_double, epsilon);
large_float += epsilon;
large_double += epsilon;
small_float += epsilon;
small_double += epsilon;
epsilon /= 2;
}
return 0;
}
Running this program produces the following output:
large_float large_double small_float small_double epsilon
1048576.000000 1048576.000000 1.000000 1.000000 0.100000
1048576.125000 1048576.100000 1.100000 1.100000 0.050000
1048576.125000 1048576.150000 1.150000 1.150000 0.025000
1048576.125000 1048576.175000 1.175000 1.175000 0.012500
1048576.125000 1048576.187500 1.187500 1.187500 0.006250
1048576.125000 1048576.193750 1.193750 1.193750 0.003125
1048576.125000 1048576.196875 1.196875 1.196875 0.001563
1048576.125000 1048576.198438 1.198437 1.198438 0.000781
1048576.125000 1048576.199219 1.199219 1.199219 0.000391
1048576.125000 1048576.199609 1.199609 1.199609 0.000195
As you can see the large_float
value is less precise than the small_float
which can lead the final result to be less accurate as well.