I've been trying to round a float value to 4 precision without success.
float fconv = 1.0f;
float fdata = 39.934543423412f;
float fres = RoundTo(fdata*fconv, -4);
if(fres <= 39.9345f){do something;} //<-- unwanted behavior
Wanted result is 39.934500000000
Actual result is 39.934543423412
I've tried many methods including Round a float to a given precision without success.
I'm working on an AMD FX83xx 64bit. Program is built in 32bit Debug using XE7
Thanks
Your desired precision of 6 decimal digits is very near the precision limits for a float data type. The epsilon, or delta between consecutive representable floating point values, for a number around 40f is about 7.63E-6, so there's only a couple of bits different between the 'best' value and what you're getting. This is possibly due to rounding that close to the limit, but I'm not sure.