Can anyone explain me why the result of the below does not equal zero?
? CSng("0.199881939681229")
? CSng(0.1998819)
? CSng(CSng(0.199881939681229) - CSng(0.1998819))
1st line returns 0.1998819 2nd line returns 0.1998819 too but the 3rd returns 4.470348E-08
Where is this figure coming from if both figures are represented exactly by the same value in a the single-precision data type.
seems that the other answer for this was deleted so i will respond myself.
VBA is returning 0.1998819 but it is not actually storing both numbers as the same binary.
see http://www.binaryconvert.com/result_float.html?decimal=048046049057057056056049057051057054056049050050057 and http://www.binaryconvert.com/result_float.html?decimal=048046049057057056056049057 .
Nearly identical but not.
mystery solved, VBA representation of floats in the immediate window was the issue.