javaexcelprecisionjxlmarkov-chains

Emulate excel calculus behavior in Java for testing purposes


I'm implementing in Java an algorithm written with Excel formulas, based on Markov's chains, and comparing the results with Jxl library. The problem is, as I have read, that the two languages haven't the same precision. In fact, they return equal results only to the second decimal digit. Is there a method to "emulate" Excel precision in Java, to test if the algorithms behave exactly the same way?


Solution

  • You can find an analysis how Excel precision works here:

    https://community.oracle.com/thread/2053255?start=15&tstart=0

    It appears that the 16th decimal digit is rounded down when processing.

    See this question for a method to emulate the floating point precision:

    Matching Excel's floating point in Java

    I suppose that this all taken together means that internally, meaning inside the formula of a single cell, Excel is using normal doubles. But when setting the value of the cell, the 16 decimal place is rounded down. Which means for your algorithm that whenever there used to be an intermediate result written to a cell, you need to use the rounding method from the linked answer. Test this, for me it is not clear whether these intermediate values need to be rounded or not.