Page 1 of 1

Close but not zero

PostPosted: Thu Apr 04, 2013 4:06 am
by BWL
In using Oovium to do some accounting reconciliation, I encountered an unexpected round off error with the sum of three terms (a, b, c) with one term negative (c). The order used in summing the terms makes a difference (i.e. a+b+c versus c+b+a or a+c+b).

The particular case is below:

a: 4,374,600.7
b: 437,223.5
c: -0.02
y: a+b+c [yielding 4,811,824.18]
d: 4,811,824.18
y-d: 9.3132257e-10 [expecting 0]

If y is reordered to c+a+b, y-d=0 without rounding error.

=============
OOvium 1.1.3.4
iOS 6.1.3

Re: Close but not zero

PostPosted: Thu Apr 04, 2013 5:40 am
by joe
Yeah, anytime one is using floating point to represent numbers, subtractions resulting in zero have a possibility of going awry. I actually vaguely recall a bug in the old Microsoft Calculator related to the exact value .02.

In order to fix these types of errors I should convert Oovium from using the floating points to using some sort of BigDecimal data type. I'm not sure what the performance ramifications of that would be, but it's definitely something I need to look into.

As Oovium stands right now, I suspect for most people performance isn't an issue. And any performance issue that a person is experiencing is probably not due to calculation times. However, with certain future features of Oovium calculation speed will become more of an issue, so I'm not necessarily 100% inclined to switch over to a BigDecimal data type if there is a performance hit.

I'll do some research on the issue and see where that takes me.

Again, thanks a ton for reporting this observation. I hadn't noticed any such bugs, so the report is highly useful to me.

Re: Close but not zero

PostPosted: Fri Apr 05, 2013 5:19 am
by BWL
Since rounding is unavoidable, a solution for me would be to support selective formating where floating point numbers can be rounded to the user's desired decimal places; thereby permitting user to control that which will go "awry."

Re: Close but not zero

PostPosted: Fri Apr 05, 2013 10:13 am
by joe
Yes, that's on my to do list.

In the interim you could create a rounding Mech to use in your reconciliation column.

round(x*100)/100

Re: Close but not zero

PostPosted: Sun Apr 07, 2013 7:29 am
by joe
By the way, thanks for the kind review.