Subject: Re: [xsl] floor() and idiv From: "Andrew Welch" <andrew.j.welch@xxxxxxxxx> Date: Thu, 6 Nov 2008 20:12:33 +0000 |
> >> Gestalt gives 10, 10, 10. > > Andrew> So in Gestalt how do you determine when to use the > Andrew> equivalent of BigDecimal in Java, or do you not need to > Andrew> work about that sort of thing? > > Andrew> (In Java the result using standard division is > Andrew> 9.999999999999998 which I guess just gets truncated to 9, > Andrew> but using BigDecimal.divide() you get the correct 10) > > I always use the equivalent of BigDecimal if I cannot guarantee to > avoid errors with INTEGER_64. I'm not sure how you can get a rounding > error with integer or decimal arithmetic - those constants are integer constants, > not doubles, in XPath 2.0. I believe that double arithmetic is > non-compliant. Anyway, I am far from au fait with Java myself, so i am > prepared to be surprised. As far as I understand, which is not very far, it's a fundamental problem with binary arithmetic. The standard division technique is fast but incurs this sort of problem, while the BigDecimal approach is exact but much slower. The standard approach is fine for the vast majority of uses, but for times where you need to be exact (like financial calculations) you have to use BigDecimal. I think anyway... David C might enlighten us :) -- Andrew Welch http://andrewjwelch.com Kernow: http://kernowforsaxon.sf.net/
Current Thread |
---|
|
<- Previous | Index | Next -> |
---|---|---|
Re: [xsl] floor() and idiv, Colin Paul Adams | Thread | Re: [xsl] floor() and idiv, Colin Paul Adams |
Re: [xsl] Cannot have an attribute , Andrew Welch | Date | Re: [xsl] floor() and idiv, Colin Paul Adams |
Month |