Subject: Re: [xsl] floor() and idiv From: Colin Paul Adams <colin@xxxxxxxxxxxxxxxxxx> Date: Thu, 06 Nov 2008 19:50:02 +0000 |
>>>>> "Andrew" == Andrew Welch <andrew.j.welch@xxxxxxxxx> writes: >> A bug in the spec or in Saxon ? >> Xmlizer> floor(1000000000000000000000000000000001 div Xmlizer> 100000000000000000000000000000001), Xmlizer> 1000000000000000000000000000000001 idiv Xmlizer> 100000000000000000000000000000001, Xmlizer> (1000000000000000000000000000000001 div Xmlizer> 100000000000000000000000000000001) cast as xs:integer >> Xmlizer> gives >> Xmlizer> 10, 9, 10 >> >> Saxon, I think. >> >> Gestalt gives 10, 10, 10. Andrew> So in Gestalt how do you determine when to use the Andrew> equivalent of BigDecimal in Java, or do you not need to Andrew> work about that sort of thing? Andrew> (In Java the result using standard division is Andrew> 9.999999999999998 which I guess just gets truncated to 9, Andrew> but using BigDecimal.divide() you get the correct 10) I always use the equivalent of BigDecimal if I cannot guarantee to avoid errors with INTEGER_64. I'm not sure how you can get a rounding error with integer or decimal arithmetic - those constants are integer constants, not doubles, in XPath 2.0. I believe that double arithmetic is non-compliant. Anyway, I am far from au fait with Java myself, so i am prepared to be surprised. -- Colin Adams Preston Lancashire
Current Thread |
---|
|
<- Previous | Index | Next -> |
---|---|---|
Re: [xsl] floor() and idiv, Andrew Welch | Thread | Re: [xsl] floor() and idiv, Andrew Welch |
Re: [xsl] floor() and idiv, Andrew Welch | Date | Re: [xsl] Cannot have an attribute , Andrew Welch |
Month |