>>>>> "Andrew" == Andrew Welch <andrew.j.welch@xxxxxxxxx> writes:
>> A bug in the spec or in Saxon ?
>>
Xmlizer> floor(1000000000000000000000000000000001 div
Xmlizer> 100000000000000000000000000000001),
Xmlizer> 1000000000000000000000000000000001 idiv
Xmlizer> 100000000000000000000000000000001,
Xmlizer> (1000000000000000000000000000000001 div
Xmlizer> 100000000000000000000000000000001) cast as xs:integer
>>
Xmlizer> gives
>>
Xmlizer> 10, 9, 10
>>
>> Saxon, I think.
>>
>> Gestalt gives 10, 10, 10.
Andrew> So in Gestalt how do you determine when to use the
Andrew> equivalent of BigDecimal in Java, or do you not need to
Andrew> work about that sort of thing?
Andrew> (In Java the result using standard division is
Andrew> 9.999999999999998 which I guess just gets truncated to 9,
Andrew> but using BigDecimal.divide() you get the correct 10)
I always use the equivalent of BigDecimal if I cannot guarantee to
avoid errors with INTEGER_64. I'm not sure how you can get a rounding
error with integer or decimal arithmetic - those constants are integer constants,
not doubles, in XPath 2.0. I believe that double arithmetic is
non-compliant. Anyway, I am far from au fait with Java myself, so i am
prepared to be surprised.
--
Colin Adams
Preston Lancashire