Subject: Re: [xsl] floor() and idiv From: "Andrew Welch" <andrew.j.welch@xxxxxxxxx> Date: Thu, 6 Nov 2008 19:43:40 +0000 |
> A bug in the spec or in Saxon ? > > Xmlizer> floor(1000000000000000000000000000000001 div > Xmlizer> 100000000000000000000000000000001), > Xmlizer> 1000000000000000000000000000000001 idiv > Xmlizer> 100000000000000000000000000000001, > Xmlizer> (1000000000000000000000000000000001 div > Xmlizer> 100000000000000000000000000000001) cast as xs:integer > > Xmlizer> gives > > Xmlizer> 10, 9, 10 > > Saxon, I think. > > Gestalt gives 10, 10, 10. So in Gestalt how do you determine when to use the equivalent of BigDecimal in Java, or do you not need to work about that sort of thing? (In Java the result using standard division is 9.999999999999998 which I guess just gets truncated to 9, but using BigDecimal.divide() you get the correct 10) My particular issue is concerning pixel widths of mobile phones, so it pretty unlikely to encounter these numbers :) -- Andrew Welch http://andrewjwelch.com Kernow: http://kernowforsaxon.sf.net/
Current Thread |
---|
|
<- Previous | Index | Next -> |
---|---|---|
Re: [xsl] floor() and idiv, Colin Paul Adams | Thread | Re: [xsl] floor() and idiv, Colin Paul Adams |
Re: [xsl] floor() and idiv, Colin Paul Adams | Date | Re: [xsl] floor() and idiv, Colin Paul Adams |
Month |