Subject: RE: [xsl] Rounding errors in financial app transforms From: "Michael Kay" <mike@xxxxxxxxxxxx> Date: Thu, 26 Jul 2007 14:51:53 +0100 |
> MulDiv, as, for example the Windows API function, > > "The MulDiv function multiplies two 32-bit values and then > divides the 64-bit result by a third 32-bit value. > The return value is rounded up or down to the nearest integer." > The specs do say that implementations must support 18 digits of precision for xs:integer, so it's very likely they will use 64 bits or more. There's no absolute rule that intermediate results of xs:integer calculations must retain at least 64 bits, but it's a reasonable expectation. xs:decimal leaves more scope for implementations to vary, especially with division and in their overflow behaviour, but I think you'd be unlikely to find an implementation that doesn't allow any two 32-bit values to be multiplied without loss. Michael Kay http://www.saxonica.com/
Current Thread |
---|
|
<- Previous | Index | Next -> |
---|---|---|
Re: [xsl] Rounding errors in financ, David Carlisle | Thread | Re: [xsl] Rounding errors in financ, Justin Johansson |
Re: [xsl] Rounding errors in financ, Dimitre Novatchev | Date | RE: [xsl] Rounding errors in financ, Michael Kay |
Month |