Subject: Re: [xsl] XSL SUM BUG? From: "J.Pietschmann" <j3322ptm@xxxxxxxx> Date: Thu, 19 Feb 2004 21:19:46 +0100 |
I've following xml: <aa> <plan SUMM="700000.1"/> </aa>
execute sum(//plan/@SUMM) and you'll get 700000.09999...- what the ...? but if you change 7 on 1 or 2 all works as expected- 1 preserved. tested on .NET/xmlspy.
Computers usually don't represent floating point numbers the same way humas do. Internally, a binary representation is used, which can't excatly represent fractions which aren't a power of 1/2. Therefore if you convert 0.1 into the internal representation and back to decimal, you may experience roundoff errors. The common workaround is to use the format() function to tell the processor to round to a specific precision.
Current Thread |
---|
|
<- Previous | Index | Next -> |
---|---|---|
RE: [xsl] XSL SUM BUG?, Andreas L. Delmelle | Thread | Re: [xsl] XSL SUM BUG?, David Carlisle |
Re: [xsl] collect elements into va, G. Ken Holman | Date | Re: [xsl] XSL SUM BUG?, Andrew |
Month |