Re: [xsl] XSL SUM BUG?

Subject: Re: [xsl] XSL SUM BUG?
From: "J.Pietschmann" <j3322ptm@xxxxxxxx>
Date: Thu, 19 Feb 2004 21:19:46 +0100
Andrew wrote:

I've following xml:
 <plan SUMM="700000.1"/>

execute sum(//plan/@SUMM) and you'll get 700000.09999...- what the ...?
but if you change 7 on 1 or 2 all works as expected- 1 preserved.
tested on .NET/xmlspy.

Computers usually don't represent floating point numbers the same way humas do. Internally, a binary representation is used, which can't excatly represent fractions which aren't a power of 1/2. Therefore if you convert 0.1 into the internal representation and back to decimal, you may experience roundoff errors. The common workaround is to use the format() function to tell the processor to round to a specific precision.


XSL-List info and archive:

Current Thread