Re: [xsl] XSL SUM BUG?

<-prev [Thread] next->	<-prev [Date] next->
Month Index \| List Home

Re: [xsl] XSL SUM BUG?

Subject: Re: [xsl] XSL SUM BUG?
From: David Carlisle <davidc@xxxxxxxxx>
Date: Thu, 19 Feb 2004 20:52:18 GMT


Its not a bug, it's a feature of binary computing and numerical
calculations.
You can not store 0.1 exactly as a base 2 floating point number, so you
can not get any exact answers from any calculation involving such a
number.

Use format-number() to round to a suitable number of decimal places and
the error terms will disapear.

David

-- 
http://www.dcarlisle.demon.co.uk/matthew

 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list

Current Thread
Re: [xsl] XSL SUM BUG?, (continued) G. Ken Holman - Thu, 19 Feb 2004 15:05:28 -0500 Andrew - Thu, 19 Feb 2004 23:21:11 +0300 Andreas L. Delmelle - Thu, 19 Feb 2004 23:48:51 +0100 J.Pietschmann - Thu, 19 Feb 2004 21:19:46 +0100 David Carlisle - Thu, 19 Feb 2004 20:52:18 GMT <= Dimitre Novatchev - Thu, 19 Feb 2004 22:19:23 +0100 Dimitre Novatchev - Thu, 19 Feb 2004 22:42:40 +0100 Saverio Perugini - Thu, 19 Feb 2004 18:00:53 -0500 (EST) Dimitre Novatchev - Fri, 20 Feb 2004 06:49:14 +0100

Current Thread

Re: [xsl] XSL SUM BUG?, (continued)

<- Previous	Index	Next ->
Re: [xsl] XSL SUM BUG?, J.Pietschmann	Thread	[xsl] The Solution (Was: Re: extrac, Dimitre Novatchev
Re: [xsl] FW: Troubleshooting a sor, David Carlisle	Date	Re: [xsl] Re: Counting nodes effici, Wendell Piez
	Month