Subject: Re: [xsl] Magic numbers From: David Carlisle <davidc@xxxxxxxxx> Date: Tue, 18 Nov 2003 13:17:24 GMT |
> which works fine and shows the expected result... now...if you change > .05 for .07 you get something totally unexpected (at least is unexpected > to me). It is the result for 0.05 that is more surprising than the result for 0.07. You wrote your numbers using decimal ".05" or ".07" but most computers these days are binary machines. You can not store either of those numbers exactly in base two floating point, just as you can't store 1/3 in base 10: its 0.333333... as accurate as you want to be but you can't store it exactly. So whenever you do floating point arithemetic on a machine you should _expect_ to see roundoff errors. Sometimes you get lucky and the roundoff error is smaller than the number of digits output so you get what looks like an exact answer, but in general this can't happen. David ________________________________________________________________________ This e-mail has been scanned for all viruses by Star Internet. The service is powered by MessageLabs. For more information on a proactive anti-virus service working around the clock, around the globe, visit: http://www.star.net.uk ________________________________________________________________________ XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
Current Thread |
---|
|
<- Previous | Index | Next -> |
---|---|---|
[xsl] Magic numbers, Yago Alvarado | Thread | Re: [xsl] Magic numbers, Joe Fawcett |
RE: [xsl] SAXON: How to run?, SHEIKH Sajjad | Date | [xsl] keep-together and FOP, Tomas Salkauskas |
Month |