Subject: RE: [xsl] Re: XSLT 2.0 Decimal number silliness From: Rowland Shaw <Rowland.Shaw@xxxxxxxxxxxxxxxxxxx> Date: Mon, 29 Mar 2004 06:15:38 -0800 |
Maybe I'm being really dumb here, but what is the difference between a decimal number and a floating point number? -----Original Message----- From: Jeff Kenton [mailto:jkenton@xxxxxxxxxxxxx] Sent: 29 March 2004 14:53 To: xsl-list@xxxxxxxxxxxxxxxxxxxxxx Subject: Re: [xsl] Re: XSLT 2.0 Decimal number silliness Pete, You missed the point. We're talking about decimal numbers here, not floats. --jeff Pete Forman wrote: > Jeff Kenton <jkenton@xxxxxxxxxxxxx> writes: > > > I've been looking at XSLT 2.0 decimal numbers (again), and I still > > think the WC needs to do better. > > [...] > > <xsl:value-of select="2.0 div 3.0" />~ > > <xsl:value-of select="0.444 div 0.666" />~ > > [...] > > Both lines of output should be equal. > > Why? The majority of implementations will use a binary format such as > IEC 60559 to hold floating point numbers. 2.0 and 3.0 can be > converted exactly but the representations of 0.444 and 0.666 will be > inexact (0.44400000000000001 and 0.66600000000000004 in my test). You > might get a better match with 0.5 div 0.75. If you want fixed point > number semantics try moving the decimal points to the right before > dividing. > > Floating point arithmetic is inexact by definition. Your results > _are_ the same to within the precision specified by the language > and/or implementation.
Current Thread |
---|
|
<- Previous | Index | Next -> |
---|---|---|
RE: [xsl] It's possible to give ext, Roberta Granata | Thread | Re: [xsl] Re: XSLT 2.0 Decimal numb, Andrew Curry |
Re: [xsl] Re: XSLT 2.0 Decimal numb, Jeff Kenton | Date | Re: [xsl] Re: XSLT 2.0 Decimal numb, Andrew Curry |
Month |