RE: [xsl] Loosing Precision after Transformation

Subject: RE: [xsl] Loosing Precision after Transformation
From: "Michael Kay" <michael.h.kay@xxxxxxxxxxxx>
Date: Thu, 18 Apr 2002 09:49:20 +0100
It's the arithmetic that causes the loss of precision, so you should avoid
the multiplication. Perhaps you might get the required result with

<xsl:value-of select="translate(
                      format-number(number,'00000.00000'),
                      '.', '')"/>

Michael Kay
Software AG
home: Michael.H.Kay@xxxxxxxxxxxx
work: Michael.Kay@xxxxxxxxxxxxxx

> -----Original Message-----
> From: owner-xsl-list@xxxxxxxxxxxxxxxxxxxxxx
> [mailto:owner-xsl-list@xxxxxxxxxxxxxxxxxxxxxx]On Behalf Of
> Garvin Franco
> Sent: 18 April 2002 05:45
> To: XSL-List@xxxxxxxxxxxxxxxxxxxxxx
> Subject: [xsl] Loosing Precision after Transformation
>
>
> Has anyone had problems loosing precision after transforming
> a message using
> Xalan??
>
> Problem:
> I have content stored in an element as follows
> <aNumber>12345.12345</aNumber>
>
> I format the content in this element for output as follows

> note: length 10
> with 0 padding )
>
> <xsl:value-of select="format-number(number*100000,'0000000000')"/>
>
> On the ouput I get 1234512344, which is off by .00001. I need
> the exact
> value on the source document.  I tried a substring-before and
> substring-after the the decimal separator '.', then
> concatenated the total
> and fractional parts, then formatted as I have shown above,
> and I got the
> same thing. I know its loosing precision when the string is
> changed to a
> number, but the funny thing is, if a string less than the
> full length ( 10
> in this case ) of the field is passed, all is well.
> Example
> <number>1234.12345</number> ----> 01234.12345 ( which makes me happy )
>
> Does anyone know what approach I can use here to solve this problem?
>
> Thanking you in advance
> Garvin
>
> _________________________________________________________________
> Send and receive Hotmail on your mobile device: http://mobile.msn.com
>
>
>  XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list
>


 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list


Current Thread