Re: [xsl] decimal vs double value spaces

Subject: Re: [xsl] decimal vs double value spaces
From: Syd Bauman <Syd_Bauman@xxxxxxxxx>
Date: Thu, 5 Nov 2009 17:36:17 -0500
> Some numbers are in the decimal space but not in the double space.
> For example, the decimal value space includes the value 0.1 (one
> tenth), because that number can be represented using a finite
> number of decimal digits. The double value space does not include
> that number, because in binary notation the representation of that
> number is a repeating fraction.

But there's a bit more nuance to it than that. 

First, xsd:double (aka an IEEE 754 "double" or "binary64") has
distinct limits, with an approximate range of +/- 1.8 E 308 (base
10). So if you want to express a number larger than that positive or
smaller than that negative, you're out of luck. (Note that you
probably don't: 1.8 E 308 exceeds the estimate of the number of sub-
atomic particles in the universe by perhaps two hundred orders of
magnitude.)

Also a binary64 has the property of the gaps between representable
numbers being unequal -- they get larger as the numbers get further
from zero. There is actually an interesting formula for calculating
the space between representable numbers, but I don't remember it.

See
http://en.wikipedia.org/wiki/Double_precision_floating-point_format.

The point of the post is, in part, to say that if you want precision,
use xsd:decimal. If you need a (*very*) wide range of values and can
live without some precision, use xsd:double.[1]

Any more about this should probably be taken off-list, as it isn't
really that XSL-specific. (But it is interesting. :-)

Note
----
[1] This is precisely why the TEI data.decimal macro was defined as
    "xsd:decimal | xsd:double".

Current Thread