Subject: Re: [xsl] Comparing numbers with different precisions From: Michael Kay <mike@xxxxxxxxxxxx> Date: Thu, 09 Feb 2012 10:20:03 +0000 |
Hi!Presumably by "precision" you mean the number of decimal digits in the lexical form of the number. That means that trailing zero digits change the result, and since trailing zeroes are ignored for all the system-supplied numeric data types, it means you are outside the range of what the system data types can do.
I've a slightly awkward comparison, where I have two numbers of differing precisions, and I want to regard them as the same if the less preicse is the more precise rounded down to the same precision.
Examples are: 17.166666666666668 ~= 17.1666666666667 and 8.333333333333334 ~= 8.33333333333333
Unfortunately, the precision is not always the same for all the numbers I'm trying to compare.
I just wondered if anyone on this list had done anything similar, and had a good way of doing it, while I try and figure out a suitable way to do it.
Yours,
Clint Redwood.
Michael Kay Saxonica
Current Thread |
---|
|
<- Previous | Index | Next -> |
---|---|---|
[xsl] Comparing numbers with differ, Clint Redwood | Thread | [xsl] Comparing numbers with differ, Clint Redwood |
Re: [xsl] Rule of thumb: Are XPath , Andrew Welch | Date | Re: [xsl] Rule of thumb: Are XPath , TW |
Month |