Re: [xsl] How do you prove that your XSLT programs are correct?

Subject: Re: [xsl] How do you prove that your XSLT programs are correct?
From: "Eliot Kimber eliot.kimber@xxxxxxxxxxxxxx" <xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx>
Date: Fri, 15 Apr 2022 14:38:55 -0000
One of the practical challenges in testing the output of XSLTs is the sheer
cost of maintaining the test cases themselves: if youre using some kind of
diff-based checker (this is something we used for the GAO HTML publishing
system using a home-grown XML diff mechanism) then you have to maintain the
exemplars youre testing against, which can be a significant time cost.

In general Ive found it more practical to depend on output inspection using
both artificial test case documents where the content says what the correct
expected result is as well as realistic content. This puts the time cost in
the inspection and depends on the knowledge of the inspectors but tends to be
a lower time cost overall. This is usually in the context of relatively
low-budget projects or projects where absolute correctness is not a top
concern (i.e., not safety critical or in a regulated industry).

Testing PDF output is even more of a challengeAntenna House sells a PDF
comparator tool that appears to do what you need (Ive never actually used the
product) but you face the same exemplar maintenance challenge as for XML or
HTML outputs.

Cheers,

E.
_____________________________________________
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com<https://www.servicenow.com>
LinkedIn<https://www.linkedin.com/company/servicenow> |
Twitter<https://twitter.com/servicenow> |
YouTube<https://www.youtube.com/user/servicenowinc> |
Facebook<https://www.facebook.com/servicenow>

From: Wendell Piez wapiez@xxxxxxxxxxxxxxx
<xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx>
Date: Friday, April 15, 2022 at 8:46 AM
To: xsl-list@xxxxxxxxxxxxxxxxxxxxxx <xsl-list@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [xsl] How do you prove that your XSLT programs are correct?
[External Email]

Hi,

What's more, Mike doesn't even touch on the question of defining what is
correct, or for that matter how one of XSLT's great strengths is in its
capability to handle things that are incorrect by one definition while correct
by another.

But yes, unit testing. It is not only the only way to know, it is the only way
to know you really know.

Cheers, Wendell

On Thu, Apr 14, 2022 at 12:52 PM Michael Kay
mike@xxxxxxxxxxxx<mailto:mike@xxxxxxxxxxxx>
<xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx<mailto:xsl-list-service@xxxxxxxxxxxx
rytech.com>> wrote:
You're not talking about proving correctness here, you're talking about
testing.

If you want to prove properties of the stylesheet statically, for example that
the output will conform to a particular schema, then consider writing
schema-aware XSLT code. (However, I'm very reluctant to talk about proviing
"correctness". You can prove a hypothesis about the behaviour, but you can't
prove that such behaviour will be considered "correct" in the real world.)

For testing, XSpec is a popular framework, which roughly does what you
describe in (2). This is also how the W3C XSLT test suite at
https://github.com/w3c/xslt30-test<https://urldefense.com/v3/__https:/github.
com/w3c/xslt30-test__;!!N4vogdjhuJM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0Dwh
WgF2vKxLOw6cg3DXHTq7eRdXAW-pkoiBlgSSc1ghRE7PKIXjqC6SSQwc1R1GX88uKI$> works.
(That test suite, of course, is geared more to coverage of constructs in the
XSLT language, so the details will be rather different from tests of a user
stylesheet, but the principle is much the same.)

The biggest weakness I see in the way most users test their stylesheets is
that they use far too few different source documents to achieve good coverage
of the logic, especially the error cases.

Michael Kay
Saxonica

> On 14 Apr 2022, at 17:08, Roger L Costello
costello@xxxxxxxxx<mailto:costello@xxxxxxxxx>
<xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx<mailto:xsl-list-service@xxxxxxxxxxxx
rytech.com>> wrote:
>
> Hi Folks,
>
> Scenario: You write an XSLT program that takes some input and outputs XML.
You want to be sure that your XSLT program is correct.
>
> As I see it, there are two ways to help ensure the correctness of an XSLT
program :
>
> 1. Pepper the XSLT program with assert statements, using xsl:assert.
>
> 2. Create a second XSLT program that queries the XML that was generated by
the first XSLT program. This second XSLT program contains a series of XPath
expressions to check various parts of the XML.
>
> Which of those do you use? Or do you use both? Or do you use something
else?
>
> How do you ensure the correctness of your XSLT programs?
>
> /Roger
>
>



--
...Wendell Piez... ...wendell -at- nist -dot- gov...
...wendellpiez.com... ...pellucidliterature.org... ...pausepress.org...
...github.com/wendellpiez.<https://urldefense.com/v3/__http:/github.com/wende
llpiez.__;!!N4vogdjhuJM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhWgF2vKxLOw6c
g3DXHTq7eRdXAW-pkoiBlgSSc1ghRE7PKIXjqC6SSQwc1R14Qb9XXQ$>..
...gitlab.coko.foundation/wendell...
XSL-List info and
archive<https://urldefense.com/v3/__http:/www.mulberrytech.com/xsl/xsl-list__
;!!N4vogdjhuJM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhWgF2vKxLOw6cg3DXHTq7e
RdXAW-pkoiBlgSSc1ghRE7PKIXjqC6SSQwc1R12QKqii8$>
EasyUnsubscribe<https://urldefense.com/v3/__http:/lists.mulberrytech.com/unsu
b/xsl-list/3453418__;!!N4vogdjhuJM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhW
gF2vKxLOw6cg3DXHTq7eRdXAW-pkoiBlgSSc1ghRE7PKIXjqC6SSQwc1R1RIqInF0$> (by
email<>)

Current Thread