Re: [xsl] How do you prove that your XSLT programs are correct?

["G. Ken Holman g.ken.holman@xxxxxxxxx" <xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx>]

At 2022-04-15 14:39 +0000, Eliot Kimber eliot.kimber@xxxxxxxxxxxxxx wrote:
> One of the practical challenges in testing the
> output of XSLTs is the sheer cost of maintaining
> the test cases themselves: if youre using some
> kind of diff-based checker (this is something we
> used for the GAO HTML publishing system using a
> home-grown XML diff mechanism) then you have to
> maintain the exemplars youre testing against,
> which can be a significant time cost.
>
> In general Ive found it more practical to
> depend on output inspection using both
> artificial test case documents where the content
> says what the correct expected result is as well
> as realistic content. This puts the time cost in
> the inspection and depends on the knowledge of
> the inspectors but tends to be a lower time cost
> overall. This is usually in the context of
> relatively low-budget projects or projects where
> absolute correctness is not a top concern (i.e.,
> not safety critical or in a regulated industry).
>
> Testing PDF output is even more of a
> challengeAntenna House sells a PDF comparator
> tool that appears to do what you need (Ive
> never actually used the product) but you face
> the same exemplar maintenance challenge as for XML or HTML outputs.

I can attest to how valuable the Antenna House
PDF comparator tool AHRTS has been for the
https://RealtaOnline.com project that I'm on. I
could not live without it. Having it catch
something as subtle as adding an inherited
attribute and not realizing the impact that it
has by moving content over by just one pixel when
I didn't want it to. Never would I have noticed by eye.

And in our situation I have addressed many
customers by writing onion-skin shells around a
core set of stylesheets ... being able to test a
change in the core against all of the customers'
differing uses of the core means that AHRTS will
catch me when a change I make in the core for one
customer impacts the expected results of another customer.

I would not have the confidence in my changes
without this tool watching my back!

. . . . . . Ken



> Cheers,
>
>
>
> E.
>
> _____________________________________________
>
> Eliot Kimber
>
> Sr Staff Content Engineer
>
> O: 512 554 9368
>
> M: 512 554 9368
>
> <https://www.servicenow.com>servicenow.com
>
> <https://www.linkedin.com/company/servicenow>LinkedIn
> | <https://twitter.com/servicenow>Twitter |
> <https://www.youtube.com/user/servicenowinc>YouTube
> | <https://www.facebook.com/servicenow>Facebook
>
>
>
> From: Wendell Piez wapiez@xxxxxxxxxxxxxxx
> <xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx>
> Date: Friday, April 15, 2022 at 8:46 AM
> To: xsl-list@xxxxxxxxxxxxxxxxxxxxxx <xsl-list@xxxxxxxxxxxxxxxxxxxxxx>
> Subject: Re: [xsl] How do you prove that your XSLT programs are correct?
>
> [External Email]
>
>
>
> Hi,
>
>
>
> What's more, Mike doesn't even touch on the
> question of defining what is correct, or for
> that matter how one of XSLT's great strengths is
> in its capability to handle things that are
> incorrect by one definition while correct by another.
>
>
>
> But yes, unit testing. It is not only the only
> way to know, it is the only way to know you really know.
>
>
>
> Cheers, Wendell
>
>
>
> On Thu, Apr 14, 2022 at 12:52 PM Michael Kay
> <mailto:mike@xxxxxxxxxxxx>mike@xxxxxxxxxxxx
> <<mailto:xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx>xsl-list-service@xxxxxxxxxx
errytech.com>
> wrote:
>
> You're not talking about proving correctness
> here, you're talking about testing.
>
> If you want to prove properties of the
> stylesheet statically, for example that the
> output will conform to a particular schema, then
> consider writing schema-aware XSLT code.
> (However, I'm very reluctant to talk about
> proviing "correctness". You can prove a
> hypothesis about the behaviour, but you can't
> prove that such behaviour will be considered "correct" in the real world.)
>
> For testing, XSpec is a popular framework, which
> roughly does what you describe in (2). This is
> also how the W3C XSLT test suite at
> <https://urldefense.com/v3/__https:/github.com/w3c/xslt30-test__;!!N4vogdjhu
JM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhWgF2vKxLOw6cg3DXHTq7eRdXAW-pkoiBl
gSSc1ghRE7PKIXjqC6SSQwc1R1GX88uKI$>https://github.com/w3c/xslt30-test
> works. (That test suite, of course, is geared
> more to coverage of constructs in the XSLT
> language, so the details will be rather
> different from tests of a user stylesheet, but the principle is much the
same.)
> The biggest weakness I see in the way most users
> test their stylesheets is that they use far too
> few different source documents to achieve good
> coverage of the logic, especially the error cases.
>
> Michael Kay
> Saxonica
>
> > On 14 Apr 2022, at 17:08, Roger L Costello
> <mailto:costello@xxxxxxxxx>costello@xxxxxxxxx
<<mailto:xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx>xsl-list-service@xxxxxxxxxxx
rrytech.com>
> wrote:
> >
> > Hi Folks,
> >
> > Scenario: You write an XSLT program that
> takes some input and outputs XML. You want to
> be sure that your XSLT program is correct.
> >
> > As I see it, there are two ways to help
> ensure the correctness of an XSLT program :
> >
> > 1. Pepper the XSLT program with assert statements, using xsl:assert.
> >
> > 2. Create a second XSLT program that queries
> the XML that was generated by the first XSLT
> program. This second XSLT program contains a
> series of XPath expressions to check various parts of the XML.
> >
> > Which of those do you use? Or do you use
> both? Or do you use something else?
> >
> > How do you ensure the correctness of your XSLT programs?
> >
> > /Roger
> >
> >
>
>
>
>
> --
>
> ...Wendell Piez... ...wendell -at- nist -dot- gov...
> ...wendellpiez.com... ...pellucidliterature.org... ...pausepress.org...
> ...<https://urldefense.com/v3/__http:/github.com/wendellpiez.__;!!N4vogdjhuJ
M!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhWgF2vKxLOw6cg3DXHTq7eRdXAW-pkoiBlg
SSc1ghRE7PKIXjqC6SSQwc1R14Qb9XXQ$>github.com/wendellpiez...
> ...gitlab.coko.foundation/wendell...
>
> <https://urldefense.com/v3/__http:/www.mulberrytech.com/xsl/xsl-list__;!!N4v
ogdjhuJM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhWgF2vKxLOw6cg3DXHTq7eRdXAW-
pkoiBlgSSc1ghRE7PKIXjqC6SSQwc1R12QKqii8$>XSL-List
> info and archive
>
> <https://urldefense.com/v3/__http:/lists.mulberrytech.com/unsub/xsl-list/345
3418__;!!N4vogdjhuJM!EGPurchrCkklwpQF6l9me3Je_xTvv1dU3Y_XP0DwhWgF2vKxLOw6cg3D
XHTq7eRdXAW-pkoiBlgSSc1ghRE7PKIXjqC6SSQwc1R1RIqInF0$>EasyUnsubscribe
> (by email)
> <http://www.mulberrytech.com/xsl/xsl-list>XSL-List info and archive
> <http://lists.mulberrytech.com/unsub/xsl-list/96802>EasyUnsubscribe
> (<>by email)


--
Contact info, blog, articles, etc. http://www.CraneSoftwrights.com/s/ |
Check our site for free XML, XSLT, XSL-FO and UBL developer resources |
Streaming hands-on XSLT/XPath 2 training class @US$125 (5 hours free) |
Essays (UBL, XML, etc.) http://www.linkedin.com/today/author/gkholman |