Re: [xsl] XSLT Unit testing

[Florent Georges <lists@xxxxxxxxxxxx>]

Andrew Welch wrote:

  Hi,

> > My experience does suggest that it's not
always going to be a
> > worthwhile investment of effort, but that when it is
> > worthwhile it can be very worthwhile.

> That kind of sums up how I feel
about writing tests in general,
> most of the time its a chore but once they
are in place you are
> really glad you did it :)

  And that's what I think of
XSpec.  For instance, EXPath modules
are (more or less) tested using XSpec.
For some modules, like
the HTTP Client and the ZIP Facility, that's sometimes
a bit
tricky to set up the tests correctly, due to the side effects
(and
because we want to test exactly those side effects).  And
because XSpec
supports both XSLT and XQuery, the same suites can
be used to test
implementations in both of them.

  But once it's been done, it is really
great to have a automatic
safety net when you develop.

> For what its worth,
I've not used xslt specific test
> frameworks, instead using standard junt
tests that run the
> transform and then use xpaths on the result (well,
xquery).
> Because its just a normal junit test, it slots straight in with
>
existing build process.

  For the build processes, because XSpec is based on
standard
XSLT transforms, I think it is not such a big deal.  "All one has
to
do" is to plug XSLT transforms into his/her build system.  Of
course, this is
not that easy, because you have better things to
do than creating a new Ant
task or a new Maven archetype.

  That's why I'd like to have several test
harnesses for XSpec,
for the major build processes out there.  For instance,
jxsl
<http://code.google.com/p/jxsl/> plug it into Maven.  By using
their
archetype, you just put your XSpec suites in the test/
subdirectory in your
project, and Maven will then run them
automatically.

  Any help welcome! :-)
Further than the build processes (which is probably the next
big step for
XSpec), it would be nice to have some IDE support.
For oXygen and other XML
IDEs, of course.  But also in NetBeans,
Eclipse, Visual Studio, and other Java
and C# IDEs.  So the
developer could develop his/her test suite like he/she
does for,
say, JUnit, and ust right click the suite to run it from the IDE
(with the same green/red bar and bullets he/she is used to).

  Here also, any
help welcome :-)

  What I do, is to create a simple Makefile or a simple
shell
script in the test/ directory of my projects in order to compile
the
suites, run them, then format the report to HTML.  Each of
those step is a
regular XSLT transform.  The second step can be
an XQuery evaluation in case I
am testing XQuery instead (with
eXist for instance I use its REST API to
evaluate the query on
the eXist database, from the script on the command
line).

  That is not a lot of work, but I know people prefer to click on
buttons and see the result directly.

> The basic code is below, most of it is
boilerplate to run the
> transform (using Saxon) and make the actual test
methods
> small. To use it, just copy and paste it, edit the XML and XSLT
>
paths, and then start adding tests.

  I think that is one of the advantage of
XSpec.  All that
already exists, and more.  And maybe another difference is
that
the test suites are more clear.  You don't have to look into the
Java
code for what is relevant to the test and what is not.  Your
example would be,
in XSpec, the following:

    <t:description
xmlns:t="http://www.jenitennison.com/xslt/xspec";
stylesheet="/tmp/andrew.xsl">

       <t:scenario label="the sample
transform">

          <t:context href="/tmp/andrew.xml"/>
<t:expect label="the first foo string"
test="//foo[1]/string(.) eq '123'"/>

          <t:expect label="foo content">
<root>
                <foo>123</foo>
                <foo>456</foo>
</root>
          </t:expect>

       </t:scenario>

    </t:description>
With the added value that the HTML report presents clearly the
labels, and in
case of failure present a colored diff between the
expected XML and the actual
value.  To run this example (with the
appropriate tested stylesheet and test
input in /tmp), use the
following command:

    > xspec.sh andrew.xspec
Testing with SAXON HE 9.3.0.5
    the sample transform
    the first foo
string
    foo content
    passed: 2 / pending: 0 / failed: 0 / total: 2
The report is in a new subdirectory: xspec/andrew-result.html,
as well as the
XML report and the compiled stylesheet (or query).
If you want to control
everything (like the exact filenames, or
if you have to integrate that process
in your build environment),
the following are the equivalent transform using
Saxon from the
command line:

    # compiling the suite to the equivalent
stylesheet
    > saxon -xsl:$XSPEC_HOME/generate-xspec-tests.xsl \
-s:andrew.xspec -o:andrew-compiled.xsl

    # actually run the tests
    >
saxon -xsl:andrew-compiled.xsl -o:andrew-report.xml \
-it:{http://www.jenitennison.com/xslt/xspec}main
    Testing with SAXON HE
9.3.0.5
    the sample transform
    the first foo string
    foo content
# formating the report to HTML
    > saxon
-xsl:$XSPEC_HOME/format-xspec-report.xsl \
        -s:andrew-report.xml
-o:andrew-report.html
    passed: 2 / pending: 0 / failed: 0 / total: 2

  You
also have an Ant build script and an XProc pipeline to run
a suite available
on the XSpec project website:

    http://code.google.com/p/xspec/

  Regards,
-- 
Florent Georges
http://fgeorges.org/
http://h2oconsulting.be/