RE: [xsl] Latest XSLTMark benchmark

["Kevin Jones" <kjouk@xxxxxxxxxxx>]

>As far as I can see the drivers for Oracle, MSXML, and Sun XSLTC parse and
>load the source document once, and then transform it many times from the
>same in-memory tree, while the drivers for xt, Saxon, and Xalan parse the
>source document and build the tree each time round the loop.
>
>Since parsing and building the tree account for a significant proportion of
>the transformation time, this would significantly skew the results.
>
>Are my suspicions correct?
>

I have just had a look at the Saxon driver and think you are correct. This
also appear to have been true in the earlier (1_2_1) release as well. It was
clearly intended that the loadInput() call should actually load the input as
in the other drivers not just open an input stream.

There is also a functionality issue here about using an object model as
input because its not clear if all the processors will perform white-space
stripping etc on a pre-loaded document or just use it as is. Another angle
of variance is how a processor saves the output, some of the tests can
become I/O bound so it can make a big difference. The MS driver has lots of
output handling code presumably for performance reasons.

>(Even if this is corrected, the methodology is flawed, because a processor
>could legitimately cache the results of parsing the document in memory,
>unknown to the driver. Ideally the source file should be trivially modified
>between each iteration to prevent this).

I don't think this is an issue with XSLTMark as its currently written since
the driver is (normally??) requested to load the XSL and XML before the
timing starts. It could be a problem with the little benchmark I have but I
manually check to make sure the files are being reloaded.

Regards,
Kev.

>
>Michael Kay
>Software AG
>
>
> XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list


 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list