Re: [xsl] Latest XSLTMark benchmark

Subject: Re: [xsl] Latest XSLTMark benchmark
From: David_N_Bertoni@xxxxxxxxx
Date: Fri, 30 Mar 2001 12:25:37 -0500
Mike Kay wrote:
> A new set of results for the XSLTMark benchmark has been published at
> The benchmarking methodology is to compile the stylesheet, and then run
it a
> large number of times over the same source document, averaging the
> I have taken a look at some of the drivers for different XSLT processors.
> As far as I can see the drivers for Oracle, MSXML, and Sun XSLTC parse
> load the source document once, and then transform it many times from the
> same in-memory tree, while the drivers for xt, Saxon, and Xalan parse the
> source document and build the tree each time round the loop.
> Since parsing and building the tree account for a significant proportion
> the transformation time, this would significantly skew the results.
> Are my suspicions correct?

That was my conclusion as well.  I also examined their Xalan-C++ driver,
which is poorly written, leaks memory, and uses a suboptimal source tree
for the transformation.  It took 15 minutes for me to re-write their
driver, and the results are significantly improved.  I only wish someone
had bothered to post a request to the Xalan list for a review of their
driver, or at least some notice that they were doing this.

> (Even if this is corrected, the methodology is flawed, because a
> could legitimately cache the results of parsing the document in memory,
> unknown to the driver. Ideally the source file should be trivially
> between each iteration to prevent this).

Companies which post benchmarks ought to be more responsible.

Dave Bertoni
IBM Corporation (Xalan-C++ committer)

 XSL-List info and archive:

Current Thread