RE: [xsl] What are the best XSLT 1 or XSLT 2 performance benchmarks available today?

["Michael Kay" <mike@xxxxxxxxxxxx>]

It's unfortunate that XSLTMark disappeared. I have a copy, and the license
says:

Permission is hereby granted, free of charge, to any person obtaining 
a copy of this software and associated documentation files (the "Software"),
to:
 1. use the Software
 2. publicly distribute unmodified copies of the Software

which I would do, except that I unfortunately don't have an unmodified copy
and can't easily reconstruct one; the license doesn't allow me to distribute
a modified copy.

I use XSLTMark (mainly to check for regression between Saxon releases), but
like any benchmark it has its limitations. I'm not aware of anything better.

Before running or creating a benchmark I think the key things you need to
ask yourself are:

(a) how important is document loading time? In some workloads it is
absolutely dominant, in others it makes no significant contribution.
Benchmarks typically either attach no weight to it, or attach far too much.

(b) similarly for stylesheet compilation time.

(c) are you running a server workload where multi-threading plays a
significant role?

The worst examples of benchmarks I have seen were published with drivers for
different products that balanced these factors differently depending on the
product, for example one driver would build a source document tree once and
then transform it repeatedly, whereas another driver would build the tree
repeatedly once for each transformation. This just betrays a total lack of
understanding of the factors that drive transformation performance.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

> -----Original Message-----
> From: Dimitre Novatchev [mailto:dnovatchev@xxxxxxxxx] 
> Sent: 14 January 2010 22:23
> To: xsl-list@xxxxxxxxxxxxxxxxxxxxxx
> Subject: [xsl] What are the best XSLT 1 or XSLT 2 performance 
> benchmarks available today?
> 
> Hi,
> 
> Does anybody know what, if any, XSLT 1 or XSLT 2 performance 
> benchmarks are available and which of them are the best?
> 
> I tried to find XSLTMark, but it seems to have disappeared.
> 
> Also, is it meaningful to try to compare the performance of 
> an XSLT 1.0 processor with that of an XSLT 2.0 (basic or SA)?
> 
> If, for example, people are considering to make the move from 
> using XSLT 1.0 to XSLT 2.0 and need assuarance that the 
> famous XSLT 2.0 processor "X" is not "significantly less 
> performant" than their current XSLT 1.0 processor "Y", what 
> would be the best performance experiment to provide some 
> *meaningful and useful* data for them to decide?
> 
> --
> Cheers,
> Dimitre Novatchev
> ---------------------------------------
> Truly great madness cannot be achieved without significant 
> intelligence.
> ---------------------------------------
> To invent, you need a good imagination and a pile of junk
> -------------------------------------
> Never fight an inanimate object
> -------------------------------------
> You've achieved success in your field when you don't know 
> whether what you're doing is work or play
> -------------------------------------
> I enjoy the massacre of ads. This sentence will slaughter ads 
> without a messy bloodbath.