Re: [xsl] XSLTPROC performance

[Colin Paul Adams <colin@xxxxxxxxxxxxxxxxxx>]

>>>>> "Andrew" == Andrew Welch <andrew.j.welch@xxxxxxxxx> writes:

    Andrew> On 21/12/2007, Colin Paul Adams <colin@xxxxxxxxxxxxxxxxxx> wrote:
    >> >>>>> "Andrew" == Andrew Welch <andrew.j.welch@xxxxxxxxx>
    >> writes:
    >> 
    Andrew> On 21/12/2007, Colin Paul Adams <colin@xxxxxxxxxxxxxxxxxx> wrote:
> >> The cost of the JVM start-up cannot be eliminated, but it can
    >> >> be amortised. For a fair comparison, you need to know what
    >> this >> overhead is, so you can make sensible judgements.
    >> 
    Andrew> I don't think it needs to be amortised... If you start the
    Andrew> timing at the correct point and ignore the first few runs
    Andrew> then I'd say you can safely forget about JVM startup and
    Andrew> warm-up for the remaining runs.
    >> 
    >> Then you've ignored it completely, so you don't get a fair
    >> comparison.

    Andrew> aahhh I see what you mean.... (the penny drops :)

    Andrew> I don't agree though - if you are comparing say Gestalt
    Andrew> Win32 version with Saxon running on Windows, then surely
    Andrew> it's only really fair to make the comparison after the
    Andrew> Java bytecode has become native machine code - after JVM
    Andrew> startup.

That depends upon what you mean by fair (that is, what are you trying
to measure?).

So, if what you are interested in is the performance of a web-server
system, then yes, that is probably the only meaningful measurement.
And for Gestalt, you would have to adopt the same methodology, in a
Goanna servlet (although i doubt if it would be significant, but I
have never measured it, so i could be very wrong).

If you workload (which is typically what I use XSLT for, but I am very
probably highly atypical, as I don't use XSLT much at all these days,
except for testing) is occasionally transforming gobodoc xml into an
html page, then ignoring the JVM start-up time would give me an absurd
conclusion, as it completely dominates the overall run-time.

In general, the run-time of an XSLT processor is going depend upon the
sum of a number of factors. For a Java-based processor, the JVM
warm-up time is such a factor (it is some kind of non-constant
workload-dependent function).
So to fully characterize the performance, you are going to need to specify a
number of parameters. In general, I mean. For a given workload,
processor comparisons are generally much simpler to do.

The only real value of a general inter-processor benchmark has, I
think, is to give a prospective user an idea of which processors to
measure against their own workloads. But still, I find it an
interesting topic (for about 9 years my principle job was a
performance analyst on mainframe systems).
-- 
Colin Adams
Preston Lancashire