Re: [xsl] Latest XSLTMark benchmark

Subject: Re: [xsl] Latest XSLTMark benchmark
From: Uche Ogbuji <uche.ogbuji@xxxxxxxxxxxxxxx>
Date: Wed, 04 Apr 2001 11:39:16 -0600
> > I only wish someone had bothered to post a request to the Xalan list for
> > a review of their driver, or at least some notice that they were doing 
> > this.
> It is important to note that XSLTMark was announced on xsl-list in October
> of 2000 and included a Xalan-C++ driver then. Many developers (Michael Kay,
> the folks at Microsoft, Oracle and others) contacted us at that time and
> made comments on the compliance tests, the drivers, and so on.

You did not cover 4XSLT then, nor did any one of you contact the 4Suite list 
to ask about realistic test driver methodology.

> There was 
> then another release, and more work, and only then did it receive wider 
> distribution on All the source code was always available, and 
> indeed you can still download all the versions and corresponding results
> from the website.

Someone else's comment that 1.2.1 is the wrong version led me to dig around 
some more, and I did find the 2.0 packages.

It does include


but this file really makes no sense whatsoever.  Do you have some useful 
documentation of the 4SS driver operation?  And what is this comment all about:

### You've got to have all your stuff set up in a magic
### way for this to work.  

Was your intent to post dubious results so that you can get me to do all your 
work for you?

> Both the Saxon and the Xalan driver have not changed
> in any material way in about 5 months.
> > Companies which post benchmarks ought to be more responsible.
> Look, there was something we missed in a couple of (admittedly important)
> drivers. We will set it right. I would be grateful for your involvement
> and any help you can provide, both for Xalan and the benchmark in general.

Your lack of documentation of method, constraints, restrictions, architecture, 
environment, etc. is *highly* unprofessional.  A benchmark is a scientific 
exercise, requiring all the attention to controls of any such scientific 
inquiry.  You can't go posting benchmarks with all corners cut and expect to 
retain credibility.

Uche Ogbuji                               Principal Consultant
uche.ogbuji@xxxxxxxxxxxxxxx               +1 303 583 9900 x 101
Fourthought, Inc.                
4735 East Walnut St, Ste. C, Boulder, CO 80301-2537, USA
Software-engineering, knowledge-management, XML, CORBA, Linux, Python

 XSL-List info and archive:

Current Thread