RE: [xsl] AltovaXML bugs? (and other engines)

["Michael Kay" <mike@xxxxxxxxxxxx>]

The W3C test suites for XQuery and XML Schema are driven by a catalog of
test cases: the syntax and semantics of the catalog are standardized, and
there is a published set of tests and expected results. But there is no
standard test driver; it's up to each implementor (or other user of the test
suite) to write their own. The XQuery test suite also defines a standard
format for reporting results.

The W3C XSLT test suite is exactly the same, except that neither the catalog
format nor the test cases are published outside W3C.

The "framework" is thus essentially the catalog format. I don't think there
would be any problem in publishing the catalog format (syntax and semantics)
used by the W3C XSLT test suite, it's only the actual tests that are
problematic.

Michael Kay
http://www.saxonica.com/


> -----Original Message-----
> From: Scott Trenda [mailto:Scott.Trenda@xxxxxxxx] 
> Sent: 01 April 2009 22:53
> To: xsl-list@xxxxxxxxxxxxxxxxxxxxxx
> Subject: RE: [xsl] AltovaXML bugs? (and other engines)
> 
> Michael,
> 
> I think I get what you're asking. For public specifications 
> (like those offered by the W3C), there should be a common 
> flow to conformance testing:
> - run the implementation with the test document as the input
> - compare the result against the expected test result
> - repeat for all tests in the package
> 
> And it seems like that could be automated to some extent. Is 
> that automated process the "framework" you are alluding to? 
> If so, I'm also curious as to whether or not one exists! The 
> Acid tests for CSS take that framework for granted, since it 
> is, after all, the browser rendering the test. Where else do 
> you think we could look for such a framework, if the W3C 
> doesn't have a publicly available one? (I'm coming up short 
> for ideas on who else provides public specifications for 
> other companies to implement.)
> 
> It'd be nice if we could find or create an external one (for 
> XSLT 2.0 engines and some XSLT 1.0 engines, which largely 
> reside as their own executable files), and one that runs in 
> the browser (for the XSLT 1.0 engines provided as part of a 
> browser, like Transformiix and Opera).
> 
> ~ Scott
> 
> 
> -----Original Message-----
> From: Michael Ludwig [mailto:mlu@xxxxxxxxxxxxx]
> Sent: Wednesday, April 01, 2009 4:53 AM
> To: xsl-list@xxxxxxxxxxxxxxxxxxxxxx
> Subject: Re: [xsl] AltovaXML bugs? (and other engines)
> 
> I agree a publicly available test suite for both XSLT 1.0 and 
> 2.0 would be an excellent facility.
> 
> Is there an established standard or pattern or best practice 
> for coding tests? A framework you would just drop a new test 
> in so it becomes part of the test suite?
> 
> Maybe the W3C has come up with such a framework and is 
> willing to make it a public resource for the advancement of 
> the Good Cause while continuing, of course, to withhold the 
> test suite due to its partly shady, or intraceable, origins?
> 
> Michael Ludwig