Subject: Getting Some Facts Straight on Performance Claims (Was: : [ANNOUNCE] EZ/X - XML/XSL Processor Preview Now Available) From: "Steve Muench" <smuench@xxxxxxxxxxxxx> Date: Wed, 1 Dec 1999 20:49:09 -0800 |
Given the provocative comment: | XSL processing with EZ/X is usually 2-3 | times faster than Lotus/IBM/Apache or | Oracle, and even faster than that | when dealing with complex XML/XSL. you can imagine it was fun to dig in and try to see what was behind this claim... When I discovered that the benchmark program was running the EZ/X XSLT Engine with a "NullOutputStream" instead of actually writing out a file, I started to get suspicious... When I saw that the Oracle test was not using our latest 2.0.2.5 release and not using the proper XSLProcessor.processXSL() API to write output to a PrintWriter obeying all the rules of <xsl:output>, I got a little flustered... When I saw in the "readme.txt" file for the benchmarks the comment: | NOTE: The Oracle times will be very fast, | deceptively so, since they are not | doing much work. The stylesheet is | simply passed through because Oracle | does not yet conform with the October | 1999 specification. I knew this was just plain wrong, so decided I had to roll up my sleeves and debunk some of these statements. To compare apples to apples, I modified the benchmark to create a FileOutputStream in both cases and write the transformed output to a file instead of to some "magic" NullOutputStream. I used our latest release from http://technet.oracle.com/tech/xml (which *is* compliant with the XSLT REC) and ran their benchmarks again... Here's the results I came up with below. As you can see, while we don't win every single test, it is certainly not the case that EZ/X is "2-3 times faster." Once the test was modified to actually produce output to a file, it was clear that their output documents were appreciably larger in 3 of the 4 cases. I'd recommend the EZ/X folks do their homework a little better before making such claims in the future... Overview ~~~~~~~~ Machine: Pentium III, Dell Optiplex, 400Mhz JDK 1.2.2 that comes with JDeveloper 3.0 XST Engines Compared -------------------- EZ/X Build 60 Oracle XSLT Release 2.0.2.5 Benchmark Description --------- ------------------------------ 1 Ken Holman's showtree-19991008.xsl on a 667 byte source document 2 Mike Brown's Fancy_XML_Tree_Viewer_34.xsl on the same 667 byte source document 3 formatPlay.xsl on a 2,096,966 byte source document produced by taking Jon Bosak's 209,710 byte all_well.xml (Shakespeare's "All's Well that Ends Well") and repeating it ten times inside the file. 4 dict2.xsl on a 197,164 byte source document that is file 1 of 4 of Webster's Revised Unabridged Dictionary. Performance Info ~~~~~~~~~~~~~~~~ Benchmark EzX OraXSL Diff% --------- -------- -------- ----------- 1 63ms 66ms 5% slower 2 220ms 102ms 54% faster 3 41764ms 28503ms 32% faster 4 1574ms 2005ms 27% slower File Size Information Benchmark XML Src EzX Out OraXSL Out Diff% --------- ---------- ---------- ---------- ------------ 1 667 2,323 1,658 29% smaller 2 667 31,367 8,592 73% smaller 3 2,096,966 6,076,459 4,090,477 33% smaller 4 197,164 134,436 138,053 3% larger =========================================================== Benchmark 1 =========================================================== E:\xml\ezx\examples\xsl\benchmark> run Running model.xml with showtree-19991008.xsl 100 times... xsl.processor property not set, using 'Ezx' Testing EZ/XSL build 60 100 times... After 10: 76.2ms After 20: 62.0ms After 30: 61.1ms After 40: 62.1ms After 50: 61.1ms After 60: 62.1ms After 70: 62.1ms After 80: 61.1ms After 90: 62.1ms After 100: 61.1ms Average transform time is 63ms E:\xml\ezx\examples\xsl\benchmark> run_oracle Running model.xml with showtree-19991008.xsl 100 times... Testing Oracle XML Parser 2.0.2.5.0 Production 100 times... After 10: 77.1ms After 20: 65.1ms After 30: 67.1ms After 40: 65.1ms After 50: 65.1ms After 60: 64.1ms After 70: 68.1ms After 80: 65.1ms After 90: 65.1ms After 100: 65.1ms Average transform time is 66ms =========================================================== Benchmark 2 =========================================================== E:\xml\ezx\examples\xsl\benchmark> run2 Running model.xml with Fancy_XML_Tree_Viewer_34.xsl 100 times... xsl.processor property not set, using 'Ezx' Testing EZ/XSL build 60 100 times... After 10: 233.4ms After 20: 219.3ms After 30: 219.3ms After 40: 219.3ms After 50: 220.3ms After 60: 220.3ms After 70: 218.4ms After 80: 220.3ms After 90: 219.3ms After 100: 219.3ms Average transform time is 220ms E:\xml\ezx\examples\xsl\benchmark> run2_oracle Running model.xml with Fancy_XML_Tree_Viewer_34.xsl 100 times... Testing Oracle XML Parser 2.0.2.5.0 Production 100 times... After 10: 118.2ms After 20: 102.1ms After 30: 100.2ms After 40: 99.1ms After 50: 100.2ms After 60: 100.1ms After 70: 102.1ms After 80: 102.2ms After 90: 101.1ms After 100: 101.2ms Average transform time is 102ms =========================================================== Benchmark 3 =========================================================== E:\xml\ezx\examples\xsl\benchmark> run3 Running all_well_10x.xml with formatPlay.xsl 5 times... xsl.processor property not set, using 'Ezx' Testing EZ/XSL build 60 5 times... Average transform time is 41764ms E:\xml\ezx\examples\xsl\benchmark> run3_oracle Running all_well_10x.xml with formatPlay.xsl 5 times... Testing Oracle XML Parser 2.0.2.5.0 Production 5 times... Average transform time is 28503ms =========================================================== Benchmark 4 =========================================================== E:\xml\ezx\examples\xsl\benchmark> run4 Running sample200k.xml with dict2.xsl 50 times... xsl.processor property not set, using 'Ezx' Testing EZ/XSL build 60 50 times... After 10: 1575.2ms After 20: 1554.3ms After 30: 1581.2ms After 40: 1570.3ms After 50: 1592.3ms Average transform time is 1574ms E:\xml\ezx\examples\xsl\benchmark> run4_oracle Running sample200k.xml with dict2.xsl 50 times... Testing Oracle XML Parser 2.0.2.5.0 Production 50 times... After 10: 2042.9ms After 20: 1990.9ms After 30: 2013.9ms After 40: 1988.8ms After 50: 1991.9ms Average transform time is 2005ms _________________________________________________________ Steve Muench, Consulting Product Manager & XML Evangelist Business Components for Java Development Team XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
Current Thread |
---|
|
<- Previous | Index | Next -> |
---|---|---|
[ANNOUNCE] EZ/X - XML/XSL Processor, Rick Ross | Thread | Test, Daniel Hinz |
Interpretation of (preceding::foo)[, Paul W. Abrahams | Date | Re: Interpretation of (preceding::f, James Clark |
Month |