EZ/X Speed: Rebunking Oracle's Debunk: WAS (Getting Some Facts Straight on Performance Claims)

["Michael Sick" <mike@xxxxxxxxxxxxx>]

REBUNKING ORACLE'S DEBUNK
Last week, Oracle's Steve Muench suggested that XSL transforms with
Activated EZ/X are not, in fact, usually 2-3X faster than his company's most
recent XML/XSL offering. Steve is dead wrong, and though he curtly suggested
we do our homework better, we'll need to help him with his this time...

WHO'S FASTEST?
First, however, before any major flame erupts about Java XSL speed,
Activated Intelligence wants to acknowledge that the current speed leader is
James Clark's XT - a fine package and worthy of respect for many reasons. We
do not claim EZ/XSL to be faster than XT yet <grin>, though our EZ/XML
parser speeds past his XP parser package... If Activated also manages to
beat James' XSL package it will be a good day for all Java XSL developers,
because a new performance standard will have been set. We look forward to
that day, hopefully not too far off.

A REVIEW
Steve didn't offer any source code, made some questionable assumptions, and
then posted his own benchmark numbers. These may have satisfied him, but
since he didn't post any code we couldn't help him identify what he did
wrong.

THANKS FOR HELPING, STEVE!
Steve's comments helped us identify an EZ/X filesize quirk we had
overlooked. We're grateful for his critical insights, Activated is always
glad to get help from our competitors. In fact, we have now addressed an
issue that caused EZ/X to produce output files that were legal but
unnecessarily large. Happily, fixing this quirk made EZ/X even faster -
thanks, Steve!

ACTUAL FILES VS. NULL STREAMS
Steve suggested that writing to our NullOutputStream class was somehow
causing EZ/X to report unfairly fast results. Steve's suggestion is simply
wrong, and our goal was to measure the transform time and not Java's i/o
speed. We've now included an additional benchmark that shows EZ/X speed
compared to Oracle when writing to a BufferedOutputStream. Writing to actual
files rather than to our null stream really doesn't change anything.

THE TEST RESULTS
As you can see, EZ/X is just about 2-3x faster than Oracle's latest version,
affirming our prior claim. We'll be delighted for Steve to produce code that
backs up his claims, but you can try ours out right now - just download EZ/X
and give it a shot - http://www.activated.com/download/ezx.zip

K6-450 Processor running Windows NT 4.0 SP5
-------------------------------------------
Benchmark   EZ/X     Oracle    Times Faster With EZ/X
--------- --------  -------- -----------
  1           32ms      81ms     2.5
  2           42ms      96ms     2.3
  3          735ms    1819ms     2.5
  4         9110ms   24256ms     2.7

Dual Pentium II 500 Processors running Windows 2000 Build 2183 (RC3)
-------------------------------------------------------------------
Benchmark   EZ/X     Oracle    Times Faster With EZ/X
--------- --------  -------- -----------
  1           26ms      58ms     2.2
  2           39ms      78ms     2.0
  3         7485ms   21423ms     2.9
  4          541ms    1549ms     2.9

Dual Pentium Pro 200 Processors running Windows NT 4.0 SP3
----------------------------------------------------------
Benchmark   EZ/X     Oracle    Times Faster With EZ/X
--------- --------  -------- -----------
  1           67ms     137ms     2.0
  2           95ms     183ms     1.9
  3        18749ms   48899ms     2.6
  4         1367ms    3680ms     2.7


JVM
---
All benchmarks were run using Sun JDK version "1.2.2", build
JDK-1.2.2-001 with native threads and symcjit enabled. In all the
tests the VM is set to use the default heap settings, except in one
case when running Oracle to process "all_well_10x.xml" with
"formatPlay.xsl". We found it necessary to increase the Java heap
significantly in order to keep Oracle's package from throwing an
OutOfMemoryException.


Benchmark Description (from Steve's mail)
--------- ------------------------------
   1      Ken Holman's showtree-19991008.xsl
          on a 667 byte source document

   2      Mike Brown's Fancy_XML_Tree_Viewer_34.xsl
          on the same 667 byte source document

   3      formatPlay.xsl
          on a 2,096,966 byte source document
          produced by taking Jon Bosak's
          209,710 byte all_well.xml (Shakespeare's
          "All's Well that Ends Well") and repeating
          it ten times inside the file.

   4      dict2.xsl
          on a 197,164 byte source document
          that is file 1 of 4 of Webster's
          Revised Unabridged Dictionary


Best Regards,

Michael Sick
Activated's EZ/X Team

begin:vcard 
n:Sick;Mike
tel;pager:917 975 6860
tel;cell:917 975 6860
tel;fax:(212) 896-8222
tel;work:(919) 319 3460
x-mozilla-html:TRUE
url:www.activated.com
org:Activated Intelligence
adr:;;1001 Winstead Drive;Cary;NC;27513;US
version:2.1
email;internet:mike@xxxxxxxxxxxxx
title:Vice President
note:Activated Intelligence - Bringing Life To The Net, and The Net To Life
x-mozilla-cpt:;-19184
fn:Mike Sick
end:vcard