RE: [xsl] Max size?

Subject: RE: [xsl] Max size?
From: "Michael Kay" <michael.h.kay@xxxxxxxxxxxx>
Date: Wed, 8 Jan 2003 11:07:48 -0000
> If you are using xalan with large files look at SAX2SAX 
> sample.  This uses streaming so it avoids building the data 
> structure for the document as much as possiable. 
> http://xml.apache.org/xalan-j/samples.html#sax2sax

This is a popular misconception. If you supply SAX input to an XSLT
processor, it will build its own tree internally. It will probably be
more efficient (in both time and memory) than supplying a DOM as input,
but the tree is still needed.

But as others have said, no respectable processor should have trouble
with 1Mbyte. At 10Mb you may need to start thinking about how much
memory to allocate, at 50Mbyte you may need to buy some more memory... I
have heard of Saxon users processing 100Mbytes successfully.

Michael Kay
Software AG
home: Michael.H.Kay@xxxxxxxxxxxx
work: Michael.Kay@xxxxxxxxxxxxxx  

> 
> If you are working on linux I found libxml2 to have similar 
> performance to MSXML3.0 on files of about 1Meg size but have 
> not tested it on anything larger.  It can be found at. 
> http://xmlsoft.org/
> 
> Edward Middleton
> 
> -----Original Message-----
> From: Peter Finch [mailto:peter@xxxxxxxxxxx]
> Sent: Wednesday, January 08, 2003 6:39 PM
> To: xsl-list@xxxxxxxxxxxxxxxxxxxxxx
> Subject: Re: [xsl] Max size?
> 
> 
> Hi Heinz,
> 
>  > I have a "hypothetical" question.  If I were to generate 
> very large XML  > files (upwards of 1 Meg?), would I 
> encounter problems when processing with  > XSLT, considering 
> that I would want to sort on various elements.  >  > We are 
> busy doing some tests and I would be interested to compare 
> our  > results to existing findings. (We are using 
> xalan/xercses on Linux).
> 
> It Depends on your machine, parser and the style sheet, but a 
> 1 Meg XML file should not be a problem. Some of my customers 
> work with BIG files... 10's to 100's of Meg and more (believe 
> it or not).
> 
> I recently did some work on a 157Meg document. Of all the 
> parsers I tried I have to admit the one from Microsoft 
> (msxsl) flew and used up a relatively small amount of memory 
> so I can recommend it as a place to start looking (Hats off 
> to the guys-n-girls at MS for that one at least).
> 
> On a Dual AMD1400 with 512meg RAM a 13 Meg XML 5 Seconds and 
> a 157 Meg XML file took about 4 minutes with memory usage 
> peeking around 380Meg. It's all about memory as (I think) 
> most of the parsers build a tree in memory before they 
> starting to processes the transformations.
> 
> I'm pretty sure most of the Java based pasers will have no 
> trouble with a 1Meg(ish) document.
> 
> I hope this helps.
> 
> Cheers,
> Peter
> 
> -- 
>     ___
>    (OvO)
>    /:::\
>    \|:|/
> /--m-m--------------------------------+
> | Peter Finch (p.finch@xxxxxxxxxxx)   |
> | Home Planet Software                |
> | http://www.homepla.net/             |
> /-------------------------------------/
> 
> 
> 
> 
>  XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list
> 
> 
>  XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list
> 
> 


 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list


Current Thread