Subject: RE: [xsl] Performance optimization and avoiding Out of Memory errors From: "James Fuller" <james.fuller@xxxxxxxxxx> Date: Fri, 6 Sep 2002 11:09:43 +0100 |
> -----Original Message----- > From: owner-xsl-list@xxxxxxxxxxxxxxxxxxxxxx > [mailto:owner-xsl-list@xxxxxxxxxxxxxxxxxxxxxx]On Behalf Of Lewden Romain Hello Lewden, > I'm working with quite important XML Data Base ( 10 Mo) and trying to > extract information and transform it into HTML files and rtf format. > I work with Xt processor (release 19990511) > I have two big problems : > 1)time of genration is very important > I create a predocument xml of 10 Mo (with 10 chained requests about 6,000 > objects) > This step takes 3 hours, with a station of 512 RAM Memory first off, with large processing scenarios, as you have....have you considered using a SAX approach ? what is the format of this pre-document.....I assume it is XML. what are you doing in this pre-step ? have you thought of using an XML native repository like xindice or Tamino....seems like you also benefit from db style optimisations dealing with so much data. > Then I create in HTML a file for each object; it takes me 14 hours. > it is a recursive call : > <dog level="0" name="Toto1"> > <dog level="1" name="Toto1"> > <dog level="2" name="Toto1" > <dog level="3" name="Toto1"> > <cat>ieodfli</cat> > <cat>ieodfli</cat> > </dog> > </dog> > <dog level="2" name="Mydog"> > <dog level="3" name="Toto1"> > <cat>ieodfli</cat> > <cat>ieodfli</cat> > </dog> > </dog> > </dog> > </dog> > I've a template for "dog", with recursive calls, and if I'm in level "2" I > create a separate HTML page. I've 6000 elements at level 2 and it takes me > 10 hours. How can I optimize this time ? hmmmm, why do you store level ? I think that if u described your entire process that we could come up with something optimised.....otherwise the structure of your XML ( is this lorem ipsum, or are u really doing something with dogs and cats ?) is slightly confusing. without seeing your template, or use of keys, etc.....it is hard to comment. > 2) When I work in rtf, I can have out of memory errors, because I > write less > files but they become huge (9 Mo octets). > All the output file is kept in memory in memory until the stylesheet is > performed. > How can I introduce an option to force my program to write "in > real time" in > the output file ? my gut feeling, is that you should investigate Saxon ( saxon.sourceforge.net ) as it has some switches to assist in large file processing.....and it is also a java XSLT processor, which will integrate with your environment easily. gl, jim fuller XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
Current Thread |
---|
|
<- Previous | Index | Next -> |
---|---|---|
[xsl] Performance optimization and , Lewden Romain | Thread | [xsl] XSLT class, Chen, Gin |
RE: [xsl] How to preserve numerical, Michael Kay | Date | Re: [xsl] A few questions to the ex, juggy |
Month |