RE: [xsl] 10,000 document()'s

Subject: RE: [xsl] 10,000 document()'s
From: Rod Humphris - FLPTN <rod.humphris@xxxxxxxxxxxxxx>
Date: Tue, 8 Apr 2003 17:25:56 +0100
Peter

Forgive me if I've missed the point, but why not use old fashioned
procedural code? 

as in:

load the input into a dom object and then itterate through each file element
querying the specific document and  adding a wordcount element as you go.

That way you have no memory issues, and as you say speed is not a problem.

Cheers 

Rod

-----Original Message-----
From: Peter Binkley [mailto:Peter.Binkley@xxxxxxxxxxx]
Sent: 08 April 2003 17:06
To: 'xsl-list@xxxxxxxxxxxxxxxxxxxxxx'
Subject: [xsl] 10,000 document()'s


I need advice on how to tackle this problem: I've got a file that contains a
list of about 10,000 other files, and I want to process the list so as to
add a wordcount for each of the external files. Something like this:

Input:

<files>
	<file>
		<filename>/path/to/file/2844942.xml</filename
	<file>
	<file> .... </file>
<files>

Output:

<files>
	<file>
		<filename>/path/to/file/2844942.xml</filename
		<wordcount>2938</wordcount>
	<file>
	<file> ....	</file>
<files>

The obvious approach is to use a for-each loop that includes a variable that
opens the external file using a document() call. The problem is that the
process inevitably runs out of memory, both with Saxon and Xalan. It seems
that the variables are passing out of scope and being destroyed as they
should, but I gather from a posting by Michael Kay
(http://www.biglist.com/lists/xsl-list/archives/200212/msg00507.html) that
all of those document() source trees are remaining in memory throughout the
transformation, adding up to megabytes of data.

Can anyone suggest a strategy? The process doesn't have to be fast, it just
has to finish.

Peter Binkley
Digital Initiatives Technology Librarian
Information Technology Services
4-30 Cameron Library
University of Alberta Libraries
Edmonton, Alberta
Canada T6G 2J8
Phone: (780) 492-3743
Fax: (780) 492-9243
e-mail: peter.binkley@xxxxxxxxxxx




 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list


________________________________________________________________________
This email has been scanned for all viruses by the MessageLabs SkyScan
service. For more information on a proactive anti-virus service working
around the clock, around the globe, visit http://www.messagelabs.com
________________________________________________________________________


________________________________________________________________
Any opinions expressed in this email are those of the individual and not necessarily the Company. Unless expressly stated to the contrary, this email is not intended to give rise to a new, or affect an existing, contractual or other legal relationship.This email and any files transmitted with it, including replies and forwarded copies which may contain alterations) subsequently transmitted from the Company, are confidential and solely for the use of the intended recipient. The unauthorised use, disclosure or copying of this email, or any other information contained or attached,is prohibited and could, in certain circumstances, be a criminal offence.

If you have received this email in error please notify the sender as soon as possible.

This footnote also confirms that this email message has been swept for the presence of computer viruses.

www.focusdiy.co.uk
_________________________________________________________________

________________________________________________________________________
This email has been scanned for all viruses by the MessageLabs SkyScan
service. For more information on a proactive anti-virus service working
around the clock, around the globe, visit http://www.messagelabs.com
________________________________________________________________________

 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list


Current Thread