[xsl] Transform a million XML documents

Subject: [xsl] Transform a million XML documents
From: "Costello, Roger L. costello@xxxxxxxxx" <xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx>
Date: Fri, 10 Feb 2017 18:43:09 -0000
Hi Folks,

Eliot Kimber raised a neat question on the SAXON mailing list.

Here is a summary of the ensuing discussion.

Scenario: There are a million XML documents that need to be transformed. Each
file is in the 1-4KB range. The files are organized into directories about 4
or 5 deep and some directories have 100s or 1000s of files.

Use XSLT to do the transformations.

Specifically, use the XSLT collection() function along with

Transforming a million files is easily handled by Saxon-EE, which uses
multiple threads for document parsing (equally xsl:result-document will use
multiple threads for writing the result). A key thing to remember is to use
saxon:discard-document() to ensure that the documents are garbage collected
after processing.

Here is the XSLT code:

<xsl:for-each select="for $x in
return saxon:discard-document(f.doTransform($x))">

Current Thread