Subject: Re: [xsl] Running the same transformation on many input files, optimisation possible? From: "Dimitre Novatchev dnovatchev@xxxxxxxxx" <xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx> Date: Sun, 15 Dec 2019 17:49:37 -0000 |
I would definitely use the `collection()` function, then would try to process the documents in parallel using the `saxon:threads` extension attributes with a value dependent on the number of cores on the machine. https://www.saxonica.com/html/documentation/extensions/attributes/threads.html Trying to generalize this a little bit further, if we have N machines, we could send N HTTP requests (why not using the document() function) giving each machine a non-overlapping pattern for the set of files it should process. Of course, besides using `collection()` extensively, I haven't ever tried the other stuff I proposed above -- would be really interesting to try. Cheers, Dimitre On Sun, Dec 15, 2019 at 1:02 AM Trevor Nicholls trevor@xxxxxxxxxxxxxxxxxx < xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx> wrote: > Hi > > > > An application I am working on contains a large number of source documents > which are all run through the same series of transformations. While > initially the build process didn't take long the cost of repeatedly > initialising the XSL processor soon adds up, so I am looking at ways to > streamline it. > > > > Our processor of choice is Saxon (currently we are using 8.7.3) so I can > shift this question to the Saxon list if there are extensions there that > are relevant. > > > > So the question; given a script that essentially includes the following: > > > > cd documents > > for d in `cat dlist`; do > > cd $d > > for f in `cat flist`; do > > java -jar $SAXONDIR/saxon8.jar -o $f.new.xml $f.xml > $SCRIPTDIR/transform.xsl doc=$d file=$f > > done > > done > > > > is there a mechanism which would allow a single Java process to perform > the equivalent? > > > > Thanks > > T > > > XSL-List info and archive <http://www.mulberrytech.com/xsl/xsl-list> > EasyUnsubscribe <http://lists.mulberrytech.com/unsub/xsl-list/782854> (by > email <>) > -- Cheers, Dimitre Novatchev --------------------------------------- Truly great madness cannot be achieved without significant intelligence. --------------------------------------- To invent, you need a good imagination and a pile of junk ------------------------------------- Never fight an inanimate object ------------------------------------- To avoid situations in which you might make mistakes may be the biggest mistake of all ------------------------------------ Quality means doing it right when no one is looking. ------------------------------------- You've achieved success in your field when you don't know whether what you're doing is work or play ------------------------------------- To achieve the impossible dream, try going to sleep. ------------------------------------- Facts do not cease to exist because they are ignored. ------------------------------------- Typing monkeys will write all Shakespeare's works in 200yrs.Will they write all patents, too? :) ------------------------------------- Sanity is madness put to good use. ------------------------------------- I finally figured out the only reason to be alive is to enjoy it.
Current Thread |
---|
|
<- Previous | Index | Next -> |
---|---|---|
Re: [xsl] Running the same transfor, Graydon graydon@xxxx | Thread | Re: [xsl] Running the same transfor, Dimitre Novatchev dn |
Re: [xsl] Running the same transfor, Graydon graydon@xxxx | Date | Re: [xsl] Running the same transfor, Dimitre Novatchev dn |
Month |