Re: [xsl] xsl:for-each-group using multiple threads?

["David Rudel fwqhgads@xxxxxxxxx" <xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx>]

Hi Marc,
Just for reference---it may not be possible or optimal in your
implementation---the way I was thinking of making this streamable is
the following:

1. First do a for-each-group using group-adjacent to break the set
into large sub-sets (large enough where multi-threading is still
useful but small enough that they fit inside your memory capacity.)
Since this uses group-adjacent it is streamable.

2. Within the for-each-group, create a map whose fibers are the
individual groups you want to process using multiple threads.

3. Process those groups using a for-each with the saxon:threads attribute.








On Tue, Jan 27, 2015 at 9:22 PM, Marc Kupietz kupietz@xxxxxxxxxxxxxxx
<xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx> wrote:
> Thanks, Michael and David for the quick replies! I already had the
> suspicion that if it was be so easily transferrable from xsl:for-each,
> in a proper way it would probably be done, already.
>
> David's map+stream approach brought be to an even easier solution for
> my problem: As it mainly concerns the 2nd stage of a 2-stage
> conversion I can easily produce several "grouped" result-documents
> (maybe named pipes) in the first stage and process them in parallel
> and using streaming in the second stage. This way I can use adjustable
> package sizes and all cores without running out of memory and
> hopefully end-up with i/o as bottleneck :)
>
> Marc
> 



-- 

"A false conclusion, once arrived at and widely accepted is not
dislodged easily, and the less it is understood, the more tenaciously
it is held." - Cantor's Law of Preservation of Ignorance.