Re: [xsl] The time to execute a function is 625 times greater than the sum of the times to execute the statements in the function

Subject: Re: [xsl] The time to execute a function is 625 times greater than the sum of the times to execute the statements in the function
From: "Graydon graydon@xxxxxxxxx" <xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx>
Date: Sat, 1 Aug 2020 19:57:38 -0000
On Sat, Aug 01, 2020 at 07:03:35PM -0000, Roger L Costello
costello@xxxxxxxxx scripsit:
> To see how much time each statement in f:train takes, Michael
> M|ller-Hillebrand suggested that I put each statement into its own
> function. Great idea! I did so. I ran the Saxon profile tool on the
> modified f:train and you can see below the gross total time required
> for each statement (each statement is now in its own function). I
> summed their times and it came to a total of 8.742 ms. So, the f:train
> function should take 8.742 ms but instead it takes 5,460.326 ms. How
> can it possibly be that the time to execute the f:train function is
> 625 times greater than the sum of the times to execute the statements
> in the function? Any suggestions?

Any starting-with-X language is going waltzing with the optimizer.

(Possibly wrong today but believed true at one time example; the BaseX optimizer responds well to variable definitions, because that's a hint about using indexes for this value you've just labelled important; Saxon would rather you didn't use variables for everything, because it's going to have more constraints on the re-writing step if you do.)

I'd do three things:

1. find the controlling -Xmx option and set it as high as the hardware will support; once the Java heap starts swapping performance dies in a pit (sometimes in former days this was -Xms being too low, too.)  It is ridiculously easy when chaining modes to burn through gigabytes of RAM as multiple copies of the parsed source content have to coexist.

If at all possible, set up some sort of perfmeter so you can see the heap size in real time.

2. make absolutely sure everything is explicitly typed, and as restrictively typed as possible. Sometimes the optimizer doesn't realize it could be doing this the easy way.  Give as many hints as you can.

3. make a copy of the program, and _comment out_ all your individual
functions in the main function -- f:train -- but the first one.  (and prepare the program as a whole to complete after f:train runs once, of course.)  Keep adding individual functions back in until you find the miscreant.

(One possibility is that f:train will run fast once; it's multiple calls
that's the problem.  If running it once is fast the whole way
through, you can at least start to look at what's going on with the
calling context.)

-- 
Graydon Saunders  | graydonish@xxxxxxxxx
^fs oferiode, pisses swa mfg.
-- Deor  ("That passed, so may this.")

Current Thread