[xsl] Performance results of an XML-based neural network versus a map-based neural network

["Dr. Roger L Costello costello@xxxxxxxxx" <xsl-list-service@xxxxxxxxxxxxxxxxxxxxxx>]

Hi Folks,

I implemented a neural network. My first implementation was XML-based. Running
the neural network revolves around processing this XML document:

<?xml version="1.0" encoding="UTF-8"?>
<neuralNetwork>
    <!-- Set number of nodes in each input, hidden, output layer -->
    <inodes>784</inodes>
    <hnodes>100</hnodes>
    <onodes>10</onodes>
    <!-- Matrix containing the weights for the input to hidden layer (wih)
-->
    <Matrix id="wih">
        <row>
            <col>-0.015882097402764903</col>
            <col>0.04906187053472448</col>
            <col>-0.025639452565869168</col>
            ...
        </row>
        ...
    </Matrix>
    <!-- Matrix containing the weights for the hidden to output layer (who)
-->
    <Matrix id="who">
        <row>
            <col>-0.029846534548482826</col>
            <col>0.09713823372280408</col>
            <col>-0.07405568240941922</col>
            ...
        </row>
        ...
    </Matrix>
    <!-- Learning rate -->
    <lr>0.3</lr>
</neuralNetwork>

There are lots of matrix operations involved in running a neural network --
lots of matrix addition, transpose, dot product, etc., operations. The
operations update the <Matrix> elements in <neuralNetwork>.

I used the SAXON profile tool to see the performance of my implementation. The
performance of the matrix operations was very slow. Here's the performance of
two of the matrix operations:

matrix:addition
average time (net/ms) = 273.170
total time (net/ms) = 27,043.813

matrix:dot-product
average time (net/ms) = 257.718
total time (net/ms) = 25,514.069

[Michael Kay: what does "net/ms" mean?]

In my second implementation I converted the first implementation to be
map-based. I replaced the above XML with this map:

<xsl:map>
    <!-- Set number of nodes in each input, hidden, output layer -->
    <xsl:map-entry key="'inodes'" select="784"/>
    <xsl:map-entry key="'hnodes'" select="100"/>
    <xsl:map-entry key="'onodes'" select="10"/>
    <!-- Learning rate -->
    <xsl:map-entry key="'lr'" select="0.3"/>
    <!-- weights between the input layer and the hidden layer (wih) -->
    <xsl:map-entry key="'wih'" select="(-0.015882097402764903,
0.04906187053472448, -0.025639452565869168, ...)"/>
    <!-- weights between the hidden layer and the output layer (who) -->
    <xsl:map-entry key="'who'" select="(-0.029846534548482826,
0.09713823372280408, -0.07405568240941922, ...) "/>
</xsl:map>

I again used the SAXON profile tool to see the performance. The performance of
the matrix operations for this implementation was astoundingly fast. Here's
the performance of two of the matrix operations:

matrix:addition
average time (net/ms) = 0.003
total time (net/ms) = 0.254

matrix:dot-product
average time (net/ms) = 0.001
total time (net/ms) = 0.131

For all the matrix operations the map-based version was millions of times
faster than the XML-based version.

Surprisingly, however, the overall time to train the XML-based neural network
was faster than the time to train the map-based neural network:

neural-network:train

XML-based:
average time (net/ms) = 1711.572
total time (net/ms) = 169,445.644

map-based:
average time (net/ms) = 3633.811
total time (net/ms) = 359,747.295

I don't understand how this could possibly happen.

In the map-based version of the train function, I have these statements to
update the weights of the input layer to hidden layer (wih) and the weights of
the hidden layer to output layer (who):

<xsl:variable name="neural-network-with-new-wih"
  	select="map:put($neural-network, 'wih', $updated-wih)" />
<xsl:variable name="neural-network-with-new-wih-and-new-who"
	select="map:put($neural-network-with-new-wih, 'who', $updated-who)" />
<xsl:sequence select="$neural-network-with-new-wih-and-new-who" />

Notice that those statements create two new maps. Every time I call the train
function, two new maps are created. The train function is called many times.
Could that be the cause of why my map-based implementation is running slower
than the XML-based implementation? Is there a way to _replace_ a value in a
map, rather than _creating_ a whole new map?

/Roger