RE: [xsl] metrics for evaluating xsl-t?

["Michael Kay" <mike@xxxxxxxxxxxx>]

I think it all depends what you are going to use the metrics for.

The only two metrics I maintain for the Saxon source code are total lines of
source, and non-comment lines of source. I use them to track how much growth
there is between releases and to check that I'm not getting lazy about
adding comments. 

I get particularly sceptical about metrics if they are associated with
targets or goals. It reminds me of a project manager who measured people on
the number of lines of code they wrote each week, and couldn't cope with one
programmer who prided himself on reporting negative numbers.

So it's not so much a question of what you want to measure, as why.

Michael Kay
http://www.saxonica.com/



> -----Original Message-----
> From: Tony Graham [mailto:tkg@xxxxxxxxxxxx] 
> Sent: 23 August 2006 12:55
> To: xsl-list@xxxxxxxxxxxxxxxxxxxxxx
> Subject: Re: [xsl] metrics for evaluating xsl-t?
> 
> "bryan rasmussen" <rasmussen.bryan@xxxxxxxxx> writes:
> > Wondering if anyone has given any thought to a metrics for 
> evaluating 
> > the complexity of a stylesheet based solution? can be XSL-T 
> 2.0 only.
> 
> I have been thinking of it, but the same is probably true for 
> many people.
> 
> When I searched on it last week, I found this thread from 
> last year started by Lars Marius Garshol:
> 
> http://www.biglist.com/lists/xsl-list/archives/200504/msg00805.html
> 
> Dimitre Novatchev provided this link to a definition of 
> Cyclomatic Complexity as part of that thread:
> 
> http://www.sei.cmu.edu/str/descriptions/cyclomatic_body.html
> 
> I personally have got a far as thinking of writing a 
> stylesheet to run on stylesheets to measure complexity, much 
> as you seem to be thinking, though I'd write it in XSLT 1.0 
> so it's usable by more people.
> 
> If such a stylesheet were to be written and publicly 
> available, then we could do a survey to see if there's any 
> trends in the complexity of people's stylesheets.
> 
> Since I have only vague ideas on what to measure to determine 
> complexity, I would have the complexity stylesheet output XML 
> representing the counts for the different factors being 
> measured in case the weighting for the various factors ever 
> needs to change.  It would also make it easier to write one 
> complexity stylesheet that would be used to compute multiple 
> complexity metrics, just as the cyclomatic complexity page 
> refers to multiple, complementary metrics.
> 
> The factors from the original post from last year are:
> 
>  > - size of stylesheet in bytes, lines, and XML 
> elements+attributes,  >  > - number of elements from the XSLT 
> namespace, and  >  > - combined length (in characters) of 
> XPath expressions.
> 
> I'm still thinking this through, and I would need to look 
> more at some of the metrics for conventional software, but I 
> would think to
> include:
> 
>  - Number of imported and included stylesheets
> 
>  - Number of top-level xsl:param and xsl:variable
> 
>    I'm tempted to claim that a top-level xsl:param or xsl:variable in
>    an included or imported stylesheet file adds more to the complexity
>    than does one in the base stylesheet file.
> 
>  - Number of modes
> 
>  - Number of named templates
> 
>  - Levels of nesting of xsl:if and xsl:choose
> 
>    This is a possible measure of the complexity of a template, which
>    isn't immediately useful for coming up with one magic number to
>    indicate the complexity of a stylesheet.
> 
>  - The number of namespaces in use, and the number of prefixes in
>    "exclude-namespace-prefixes"
> 
>  - Number of xsl:apply-imports
> 
>  - Number of xsl:namespace-alias
> 
>  - Number of xsl:fallback
> 
> But, having just looked in "Code Complete" by Steve McConnell 
> while writing this, I am reminded:
> 
>    Don't start by collecting data on all possible metrics -- you'll
>    bury yourself in data so complex that you won't be able to figure
>    out what any of it means... refine them and add to them as your
>    understanding of what you want to measure improves.
> 
> Regards,
> 
> 
> Tony.