[xsl] Re: Tokenize question: tokenize on words, spaces and punctuation

Subject: [xsl] Re: Tokenize question: tokenize on words, spaces and punctuation
From: Martin Holmes <mholmes@xxxxxxx>
Date: Wed, 16 Mar 2011 21:24:02 -0700
Hi there,

I think you might have misunderstood; your output is not what I need, because it throws away the punctuation.

Cheers,
Martin

On 11-03-16 07:28 PM, Suresh wrote:
In xslt 2.0:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform";
     xmlns:xs="http://www.w3.org/2001/XMLSchema";
exclude-result-prefixes="xs" version="2.0">
     <xsl:output method="text"/>
     <xsl:template match="/string">
         ('<xsl:value-of select="tokenize(.,'[\s]+')" separator="', ' ', '"/>')
         </xsl:template>
</xsl:stylesheet>


Output:


('Oh', ' ', 'what', ' ', 'a', ' ', 'great', ' ', 'day!')


Thanks.


On Wed, Mar 16, 2011 at 8:33 PM, Martin Holmes<mholmes@xxxxxxx> wrote:

Hi there,


This is really a question for XPath regex gurus:

I need to tokenize a string of text such that words, punctuation and spaces are split. So from this:

Oh, what a great day!

I need to get:

('Oh', ',', ' ', 'what', ' ', 'a', ' ', 'great', ' ', 'day', '!')

I've been hacking away at this for a while, but regexps aren't my strong suit. Can anyone help?

Cheers,
Martin




-- Suresh

Current Thread