Jade/DSSSL future

Subject: Jade/DSSSL future
From: James Clark <jjc@xxxxxxxxxx>
Date: Mon, 24 May 1999 14:41:09 +0700
Adam Di Carlo wrote:

> Honestly, I haven't seen
> any sort of comprehensive analysis of what is wrong with Jade at this
> time, and what priorities should be.  AFAIK, I have seen the following
> pointed out, and this is how I would prioritize it:

My priorities would be a bit different.

>   * some misc DSSSL features and operators missing (there was a patch
>     submitted on this list which looked pretty good to me)

A lot of these can be trivially implemented with the existing operators,
and there's little performance benefit in implementing them in C++, in
fact there may be a loss because of code bloat. A better solution for
such operators would be an ability to automatically preload a startup
file with function definitions (you have to be careful to ensure that
these preloaded definitions work even when other primitives get
redefined).

I agree that it would be nice to have the operators that cannot be
defined in terms of the others, but I don't think this will make a big
difference to the overall utility of Jade.

>   * the proper DSSSL transformation language is missing (I'm not sure
>     if this is worth implementing -- it would seem that DTD-to-DTD
>     transformations are possibly better handled with XSLT, although
>     for SGML users it would need to be accompanied by some SGML->XML
>     conversions as well)

I think this would be almost completely useless.  The jade -t sgml
approach is better, and XSLT is better yet.
 
>   * some major DSSSL advanced features are not implemented, the
>     biggest one brought up to this list being the
>     complex-page-sequence stuff...

There's not much point in implementing full page-sequence flow object in
the front-end unless it can be supported in the back-ends.  None of the
existing backends could support the complex-page-sequence.  What's
really unique about Jade is the ability to get multiple formats from a
single stylesheet.

>   * TeX backend may need a re-write (see below)

Yes.  It would be better to do more work in the backend and less work in
macros.

>   * jade is slow (is this even fixable?)

I don't think it's slow considering what it's doing.

It has a full programming language and there's no way you can stop
people writing slow programs in any language. You need to look at the
combination of jade plus the stylesheet you're interested in (I guess
it's DocBook) and figure out how the combination can be faster.  Part of
the problem is that there are no profiling tools in Jade: there's no way
for a stylesheet author to figure out why their stylesheet was slow.  So
some way of profiling Jade would be great. It may just be a matter of
optimizing the stylesheet, or the solution may be to add some additional
primitives to Jade to handle the things that are slow.  Another thing
that could help is some documentation explaining what things are slow
and what things are fast.

SGML also brings a lot of overhead to Jade.   A DSSSL stylesheet has to
be parsed as SGML, the DTD has to be parsed, all the content checking
has to be done, then the meta-DTD also has to be read and that has to be
parsed, and the architectural document has to be created; only then does
the real DSSSL parsing start.  A version of Jade that used an XML parser
rather than SP and in which DSSSL files didn't have an SGML
architectural wrapper would be a lot faster.

>   * NROFF backend would be nice (note that James himself wrote GROFF --
>      I wonder what James thinks about this. Personally, I think its
>      kinda low priority, since HTML -> ASCII with lynx works decently)

I think a decent HTML+CSS backend would be more useful. The current one
was really just an attempt at a proof of concept;  I did only enough to
discover that it wasn't going to be practical with CSS implementations
as they then were. But CSS implementations are at last starting to get
to the point where they're useable.  In particular Mozilla's is
excellent.  It should be possible to do an HTML+CSS backend which would
get you browseable output from the same stylesheet that you use for the
RTF or TeX backend.  You wouldn't have to write a separate HTML-specific
stylesheet.

Apart from the things I've mentioned, my priorities would be:

- Making the backends useable independently of the rest of Jade.  Create
a small program (using an XML parser like expat rather than SP) that
reads the FOT file and calls an FOT backend.  This would require some
tweaks to the FOT format and to the FOT interface.  The point of this is
that it would then be possible to use XSLT with DSSSL flow objects.

- Make it easier to use Jade for XML.  At the moment the fact that the
DSSSL stylesheet is an SGML document makes it awkward to use Jade with
XML.  There are several possible solutions here and I'm not sure what
the best is.

- Support XSLT in Jade. XSLT has an xsl:functions element that allows
XSLT stylesheets to include extensions written in arbitrary extension
languages. With Jade, that extension language would be the DSSSL
expression/query language.  This is probably too much work.

My general feeling is that just as the future is XML not SGML, so the
future is also XSL not DSSSL.  When XSLT and XSL are done, there will (I
hope) be nothing you can do in DSSSL that you can't do with XSL(T). 
DSSSL has not achieved widespread acceptance, and of course that's
disappointing to all of us involved in DSSSL.  But I think we have to
face up to the fact that the main reason it has failed to achieve
acceptance is not because of lack of promotion or explanation or
marketing (although that has probably been a factor), but because of
real useability problems in the language itself.  XSL started off as a
collaboration between DSSSLers and Microsoft to create a new syntax for
the DSSSL style language that would be easier to use and could achieve
wide acceptance.  It hasn't quite turned out like that.  On the XSL flow
objects side, politics and market realities have necessitated
building DSSSL functionality on top of CSS formatting objects/properties
rather than starting with the DSSSL flow objects.  On the XSLT side, as
we've continued to work on the language we've found many ways to improve
it, and the language has evolved substantially from DSSSL; however it's
still very much the same approach to transformation as jade -tsgml. 
There are big advantages to being in the mainstream, and XSLT looks set
to become a mainstream technology.  It's attracted the support of some
of the biggest players out there -- Microsoft, Oracle, IBM, Sun, Lotus,
Adobe (look at the list of WG members in Acknowledgements section of the
XSLT WD) -- and they have collectively invested millions of dollars
helping to develop it and the language has benefited enormously from
this.  I have sometimes wondered whether I wasted five years of my life
doing DSSSL and Jade, but I comfort myself with the astonishing fact
that a little bit of their spirit now seems likely to live on in, of all
places, Microsoft Windows.

So what does this mean for my involvement in Jade?  I think the reality
is that although there are improvements that I would dearly like to
make, it's unlikely, at least in the near future, that they are going to
get to the top of priority list, and I don't want to stand in the way of
others who wish to make improvements.

I do have one request: if you release something independently of me, you
call it something other than simply Jade, but preferably a name that
acknowledges Jade; maybe OpenJade or something like that.

I also have one strong recommendation on how to proceed with Jade
development.  Before you start hacking the code, create some
documentation on Jade internals.  Jade was not designed for a bazaar
development model and there is no internals/design documentation.  But
there is an internal architecture that was designed with a lot of care,
and unless those hacking on Jade really understand it, reliability and
quality is likely to suffer. For example, when writing extension
functions there are rules you have to follow to make the garbage
collector work; if you don't follow them everything will compile and
appear to work at first but you will get seemingly random,
unreproducible crashes on large stylesheets and documents; fixing this
kind of bug has taken me literally days in the past. To hack Jade, you
really need to know what you're doing.  I don't have the time to write
the internals docs myself, however I am willing to take the time to
explain it to somebody else if there is somebody who (a) has read the
code and has a good level of understanding already and (b) promises to
write up an internals document.

I am also willing to make all my RCS files available, which I would
suggest be used to start a public CVS repository.

James
(speaking for myself alone not the DSSSL WG and not the XSL WG)


 DSSSList info and archive:  http://www.mulberrytech.com/dsssl/dssslist


Current Thread