From: Kay Michael <Michael.Kay@icl.com>

To: "'xsltconf@CraneSoftwrights.com'" <xsltconf@CraneSoftwrights.com>

Subject: RE: OASIS XSLT Conformance Subcommittee

Date: Tue, 1 Feb 2000 10:27:32 -0000

MIME-Version: 1.0

X-Mailer: Internet Mail Service (5.5.2650.21)

Content-Type: text/plain;

	charset="iso-8859-1"



Very pleased to see this being set up. I can't take part in any formal way,

because I would need to justify the time and expense, and I'm in the wrong

continent, but I'm happy to contribute informally and electronically, and to

help with debugging the test suite.



I don't have any test material that's in good enough shape to contribute

immediately, but perhaps this activity will prompt me to get what I do have

into a more presentable state. I'd strongly advise NOT taking a random

collection of existing stylesheets, some of which might take a long time to

execute but exercise only a very small part of the language, but rather to

start with the LotusXSL test suite which is lots of small atomic tests, and

build on that to increase its coverage. If you take big existing

stylesheets, it will be very hard to know how good the overall test coverage

is; and it will probably be rather poor, because 80% of programs only use

20% of the language, and it tends to be the same 20%.



There's a significant problem with the weakness of the conformance criteria

in the XSLT recommendation. It says only that a processor has to generate a

result tree; but it doesn't provide any way of verifying that it has done

so. I would suggest that any processor submitted for conformance testing is

required to provide software that outputs its result tree as Canonical XML.

(Although Canonical XML as currently defined is not quite right, e.g. it

excludes comments).



Related to this is the status of xsl:output; I'd suggest that although the

xsl:output statement is optional, you provide a separate conformance test

for that. A processor that doesn't support xsl:output is not going to

provide the compatibility that users need.



I've got a test harness which I'm happy to make available or improve, it's

derived from the LotusXSL test harness but extends it to compare the results

of each test with reference results, converting both to Canonical XML (using

the original James Clark definition of same) for comparison. (This doesn't

work, of course, where the output method is HTML or text).



An area that is almost totally absent from the LotusXSL test suite is error

handling, though the standard has a lot to say about conformance in this

area. One of the conformance rules that's going to be a bit tricky to test

automatically is where the implementor has a choice, e.g. if there are two

template rules that match a node you have a choice of reporting an error or

using the last one; what you aren't allowed to do is to use the first. (I

can easily imagine the discussion that led to that silly compromise - when a

standards committee can't decide what's right, it hopes that the implementor

can.)



There are a few other areas where implementors have a choice and this makes

automatic comparison of results difficult: these need to be isolated into a

few small parts of the test suite. Areas where I had trouble comparing my

results with Lotus included generate-id(), indent="yes", collating sequences

for xsl:sort. Plus a few where the spec says nothing, e.g. does "-0" mean

negative zero or positive zero?



Regards,



Mike Kay