OASIS Interoperability/Conformance TC
IIC Conference Call Minutes: July 29, 2002
Attendees:
Michael Kass, NIST
Eric Van Lydegraf, Kinzan
Kamal Patel, ITNET
Prakash Sinha, IONA (attended only informal meetings)
Monica Martin, Drake Certivo
Steve Yung, Sun Microsystems
Minutes Taker: Mike Kass
1. Vote on ebXML MS Master List of Conformance Test Requirements - Issues/Questions/Comments?
This Conformance Test Requirements list is to be submitted to the ebXML MS TC for review, prior to beginning a Conformance Testing campaign for ebXML MS. In addition, a companion "Annotated Messaging Services Specification" document will be submitted, highlighting areas of the specification that are covered by the conformance test requirements.
Discussion of issues during the conference call regarding the Conformance Test Requirements included the following:
1) Clarification that the Conformance Test Requirements themselves are not an indicator of whether the testing harness can actually test this requirement. They are only a list of test requirements derived from the specification. The Annotated MS Specification provides a graduated scale of specification coverage provided by both the completeness of the test requirements, and the capability of the test harness. Below are the criteria for coverage designations of "full", "partial" and "none".
FULL: The test requirement(s) that address this specification item, are a good indicator of conformance to the specification item, i.e. if an MSH passes a test case that implements properly this test requirement(s), there is very strong indication that it will behave similarly in all situations identified by the spec item.
PARTIAL: The test requirement(s) that address this specification item, are only a partial indicator of conformance to the specification item, i.e. if an MSH passes a test case that implements properly this test requirement(s), this indicates that it will behave similarly in only a subset of all situations identified by the spec item. Possible reason may be: the pre-condition(s) of the test requirement(s) only identify on purpose a subset of these situations, that can be reasonably tested.- the occurrence of situations that match the pre-condition of a Test Requirement is under control of the MSH (e.g. implementation-dependent) and out of the control of the testbed. I.e. the test will be verified only in case the situation occurs during testing, which we cannot control.
NONE: The specification item cannot be tested even partially, at least with the Test Framework on which this Conformance test suite is to be implemented, and under the test conditions that are assumed.
2) Monica Martin had questions regarding four particular conformance test requirements. They are:
urn:semreq:id:69 For each received message, if the message in error has an ErrorList element with highestSeverity set to Error ---- The error is: Logged; Resolved by other means; and, No further action is taken. ** ISSUE: Will test harness have access to an MSH log? **
Discussion from implementers was that such a feature would have to be added to the test adapter/plugin to permit this kind of test service capability. The "Annotated MS Specification" currently shows this requirement as "fully" covering the specification item. If implementations of the Test Service adapter will not provide access to the log file, the spec annotation should be downgraded to "partial".
urn:semreq:id:123 For each reliably received message, if the PersistDuration parameter is present in the CPPA AND AckRequested element is present in the message AND the message is presented once and only once to the application AND the same message is received again by the MSH before PersistDuration expires -- An Acknowledgement message is sent back to the sending MSH. **ISSUE: Can we verify a "once and only once" presentation to the application?
The "Annotated MS Specification" currently shows this requirement as "fully" covering the specification item. If implementations of the Test Service will not provide verification of a "once and only once presentation to the application", then this spec annotation for this requirement should be downgraded to "partial". Eric suggested that there may be ways that the Test Service could be designed to support checking "once and only once" presentation to the application.
urn:semreq:id:190 For each received message, if the message contains a StatusRequest element AND the the StatusRequest child RefToMessageId element value is recognized AND the message is received from an party deemed to be unauthorized - The Timestamp child element of the StatusResponse element is not present in the response message. **ISSUE: How do we check if the party is authorized?
The "Annotated MS Specification" currently shows this requirement as "fully" covering the specification item. If implementations of the Test Service will not provide verification that the party is authorized, then this spec annotation should be downgraded to "partial".
Prakash stated that authorization is better addressed as an interop feature, and that this test may belong in the interop domain.
urn:semreq:id:91 For each received message with an AckRequested with the Signed attribute set to False and consistent with the CPPA, The Acknowledgment message is unsigned. **ISSUE: How is verification to be done between CPA and messages received?
Monica raised the question of how the Test Service will compare content of received messages with CPA values. Will we be using a mini-cpa, a full CPA document for configuration of the Test Service? Prakash stated that standardization of a "mini-cpa" is not imminent, and that interoperability participants are going to be using a full CPA for configuration. Even so, there are differences in CPA versions, and some kind of an agreed upon CPA schema will have to be utilized.
** For those who have not examined the Conformance Test Requirements and Annotated MS Specification, please look at them and send your comments to the IIC Conformance mailing list prior to the next conference call.
2) Vote on Conformance Levels ( as proposed in November of 2001 ).
In addition to a final vote on conformance test requirements on August 12, Jacques would like a final vote on conformance levels that best suit interoperability and conformance testing. We revisited discussion of 3 possible conformance level groupings. They are:
Most previous votes: "2 levels"
LEVEL I:
Packaging & SOAP envelope extensions, core extension elements, HTTP or SMTP, Security, Reliable Messaging, MessageOrder, All Error Handling relative to above. Be "STRONGLY" conformant ( handle RECOMMENDED and SHOULD ) semantically. Process OPTIONAL features fully or gracefully if not implemented. INTEROP ISSUES: Parties must use either HTTP or SMTP, if other XMLDSIG method is used, both must support it.
LEVEL II:
Conforms to Level I
Implements MessageStatus, Ping, Pong and Multi-Hop
Implements Error Handling relevant to above
Strongly conforming for above
Second choice: "3 levels"
LEVEL I:
Packaging & SOAP envelope extensions, core extension elements, HTTP or SMTPAll Error Handling relative to above. Be "STRONGLY" conformant ( handle RECOMMENDED and SHOULD ) semantically. Process OPTIONAL features fully or gracefully if not implemented. INTEROP ISSUES: Parties must use either HTTP or SMTP, if other XMLDSIG method is used, both must support it.
LEVEL II:
Conforms to Level I
Implements , Security, Reliable Messaging, MessageOrderImplements Error Handling relevant to above
Strongly conforming for above
LEVEL III:
Conforms to levels 1 and 2
Implements MessageStatus, Ping, Pong and Multi-Hop
Third choice: "1 level"
LEVEL I:
Implements all normative material in the spec, in a STRONGLY conformant way.
The full details for options 1, 2 and 3 are available at: http://lists.oasis-open.org/archives/ebxml-iic-conform/200207/msg00040.html Please read them and be prepared to vote for the choice that you feel best suites conformance and interoperability testing.
3) Comments/Issues on ebXML Conformance / Interop Test Suite Documents - There was no discussion of either the ebXML Conformance or Interop Test Suite documents. However, IIC members are encouraged to post their comments for these documents to the appropriate IIC lists.
4) Other Issues:
Next Conference Call: August 12
Agenda: Preliminary Agenda:
Vote on ebXML MS Conformance Test Requirements and Conformance Levels. Please send any items you wish to add to the agenda to Michael.kass@nist.gov