OASIS Open

Testing and Monitoring Internet Exchanges (TaMIE) TC

Electronic collaborations over the Internet between business partners (e-Business / e-Government) appear to be converging toward well-established types of message exchange patterns that involve both user-defined standards and infrastructure standards. At the same time, the notion of event is increasingly promoted for asynchronous communication and coordination in Event-Driven Architectures (EDA) that are considered as either complementary to or part of SOA systems. In both cases collaboration between partners or between components is achieved by the means of choreographed exchanges of discrete units of data - messages or events - over an Internet-based protocol. The TC will define an event-centric test case scripting markup and execution model for such systems.

In e-Business transactions as in EDAs, partners or components must agree on the use of a combination of standards in order to interoperate with each other. Typically, these standards can be classified into three layers:

• Messaging infrastructure standards, ranging from transport level to higher-level messaging protocols and quality of service (QoS) including reliability and security, such as those defined as SOAP extensions, or REST (Representational State Transfer).
• Multi-message exchange standards as manifested in business processes and choreographies. Included standards may conform to UMM business transaction patterns, ebXML Business Process Specification Schema (BPSS or ebBP), WS-Choreography or WS-BPEL.
• Business document standards. These may be business content structure and semantics, taxonomies in use, code lists, semantic rules, or the XML schema modeling style. They are industry-specific (e.g. RosettaNet PIP schemas, AIAG Inventory Visibility and Interoperability schemas), horizontal document standards (e.g. based on UN/CEFACT Core Components, OAGIS Business Object Documents schemas), or regional guidelines (e.g., KIEC's e-document modeling guideline).

There have been conformance and interoperability test suites (e.g, ebXML messaging V2 test suites, WS-I test assertions) and testing tools (e.g., ebXML IIC, NIST QOD, NIST Business Document Content testbed, KorBIT ebMS testbed, WS-I test tools) for each above layer individually. But the testing of integrations of standards has been ad-hoc (e.g. RosettaNet Self-Test Kit is tied to RosettaNet protocol), or limited mostly to standards in the messaging infrastructure (WS-I).

Although the need for testing some form of integration of standards has been well recognized for infrastructure standards (e.g. WS-I profiles), there has been little support for testing integrations that extend to the use of standards specific to a business - e.g. for documents or choreographies. Such integrations can be construed as user-defined profiles. For example, the level of QoS required for a business transaction may depend on the nature of business data being exchanged, or on some property defined by the related business process.

Testing and monitoring these infrastructure layers and their integration also requires that test cases access a combination of contracts - agreements, policies or business transaction patterns - represented by meta-level documents (WS-Policy, WSDL, ebXML CPPA, WS-BPEL abstract processes, ebBP definitions, XML schemas).

This compliance objective goes beyond quality assurance for the messaging function: it requires the monitoring of live transactions in production environments, as well as verifying conformance of business endpoints in operation conditions. This calls for a flexible test execution model that can accommodate performance constraints as well as different timeliness constraints - e.g. where tests are either deferred over log data, or executed on live exchanges in a monitoring mode.

Consequently, the execution model of such test cases or monitoring cases, must accommodate dynamic conditions requiring real-time or near real-time error detection or measurement, allowing to correct and report on business exchanges as they proceed.

The output of a monitoring script also must provide more information than a report of the type pass / fail. Different ways of "passing" or "failing" must be reported on, as well as identifying the types of business transactions. The output must be easy to format and feed to another decision engine, such as a rule engine that will process this input in real-time. For example, a rule may decide to generate an alert if a business transaction lasts too long, depending on the nature of the transaction and on the SLA associated to these business partners.

This reporting must be easy to consolidate, for Service Level Agreements (SLA) and Business Activity Monitoring (BAM) purposes. The reporting output may itself be subject to dynamic testing and monitoring. For example, a rule may decide to generate an alert if the current average response time for requests of a specific type, is beyond a threshold.

The TC will define a testing and monitoring model, as well as a test script markup, so that test cases or monitoring cases can be fully automated, and portable across test environments.

The scope of activity for this TC must be within the following topics:

• Use cases and Requirements: Investigation of use cases that may combine standards - or specifications submitted to an SDO - in all areas concerned by message exchanges: protocols, QoS, choreography, documents and business content. Selection and prioritization of related requirements.
• Event Management model: A model for representing, searching, correlating and managing events intended for the purpose described in this charter. In particular, an event management approach such as the one described in the event-driven Test Scripting Language (see Input material) will be considered. This event management must allow for the same test case to execute either in static mode, over a set of events all of them are already logged, or in dynamic mode, over a set of events not all of them have already occurred when the execution starts. The event model must be open for reuse by at least some other scripting and processing engines, such as Event-Condition-Action (ECA) rule-based systems.
• Test case scripting and execution model: An XML markup for representing executable test cases will be defined. The execution model will rely on the event management model. The test case scripting may reuse or profile subsets of existing dialects that are already in use for domains that have similar functional requirements (if their IP status is compatible with the TC IPR mode), in which case the markup will act as a coordination or integration language, treating these dialects either as language extensions or as external resources used via adapters. In particular, openness to the reuse of XML processing dialects and tools based on recognized standards - such as XPath, XSLT, XQuery - is a requirement. The scripting and execution model will attempt to support extensions for other scripts, in particular should allow for bindings to rule representations such as SBVR (OMG) or RIF (W3C). The design approach for the core scripting will favor (a) simplicity of use - a small number of constructs that are easy to learn and to implement as opposed to feature-rich dialects; and (b) modular units of scripts that allows for functional reuse and extensibility. Test scripts will process run time materials (i.e., messages, events, signals), as well as configurative artifacts such as policies and agreements, schemas and interface definitions. Test cases or monitoring cases will also provide for more detailed outputs than pass/fail, and will strive to support use cases defined in the requirements for SLA monitoring, BAM (Business Activity Monitoring) and Event-Driven Architectures (EDA). However, the objective will not be to specify a BAM system or an SLA enforcement system, but only the monitoring technology that can support these.
• Evaluation and Investigation: of third-party markups and dialects, log formats and existing test practices, for possible reuse or interfacing in the test case scripting language. However no third-party specification or tool can be made essential or necessary to an implementation of the specification to be delivered, unless its licensing is free and unrestricted.
• Methodology and Guidelines: In scope of this activity, are all forms of support for users, such as example scripts, best practices, primers and guideline documents, concerning all topics listed above.

The following documents are input material to this TC, which will deserve prime attention from the TC assuming their IP status is compatible with the IPR mode of the TC:

• The event-driven Test Scripting Language (eTSL) V0.85
• Conformance Testing and Certification Framework (OASIS, Conformance TC, June 2001)
• QA Framework: Specification Guidelines (W3C, November 2004)
• Test Assertion Documents for WS-I profiles (WS-I, 2003-2005)
• OASIS IIC Test Framework 1.1

Other documents may be considered by the TC, subject to a TC decision.

Explicitly Out-of-scope:

• Defining or specifying detailed test case metadata or artifacts that would complement the above scripting - e.g. such as supported in ATML, for representing test environments, complete test suites and/or their result.
• Definition of particular test harnesses.
• Use cases and requirements that refer to infrastructure specifications that are not submitted to an SDO, or that refer to business documents or content that are not approved by an organization or consortium representative of this business domain.

The TC will produce the following deliverables:

1. A requirements document, which may include use cases for Internet exchanges, test assertions for related standards, references to existing test case dialects or existing logging formats or systems.
2. A specification defining an event metadata and an event log management model that supports both the testing and monitoring described in this charter, in a way that does not preclude other test case scripting techniques than the one specified in deliverable (c).
3. A specification defining an event-centric test case scripting and execution model that supports both the testing and monitoring described in this charter, and builds upon the deliverable (b).
4. A set of examples and use cases illustrating the use of above specifications for testing or monitoring, that involve business content standards, some choreography standards, and some specifications in the domain of SOAP messaging or others.
5. An implementation of the above specifications (b) and (c) used for proof of the proposed concept and principle.

Timeline:

The TC will aim at a Public Review draft of both deliverable (b) and (c) above by first half 2009.

Royalty-Free on Limited Terms.

The TC welcomes any OASIS member with an interest and/or experience in testing and monitoring.

English.