[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: OASIS Call for Participation: Testing and Monitoring Internet Exchanges (TaMIE) Technical Committee
To: OASIS members & interested parties
A new OASIS technical committee is being formed. The OASIS Testing and
Monitoring Internet Exchanges (TaMIE) Technical Committee has been proposed by
the members of OASIS listed below. The proposal, below, meets the requirements
of the OASIS TC Process [a]. The TC name, statement of purpose, scope, list of
deliverables, audience, and language specified in the proposal will constitute
the TC's official charter. Submissions of technology for consideration by the
TC, and the beginning of technical discussions, may occur no sooner than the
TC's first meeting.
This TC will operate under our 2005 IPR Policy [b]. The eligibility
requirements for becoming a participant in the TC at the first meeting (see
details below) are that:
(a) you must be an employee of an OASIS member organization or an individual
member of OASIS;
(b) the OASIS member must sign the OASIS membership agreement [c];
(c) you must notify the TC chair of your intent to participate at least 15
days prior to the first meeting, which members may do by using the "Join this
TC" button on the TC's public page at [d]; and
(d) you must attend the first meeting of the TC, at the time and date fixed
below.
Of course, participants also may join the TC at a later time. OASIS and the TC
welcomes all interested parties.
Non-OASIS members who wish to participate may contact us about joining OASIS
[c]. In addition, the public may access the information resources maintained for
each TC: a mail list archive, document repository and public comments facility,
which will be linked from the TC's public home page at [d].
Please feel free to forward this announcement to any other appropriate lists.
OASIS is an open standards organization; we encourage your feedback.
Regards,
Mary
---------------------------------------------------
Mary P McRae
Manager of TC Administration, OASIS
email: mary.mcrae@oasis-open.org
web: www.oasis-open.org
[a] http://www.oasis-open.org/committees/process.php
[b] http://www.oasis-open.org/who/intellectualproperty.php
[c] See http://www.oasis-open.org/join/
[d] http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tamie
CALL FOR PARTICIPATION
OASIS TESTING AND MONITORING INTERNET EXCHANGES (TaMIE) TC
1a. Name and abbreviation
Testing and Monitoring Internet Exchanges (TaMIE) TC
1b. Purpose
Electronic collaborations over the Internet between business partners
(e-Business / e-Government) appear to be converging toward well-established
types of message exchange patterns that involve both user-defined standards and
infrastructure standards. At the same time, the notion of event is increasingly
promoted for asynchronous communication and coordination in Event-Driven
Architectures (EDA) that are considered as either complementary to or part of
SOA systems. In both cases collaboration between partners or between components
is achieved by the means of choreographed exchanges of discrete units of data -
messages or events - over an Internet-based protocol. The TC will define an
event-centric test case scripting markup and execution model for such systems.
In e-Business transactions as in EDAs, partners or components must agree on the
use of a combination of standards in order to interoperate with each other.
Typically, these standards can be classified into three layers.
. Messaging infrastructure standards, ranging from transport level to
higher-level messaging protocols and quality of service (QoS) including
reliability and security, such as those defined as SOAP extensions, or REST
(Representational State Transfer).
. Multi-message exchange standards as manifested in business processes and
choreographies. Included standards may conform to UMM business transaction
patterns, ebXML Business Process Specification Schema (BPSS or ebBP),
WS-Choreography or WS-BPEL.
. Business document standards. These may be business content structure and
semantics, taxonomies in use, code lists, semantic rules, or the XML schema
modeling style. They are industry-specific (e.g. RosettaNet PIP schemas, AIAG
Inventory Visibility and Interoperability schemas), horizontal document
standards (e.g. based on UN/CEFACT Core Components, OAGIS Business Object
Documents schemas), or regional guidelines (e.g., KIEC's e-document modeling
guideline).
There have been conformance and interoperability test suites (e.g, ebXML
messaging V2 test suites, WS-I test assertions) and testing tools (e.g., ebXML
IIC, NIST QOD, NIST Business Document Content testbed, KorBIT ebMS testbed, WS-I
test tools) for each above layer individually. But the testing of integrations
of standards has been ad-hoc (e.g. RosettaNet Self-Test Kit is tied to
RosettaNet protocol), or limited mostly to standards in the messaging
infrastructure (WS-I).
Although the need for testing some form of integration of standards has been
well recognized for infrastructure standards (e.g. WS-I profiles), there has
been little support for testing integrations that extend to the use of standards
specific to a business - e.g. for documents or choreographies. Such integrations
can be construed as user-defined profiles. For example, the level of QoS
required for a business transaction may depend on the nature of business data
being exchanged, or on some property defined by the related business process.
Testing and monitoring these infrastructure layers and their integration also
requires that test cases access a combination of contracts - agreements,
policies or business transaction patterns - represented by meta-level documents
(WS-Policy, WSDL, ebXML CPPA, WS-BPEL abstract processes, ebBP definitions, XML
schemas).
This compliance objective goes beyond quality assurance for the messaging
function: it requires the monitoring of live transactions in production
environments, as well as verifying conformance of business endpoints in
operation conditions. This calls for a flexible test execution model that can
accommodate performance constraints as well as different timeliness constraints
- e.g. where tests are either deferred over log data, or executed on live
exchanges in a monitoring mode.
Consequently, the execution model of such test cases or monitoring cases, must
accommodate dynamic conditions requiring real-time or near real-time error
detection or measurement, allowing to correct and report on business exchanges
as they proceed.
The output of a monitoring script also must provide more information than a
report of the type pass / fail. Different ways of "passing" or "failing" must be
reported on, as well as identifying the types of business transactions. The
output must be easy to format and feed to another decision engine, such as a
rule engine that will process this input in real-time. For example, a rule may
decide to generate an alert if a business transaction lasts too long, depending
on the nature of the transaction and on the SLA associated to these business
partners.
This reporting must be easy to consolidate, for Service Level Agreements (SLA)
and Business Activity Monitoring (BAM) purposes. The reporting output may itself
be subject to dynamic testing and monitoring. For example, a rule may decide to
generate an alert if the current average response time for requests of a
specific type, is beyond a threshold.
The TC will define a testing and monitoring model, as well as a test script
markup, so that test cases or monitoring cases can be fully automated, and
portable across test environments.
1c. Scope
The scope of activity for this TC must be within the following topics:
- Use cases and Requirements: Investigation of use cases that may combine
standards - or specifications submitted to an SDO - in all areas concerned by
message exchanges: protocols, QoS, choreography, documents and business content.
Selection and prioritization of related requirements.
- Event Management model: A model for representing, searching, correlating
and managing events intended for the purpose described in this charter. In
particular, an event management approach such as the one described in the
event-driven Test Scripting Language (see Input material) will be considered.
This event management must allow for the same test case to execute either in
static mode, over a set of events all of them are already logged, or in dynamic
mode, over a set of events not all of them have already occurred when the
execution starts. The event model must be open for reuse by at least some other
scripting and processing engines, such as Event-Condition-Action (ECA)
rule-based systems.
- Test case scripting and execution model: An XML markup for representing
executable test cases will be defined. The execution model will rely on the
event management model. The test case scripting may reuse or profile subsets of
existing dialects that are already in use for domains that have similar
functional requirements (if their IP status is compatible with the TC IPR mode),
in which case the markup will act as a coordination or integration language,
treating these dialects either as language extensions or as external resources
used via adapters. In particular, openness to the reuse of XML processing
dialects and tools based on recognized standards - such as XPath, XSLT, XQuery -
is a requirement. The scripting and execution model will attempt to support
extensions for other scripts, in particular should allow for bindings to rule
representations such as SBVR (OMG) or RIF (W3C). The design approach for the
core scripting will favor (a) simplicity of use - a small number of constructs
that are easy to learn and to implement as opposed to feature-rich dialects; and
(b) modular units of scripts that allows for functional reuse and extensibility.
Test scripts will process run time materials (i.e., messages, events, signals),
as well as configurative artifacts such as policies and agreements, schemas and
interface definitions. Test cases or monitoring cases will also provide for more
detailed outputs than pass/fail, and will strive to support use cases defined in
the requirements for SLA monitoring, BAM (Business Activity Monitoring) and
Event-Driven Architectures (EDA). However, the objective will not be to specify
a BAM system or an SLA enforcement system, but only the monitoring technology
that can support these.
- Evaluation and Investigation: of third-party markups and dialects, log
formats and existing test practices, for possible reuse or interfacing in the
test case scripting language. However no third-party specification or tool can
be made essential or necessary to an implementation of the specification to be
delivered, unless its licensing is free and unrestricted.
- Methodology and Guidelines: In scope of this activity, are all forms of
support for users, such as example scripts, best practices, primers and
guideline documents, concerning all topics listed above.
The following documents are input material to this TC, which will deserve prime
attention from the TC assuming their IP status is compatible with the IPR mode
of the TC:
- The event-driven Test Scripting Language (eTSL) V0.85,
http://www.oasis-open.org/committees/document.php?document_id=26036
- Conformance Testing and Certification Framework (OASIS, Conformance TC,
June 2001),
http://www.oasis-open.org/committees/document.php?document_id=309&wg_abbrev=ioc
- QA Framework: Specification Guidelines (W3C, November 2004),
http://www.w3.org/TR/2005/REC-qaframe-spec-20050817/
- Test Assertion Documents for WS-I profiles (WS-I, 2003-2005).
- OASIS IIC Test Framework 1.1,
http://www.oasis-open.org/committees/document.php?document_id=10896
Other documents may be considered by the TC, subject to a TC decision.
Explicitly Out-of-scope:
- Defining or specifying detailed test case metadata or artifacts that
would complement the above scripting - e.g. such as supported in ATML, for
representing test environments, complete test suites and/or their result.
- Definition of particular test harnesses.
- Use cases and requirements that refer to infrastructure specifications
that are not submitted to an SDO, or that refer to business documents or content
that are not approved by an organization or consortium representative of this
business domain.
1d. Deliverables
The TC will produce the following deliverables:
(a) A requirements document, which may include use cases for Internet exchanges,
test assertions for related standards, references to existing test case dialects
or existing logging formats or systems.
(b) A specification defining an event metadata and an event log management model
that supports both the testing and monitoring described in this charter, in a
way that does not preclude other test case scripting techniques than the one
specified in deliverable (c).
(c) A specification defining an event-centric test case scripting and execution
model that supports both the testing and monitoring described in this charter,
and builds upon the deliverable (b).
(d) A set of examples and use cases illustrating the use of above specifications
for testing or monitoring, that involve business content standards, some
choreography standards, and some specifications in the domain of SOAP messaging
or others.
(e) An implementation of the above specifications (b) and (c) used for proof of
the proposed concept and principle.
Timeline:
The TC will aim at a Public Review draft of both deliverable (b) and (c) above
by first half 2009.
1e. IPR Mode
Royalty-Free on Limited Terms.
1f. Audience
The TC welcomes any OASIS member with an interest and/or experience in testing
and monitoring.
1g. Language
English
Non-normative Information
2a. Identification of similar or applicable work that is being done in other
OASIS TCs or by other organizations, why there is a need for another effort in
this area and how this proposed TC will be different, and what level of liaison
will be pursued with these other organizations
In general the following works fall short of addressing TaMIE charter, in that
they:
. either target a specific protocol stack,
. or a particular domain of business.
Their support for testing and monitoring the implementation of a combination of
standards is either inexistent or limited, and they do not provide a versatile,
extensible scripting of test cases. The requirements that are specific to B2B
environments are not well addressed either: none is addressing both run-time
monitoring of systems in production, and conformance testing prior to
deployment. Only the latter is generally supported by existing work, e.g.
neglecting real-time monitoring cases that may lead to dynamic notification or
remedial actions, or ignoring real-time constraints about events throughput and
recovery after shutdown time. These test environments below - at the exception
of WS-I tools - do not support the combined processing of electronic documents
representing agreements or policies (metadata) and of run-time logs. Finally,
none supports a general-enough notion of event, along with adequate time
primitives and event correlation and search capability.
(a). The OASIS ebXML IIC test framework (TF v1.1)
http://www.oasis-open.org/committees/document.php?document_id=10896
This framework was designed specifically to test the conformance and
interoperability of the software implementing the ebXML Messaging Specification.
The framework defines components and harnesses (configurations) necessary to
simulate and control the environment for conformance and interoperability tests.
A shortcoming of the IIC TF is that it has been designed for ebXML messaging
2.0, and provides weak extensibility options, both for external events and for
specialized evaluations. The IIC TF does not support other suites of B2B
integration standards such as other SOAP profiles integrating several Web
Services standards. It also does not support ebMS 3.0 and electronic documents
testing.
(b). RosettaNet Self-Test Kit (RNSTK)
RNSTK
(http://serg.telecom.lth.se/education/master_theses/docs/72_Kjellin_slutrapport.
pdf)
is a test system provided by the RosettaNet consortium. The system allows
software solutions to perform self-tests for compliance to business
collaboration scenarios and the RosettaNet Integration Framework (RNIF)
specification. The RNSTK tests an integrated system implementing particular B2B
collaboration scenarios, yet is tightly dependent on RNIF messaging. RNSTK test
suites are encoded using the RNIF specification itself. Due to these reasons,
the RNSTK cannot be used to test other suites of B2B standards including the
Document Type Definition semantics [20]. Another weakness is that its
architecture only supports conformance test configurations.
(c). Web Services Interoperability (WS-I) Test Tools (for Basic Profile 1.1)
These tools do not allow for the scripting of a test suite; instead they apply a
battery of predefined tests to any Web service material provided as input. The
objective is to verify the conformance of this material to one of the WS-I
profiles. The tools only passively analyze message traffic and have no ability
to control the systems under test. Nevertheless, this capture and analysis
architecture is un-intrusive, simple, and supports deferred testing. The
analyzer tool of WS-I cannot be used as a general test engine, being tied to
profile definitions; but, it can be leveraged as a specialized module by a more
general monitoring capability, when the conformance of message material to WS-I
profiles is required.
(d). TTCN-3 (Testing and Test Control Notation Version 3, ETSI, June 2001)
TTCN-3 (http://www.ttcn-3.org/StandardSuite.htm) is a powerful, programmatically
complete procedural language with specialized constructs for defining test
procedures, test verdicts, matching mechanisms for evaluation, timer handling
and distributed test components. Due to its original focus on telecommunication
systems, it also has the ability to specify encoding information, to communicate
both synchronously and asynchronously, and to perform passive monitoring. These
capabilities are all essential for testing in message-based B2B integration.
Relative to previous test frameworks, TTCN-3 is more powerful and flexible.
However, TTCN-3 notation can be difficult for a business or domain expert to
use. TTCN-3 has not been designed to delegate some functions to external modules
and it is weak when it comes to validating business transaction events and XML
electronic documents, which are essential for B2B integration testing.
(e). ATML (Automatic Test Mark-up Language, IEEE, December 2004)
ATML
(http://grouper.ieee.org/groups/scc20/tii/ATML/Working%20Groups/Management/ATML%
20Overview.doc) has been designed for exchange of diagnostic information,
configuration data, and test instrument definitions between conforming software
applications. It was not designed to support test scripting and execution.
However, ATML representations could be complementary to executable test case
scripts.
(f). General Software Test Environments and Scripting JXUnit.
JXUnit is an XML-based test-scripting system built on top of the JUnit Java
classes. It is a data-driven testing system in which input to the system under
test and expected output are specified and then the actual output is evaluated.
There is no built-in support for B2B testing, in particular the communication
and the event-driven tests. However, as a general, test-scripting platform that
relies on a common programming language, JXUnit and JUnit could be used as an
implementation platform for essentially any test framework including the one
proposed here.
2b. First Meeting:
January 28, 2008 at 4pm PT/7pm ET. The first meeting will be a 2h conference
call. Fujitsu or KIEC will sponsor the call.
2c. Projected Meeting Schedule:
A 90mn meeting every two weeks will be scheduled, unless the TC decides on a
different frequency. Assuming the first meeting is a conference call, a
face-to-face meeting will be set within 2 months of the first meeting once the
work has actually started, to maximize the value of face time.
2d. Co-proposers
AIAG (Automotive Industry Action Group): Mohammad Abidi,
Mabidi @ aiag.org
Axway: Dale Moberg,
dmoberg @ axway.com
Fujitsu: Jacques Durand,
jdurand @ us.fujitsu.com
KIEC (Korea Institute for Electronic Commerce): Hyunbo Cho,
hcho @ postech.ac.kr
NIST (National Institute of Standards and Technology): Nenad Ivezic,
nivezic @ nist.gov
OAGI (Open Applications Group, Inc.): Dave Connelly,
Dconnelly @ openapplications.org
2e. Convener
Jacques Durand, Fujitsu
2f. Member Section
N/A
2g. List of Contributions
The following work, already submitted in October 2006 to OASIS ebXML
Implementation, Interoperability and Conformance TC, will be contributed to this
TC:
- eTSL draft V0.85, (originally derived from the Test Framework 1.1
produced OASIS IIC, 2004-2005)
2h. Draft of F.A.Q.
Will be provided later.
2i. Proposed Working Title for deliverables
Deliverable #1: eLAM - event Log Access and Management Model.
A specification defining a event representation and metadata along with the
event store management primitives for supporting event-centric testing and
monitoring.
Deliverable #2: eTSM - event-centric Test Scripting and Model.
A specification defining a test case execution model that supports both the
testing and monitoring of message and business data exchanges, and the scripting
language for defining and executing these test cases.
Deliverable #3: eTSM Use Cases Document - A set of examples and use cases
illustrating the use of above specification for testing or monitoring business
exchanges based on (a) some business content standards, (b) some choreography
standards, and (c) some specifications in the domain of SOAP messaging or
others. This document may serve to illustrate requirements that have been
addressed.
Deliverable #4: An implementation of the above deliverables #1 and #2 used for
proof of the proposed concept and principle.
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]