MDDL
User Application Apr.
2 Meeting April 11, 2001
Organizations
Present Associated
Press, Bank of New York, Bear Stearns, Fidelity Investments, FISD, Goldman Sachs,
JP Morgan Chase, Lehman Brothers, Merrill Lynch, Morgan Stanley Dean Witter Meeting
Objective The
primary purpose of the meeting was to create a functional user application requirement
for MDDL. The function of the user requirement will be to ensure that MDDL is
meeting the core business objectives of the user firms. Action
Points 1)
Prioritize the vocabulary fields. The user firms will review the working domain
structure(s) for prioritization and rationale (i.e. organization, groupings) as
it relates to usage. Strong support for a standard definition of and a relationship
diagram among fields. Rich Robinson (BONY) is the champion. 2)
Definition of functionality with examples. This is the key objective. Alex Kogon
(Bear Stearns) and Mark Rayman (Merrill) are the champions. There are two "buckets" - a)
data warehousing -- raw feed to populate databases (define what's being provided
by vendors) and
- b)
Client server applications -- how applications will request the data (define how
to get the data in/out of applications).
3)
Definition of a standard request/response. Will address communication message
format issues as well as the two types of requests (user pull and vendor push).
Myank Gupta (Morgan Stanley) is the champion.
Functionality
Requirement There
is broad consensus among FISD members that user application requirements are the
key business drivers of the MDDL initiative. User firms distribute a significant
amount of pre-defined market data to a wide variety of database applications.
As such they spend time and money translating market data formats and modifying
applications for internal communication. The
goal is this activity is to enable users to integrate data from multiple sources
by standardizing both the input feeds used for data warehousing (e.g. define what's
being provided by vendors) and the output methods by which client applications
request the data (e.g. ensure compatibility on how to get data in and out of applications).
The basis of the user requirement is for MDDL to: 1.
Support multiple vendors 2.
Common vendor interface (identical query and delivery protocol to all vendors)
3.
Common request format for different vendors with standard request types and field
names 4.
Standard snap request by vendor specific instrument names or by vendor independent
ISIN, exchange code, instrument type (or any combination) 5.
Dynamic (user pull) snapshot data elements to be included in request/response
6.
Allow requests for vendor specific fields 7.
Allow requests for vendor specific services 8.
Each request must contain user, application and host information. Each request
to a service must be made with user information to allow entitlement checking
or entitlement checking must be done by our application 9.
Have receiver respond to server with ID# to confirm receipt, redeliver if never
confirmed 10.
Tag records with sequential IDs to facilitate partitioning of data stream and
delivery guarantees 11.
For all standardized requests, response must use standard field names 12.
Allow receiver to report erroneous records to server for validation and data quality
13.
Provide standards for how fields such as IDs should be used 14.
Provide standard for common request/response interface to ensure that with proper
implementation of MDDL, applications are ensured plug-and-play compatibility between
back-ends
Request/Response Format An
ongoing discussion on request/response formats and the importance of including
a query protocol as part of MDDL 1.0 emerged as a result of this meeting. The
chain of e-mail messages is being posted to group.yahoo.com. In the meantime,
here's a copy of the e-mail discussions The
initial attempt at documenting request/response capabilities included: session/request
identifiers, user/application identifiers, multiple identifiers and types per
request, pattern matching, aggregate identifiers (i.e. exchange or options chains,
index chains, etc.), standard field sets versus or variable field lists, multi-part
requests/responses, and the ability to match the request input with the response.
--------------------------- (FROM
MIKE BENVENISTE, FIDELITY) I've had some time
to think over Monday's discussions, and I'd like to give my reasons why we should
not make a query protocol part of the MDDL 1.0 specifications 1.
We can provide a useful and viable standard without it. My criterion
for this is pretty simple; can my team create and deploy better applications more
efficiently if we receive data in an MDDL format even if the query format is application
specific? My first four potential applications for MDDL involve a scheduled feed,
a wireless application, a direct user request for a particular security, and a
portfolio-based request. The first two applications are both "push"
applications with decoupled queries and responses. The third app will require
a servlet in any case, so there is a clear integration point for query formation.
In the fourth case, a standardized request format could eliminate a conversion
tier, which is a significant gain. Even in this fourth case, the gains from standardization
of data are still quite large. Note that my requirement is not for a perfect standard,
but a viable one. 2. "Time to Market" is critical for management
buy-in. Quite simply, we have enough open issues on how to represent data.
Since the format of "downstream" data is independent of the query, deferring
this issue will not create any backwards compatibility issues within the spec.
By deferring the issue, we reach our 1.0 milestone faster. I feel the gains from
such an "early victory" outweigh the technical advantages at this time.
For proof of concept, we will create and hopefully deploy MDDL-based solutions
based on current data stores and vendor feeds. I'm prepared to write and support
inward looking filters to do so only if they are fairly simple. If the filters
have to comply with a generalized query format on Day 1, it will significantly
add to the complexity of such initial projects and the time of completion. 3.
Integrity of the Specification Are we prepared to say that in order
to call a service or application MDDL-compliant, it must use the MDDL query format?
If not, how do we answer that traditional management buy-in question: "what
is MDDL, anyway?" 4.
MDDL's need for a standardized query format is not industry specific. W3
started work on a query language in 1998, and the issues raised in that workshop
are the same ones we're examining. See http://www.w3.org/TandS/QL/QL98/pp.html
for some examples. The result of that effort is Xquery, http://www.w3.org/XML/Query.
While Xquery is still in draft form, it would fulfill the needs that Mayank laid
out with the exception of transactional or session identification. For anyone
attending XMLDevCon next week, there is a session on Xquery: <http://www.xmldevcon2001.com/NY/html/session.php?code=S5>
Using a non-industry specific standard increases the likelihood of being able
to leverage commercial tools. For example, Tamino, an XML native database from
Software AG, already implements a subset of Xquery. While dusting off my 'lex'
and 'yacc' skills might be fun, one of the major goals of XML is to eliminate
the need to write custom parsers. Let's not reintroduce that need. 5.
We have a bit of a chicken and egg problem. MDDL is likely to have
higher network bandwidth requirements and CPU processing requirements than traditional
vendor protocols. The engineering and architectual tradeoffs in determining how
and where to deploy MDDL in the enterprise are unsolved problems. Until we get
some pilot programs in the field, we probably won't know enough to solve them.
Many of these problems fall into query formation. For example, Mayank's proposal
raises the issue of transactional information in the query, and, perforce, the
response. Whether this should be part of the query or managed through an independent
session manager depends on the types of systems and communication protocols involved.
A related example involves the concept of field sets. These could be predefined
or transactional. If predefined, is it necessary to standardize? If so, do we
do so as part of the spec, or via URL/URN's? If transactional, we end up requiring
all MDDL service providers to keep state information per user. This latter requirement
has significant implications for high-availablity and large load-sharing
applications. In
summary, I don't want to sweep these issues under the rug forever, just long enough
to "declare victory" on 1.0 and to understand what we're going to do
with this thing called MDDL. All comments, revisions, and suggestions welcome.
------------------------------- (FROM
TONY ZHANG, FINPORTFOLIO) Mike raised some very good points regarding the
scope of MDDL 1.0 spec. I have also took some time to think over the issues related
to a messaging protocol in MDDL. I think it is an important decision for MDDL
and we probably want to sort it out as quickly as possible. I agree with Mike's
points and my inclination is not to include it in MDDL 1.0, but to keep the option
open for future MDDL revisions. In addition to Mike's comments, I would like to
throw in a few more points to stir up the discussion: 1.
MDDL should be a layering standard where its messaging layer can be relatively
independent of its format layer. We understand the importance of having a messaging
standard for MDDL, but we should take a progressive approach to build up the full
MDDL spec (this seems to be the spirit of our approach so far) - we may declare
MDDL 1.0 as the format standard and some future version as the format and messaging
standard. 2.
The complexity of mixing a messaging protocol with a format protocol may be a
multiple of the aggregate. We can take a look at some of the existing messaging
protocols, such as FIX (Financial Information Exchange Protocol, www.fixprotocol.org),
OFX (Open Financial Exchange, www.ofx.net)
and IFX (Interactive Financial Exchange, www.ifxforum.org).
Many of the issues raised in MDDL, such as request/response, session and transaction,
have been discussed and dealt with extensively. Unfortunately, these efforts produce
some complicated specs, which are both time-consuming and difficult to implement.
Given the timeframe we current have to wrap up MDDL 1.0, I would suspect we would
like to go down that route. 3.
As to coming up with a messaging protocol, I would recommend everyone to check
out SOAP, opposed to a mixed-in messaging protocol like the one proposed in Mayank's
email and some other protocols. SOAP (Simple Object Access Protocol, see <http://www.w3.org/TR/2000/NOTE-SOAP-20000508/>),
developed by Microsoft and IBM, has been submitted to W3C as the base for XMLP
(XML Protocol, see http://www.w3.org/TR/2001/WD-xmlp-reqs-20010319/,
and has been adopted by OASIS as a messaging layer for ebXML (see http://www.ebxml.org/news/pr_20010222.htm)
There have been many tools developed for SOAP, such as Apache/IBM SOAP 2.1, Sun
Java XML-RPC, and Microsoft SOAP Toolkit 2.0, just to name a few. 4.
As a general note, I think leveraging current standards and standard-based tools
will greatly enhance MDDL's market position. As Mike pointed out, we probably
don't want to reinvent the wheel for non-industry specific aspects of the MDDL
spec. This will help us develop MDDL spec faster and reach broader market acceptance.
----------------------------------- (FROM
MYANK GUPTA (MORGAN STANLEY) I started Monday's meeting feeling there was
no need for a request/response format...but changed my mind during the discussions.
If snapping is in scope for version 1.0 request/response criterion have to be
documented for reference. They will influence MDDL 1.0 content format decisions
both in the vocabulary and technical committees. Both
proposed structures are somewhat arbitrary. This seemed to be a good way of documenting
application requirements. The query structure is just two lists (idenfiers, fields).
The response has a table like view(ie Identifer-field(s)). Many other alternatives
could be easily substituted. I'll try to summarize the main points that Mike and
Tony have raised. Please add/remove from this as necessary. 1.
Messaging Protocol vs. Format Protocol The proposed structure borrows
heavily from the SOAP specification. It separates transactional/control elements
from the "query" portion. Both of these elements are optional. The only
condition placed is that if the query defines them...the server has to return
them back(as they were in the query) with the response (needed for asynchronous
processing). Parsers not interested in these sections (elements) would ignore
them and/or would not error out if they were missing altogether. The request/response
elements(with or without the tran/control elements) can be wrapped in any other
document. 2.
SOAP (see above). The SnapRequestHeader corresponds to a SOAPHeader, The
SnapRequest/SnapResponse sections correspond to a SOAPBody and support is included
for Faults when and where needed. SOAP does not endorse any specific elements
for dispatching, routing, user info and leaves this to individual DTD's/Schemas.
I'm proposing that we at least be aware of, if not define some of these elements
(eg. userId, application, data quality etc.) 3.
Need for Query functionality (as per Mike's use case examples) I agree
with Mike and Tony that if MDDL 1.0 only defines the format layer we've made tremendous
progress. However keeping the future in mind we need to design a robust Identifier
element capable of supporting multiple identifiers, identifier types and pattern
matching. A restriction I'm proposing is that the response from a snap include
identifiers as they were specified in the request. Additionally in the response
results (fields) be wrapped by the identifier they belong to. The proposal also
creates a status hierarchy (document level status, identifier level status, field
level status). 4.
Compliance with MDDL The proposal is an attempt to document application
requirements. Since the documents are stand alone they can be parsed independently.
Without further work on the format portion we cannot say what is a required for
MDDL compliance. 5.
Xquery A cursory review of the XQuery spec leads me to believe this
is more for querying/transforming doucments. We can view all of the vendor's content
as one big document that can be queried against to produce the "filtered"
document we're interested in. The proposed format and XQuery are not exclusive.
All vendors would have to implement - field sets are/should be optional. Some
vendors have them and it would be necessary to support them not necessary to require
them from all vendors.
|