[ogsa-d-wg] Stepping back: the high-level view

[Dave Berry]

All,

I've been tasked with rewriting the sections of the main OGSA document
that deal with data services.  This is providing interesting as a check
on the work we've been doing on the data architecture document itself.

The good news is that much of the existing description in the OGSA
document remains accurate.  This suggests we did a good job early on.

The OGSA document does describe some higher-level features that we
haven't developed sufficiently in the data architecture document.  I
think the most important is the specification of policy, QoS, etc.
We've always been aware of this during our discussions but we haven't
really developed it, at least not across the whole architecture.  I
think we need to make this the next cross-cutting issue that we address
(analogous to the way we have addressed transactions and security).  I
also think we need to raise this with the OGSA WG, as we will need to
use the same mechanisms as the rest of OGSA.  (It may be that we force
the pace of this, as I suspect QoS is more important in data services).

We also need to think about how to provide performance information, e.g.
latency, bandwicth, network location, etc.  This is clearly linked to
the QoS question.

Another cross-cutting concern is graceful degradation of availability.
The OGSA document says, "The OGSA data services provide features for
graceful degradation in the event of network or other failures. For
example, query services may be configured to return partial results when
only a subset of sources is available".  I suspect we need to address
this as another cross-cutting feature.

Another major feature from the OGSA document that we haven't addressed
is that of transformations and derived data services.  I think we took
an explicit decision to leave these to a later version of the
architecture - does this match your memories?  If we agree this is the
case, I'll discuss with the OGSA WG how best to reflect this in the OGSA
document.

The OGSA document also lists provenance as a particular feature that
data services support.  Provenance is clearly important but I doubt we
will address it explicitly in the current version of the data
architecture.  We made need to address it in the longer term.  It seems
to me that provenance should be just another form of metadata - but I do
know of projects that have developed particular architectures for
storing provenance data (e.g. the PASOA project in the UK).

Best wishes,

Dave Berry
Research Manager, National e-Science Centre
15 South College St., Edinburgh
Tel: +44 131 6514039