[ogsa-dmi-wg] PGI and DataMINX Data Transfer Service

[Mario Antonioletti]

Hi,
    Apologies if you get this more than once - Dave tried to email it 
to the list earlier today and it appears not to have got through so
trying from my account to see if this makes a difference.

Mario


-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.


---------- Forwarded message ----------
Date: Tue, 27 Oct 2009 11:48:40 -0000
From: david.meredith at stfc.ac.uk


Hi folks,

Can I introduce our DataMINX project
(http://code.google.com/p/dtsproject ). We are currently involved with a
data transfer service for supporting a range of different protocols and
would like to adopt and potentially contribute to a relevant OGF spec if
appropriate. DMI and JSDL file staging have their strengths and
weaknesses in this area, and perhaps there's an opportunity to exploit
these strengths within each standard in some way (Steve, hope you don't
mind me re-using your words here). DataMINX is a UK-Australia
collaboration currently involving the UK NGS, OMII-UK, ARCS and
Intersect.

May I also point you to our wiki
(http://code.google.com/p/dtsproject/wiki/DMI_JSDL_Issues ) where we
have put together a summary of some JSDL/DMI issues for use in a bulk
data transfer type service. The wiki presents a couple of draft
proposals (DMIB and JSDL Data Transfer). Please note both are merely
rough proposals with the aim of capturing requirements and for
compare/contrast/creating-conjecture (hopefully lots to discuss).  For
brevity, I have summarised the proposals below, but please do refer to
the wiki as there are lots of considerations/requirements and discussion
points.

Kind regards,
Dave Meredith (on behalf of the DataMinx team)


DMIB
The first proposal ('DMIB' where we have used 'B' to indicate
Bulk-transfer) re-uses the existing DMI elements in a new
document-literal wrapped style message model. This does not modify the
existing DMI (except for the addition of some new <xsd:any> extensions),
rather, it wraps together the SourceDEPR and SinkDEPR elements (both
<wsa:EndpointReferenceTypeS>) and the <dmi:TransferRequirements> into a
single wrapping element. This 'wrapping' element can be defined multiple
times in a single request packet in order to define a bulk transfer.
(The <dmi:DataLocations>, <dmi:Data> elements are not modified and are
re-used as per the dmi spec). Semantically, this appears to be similar
to defining multiple source and target JSDL DataStaging elements.

JSDL Data Transfer
The second draft proposal, defines a <DataTransfer> element (rather than
DataStaging). This is used to define some extra elements and modifies
the format of the <DataStaging> element to be more suited to bulk
transfers, since staging data to/from an intermediary compute resource
may not appropriate here (implementation detail). Additions include:  a
repositioned Credential element, new abstract URIProperties element,
client defined sub-transfer id (is used in conjunction with the overall
bulk transfer id for drilling down to determine status of
sub-transfers), added a modified <dmi:TransferRequirements/> element
that adds the <jsdl:CreationFlag/>.



-----Original Message-----
From: Steve Crouch [mailto:s.crouch at omii.ac.uk]
Sent: 26 October 2009 10:18
To: Morris Riedel
Cc: Meredith, David (STFC,DL,ESC); gerson galang; Alex Arana; Peter
Turner; Mario Antonioletti; Shahbaz Memon
Subject: PGI and DataMINX Data Transfer Service

Hi Morris,

You may recall from the PGI telecon we had on the 2nd October we
discussed a set of JSDL requirements for datastaging that have come out
of the Australian DataMINX project (including Credential and additional
'URI properties' elements within Source and Target elements).  As a
recap, they have secured funding to to develop a scalable,
production-level, heavy-use Data Transfer Service for deployment, and
wish to adopt the relevant OGF standard(s) for the service and
contribute to these standards.  They're also currently pursuing
requirements for an OGSA-DMI rendering within the DMI WG.

I've cc'd Peter Turner, PI of the DataMINX project, David Meredith from
STFC (UK), Gerson Galang from ARCS, and Alex Arana from Intersect who
are involved in the project.  Previously, David's been heavily involved
in developing the JSDL parameter sweep extension.  They can elaborate
with far greater precision what is required here.  I've also cc'd Mario
Antonioletti and Shahbaz Memon from the OGSA-DMI WG, where some progress
has been made already within DMI.  What we've realised is that both DMI
and JSDL have their strengths and weaknesses in this area, and perhaps
there's an opportunity to exploit these strengths within each standard
in some way.

I recall you were planning to come up with a JSDL profile that includes
the requirements we discussed e.g. from SRM (amongst others).  Are you
aware at this stage what your plans are for developing the profile?  If
it's ok, if you could let us know your plans for the profile and how we
could be involved that would be great.

Many thanks,

Cheers
Steve

-- 
Dr Stephen Crouch,
Software Architect,
OMII-UK,
School of Electronics and Computer Science,
Room 4067, Level 4, Building 32,
University of Southampton,
Highfield, Southampton SO17 1BJ
Tel: +44 (0)23 8059 8787        EMail: s.crouch at omii.ac.uk
Fax: +44 (0)23 8059 3045        WWW: http://www.ecs.soton.ac.uk/~stc


--
Scanned by iCritical.