[ogsa-dmi-wg] Moving DMI stuff forward

[Meredith, DJ (David)]

Hi folks, 

(apologies if you have already received this, I seem to be having
trouble posting to the list). 

 

 

A bit more material in preparation for Mondays phone meeting, items 1)
and 2). 

 

1)  Catering for moving of multiple source/sinks within a single atomic
operation. 

Basically, an important requirement for us is to define a bulk data
transfer that could accommodate the transfer of multiple files/dirs
within a single atomic operation. We realize that the way to do this in
the DMI world is probably to instantiate a number of DTI instances,
where each instance caters for a single source DEPR and an associated
sink DEPR. However, ideally we would like to be able to define multiple
transfers in a single atomic request, and importantly, be able to define
this within a single xml doc. 

 

As Gerson has shown in his previous email, we have also looked at the
JSDL HPC file staging spec that facilitates the definition of multiple
DataStaging elements (each containing a srcURI and an associated
targetURI and much like DMI, embeds required credentials). At present
however, neither JSDL File staging or DMI completely satisfy our
requirements hence the current JSDL-DMI hybrid (
http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/mai
n/resources/minx-dts.xsd). Of course, ideally we would prefer to adopt a
spec. 

 

We were wondering if something like the following would be considered
'in-scope' / suitable for DMI ? (please see the schema and sample
document in the attached examples.zip).  The attached schema only adds
the following extra element and complex type to the dmi-messages.xsd
(below). In doing this, it becomes possible to wrap multiple
source-and-sink DEPR combinations within the same doc. 

 

       Thus, rather than define a single source and sink such as:

           GetDataTransferInstance( [source DEPR], [sink DEPR],
[transfer requirements] );

       Maybe define:

           GetDataTransferInstance(
[GetWrappedDataTransferInstanceRequestMessage] );

 

 

  <!-- 

Proposed additional Request Message element for moving multiple
files/dirs in one atomic unit: 

  -->

  <element name="GetWrappedDataTransferInstanceRequestMessage"
type="dmi-plain:GetWrappedDataTransferInstanceRequestType"/>

  <complexType name="GetWrappedDataTransferInstanceRequestType">

    <sequence>

      <element name="WrappedSourceSinkDEPR"
type="dmi-plain:GetDataTransferInstanceRequestType" minOccurs="1"
maxOccurs="unbounded"/>

    </sequence>

  </complexType>

 

 

2) xsd:any extension points 

Would the working group also consider the addition of some xsd:any
extension points within the dmi.xsd ? (e.g. within the DataType complex
type). This is necessary so that we can define extra connection
properties such as the MCATZone and MdasDomain for SRB /iRODS. This is
also shown in the attached example. I think there a some other items but
I forget. 

 

Cheers, 

Dave

 

 

 

 

 

From: Gerson Galang [mailto:gerson.galang at arcs.org.au] 
Sent: 19 June 2009 13:11
To: Mario Antonioletti; OGSA-DMI
Cc: Peter Turner; Stephen Crouch; Meredith, DJ (David);
alexa at intersect.org.au; Mohammad Shahbaz Memon; Ravi Madduri; Allen
Luniewski; Carlos Aya; Christopher Mendes; Paul Coddington
Subject: Re: Moving DMI stuff forward

 

Hi guys,

Just thought of updating you on where we are at in terms of the DTS
development and schema specification. At the moment, we've finished
implementing the modules for the DTS and are starting to integrate bits
we implemented so we could demonstrate a very basic type of transfer (ie
ftp to smb). We've also started planning on how we could complicate
stuff by integrating Shibb authentication. 

So here's the stuff that you might be interested in.. the approach we've
taken is take bits and pieces off JSDL and OGSA-DMI. See the current
schema we have..

http://code.google.com/p/dtsproject/source/browse/trunk/dts-jaxb/src/mai
n/resources/minx-dts.xsd

You probably might be wondering why we decided to replicate stuff from
JSDL and OGSA-DMI instead of just importing from them. The reason for it
is, we just wanted something quick and dirty to play with. The 'quick
and dirty' start will then allow us to properly identify extension
requirements for both JSDL and DMI and ultimately for implementing with
respect to either JSDL and DMI. The plan is to import from standard
schemata and use them instead of reinventing the wheel. At the moment,
not all we needed are implemented by the schemata so we've gone for the
approach of just using standard when it has the stuff we need. The stuff
we're working on is a prototype only and modifying it to use  the
standard won't be that hard to do.

We quite like the JSDL way of specifying the job definition so we've
taken the JSDL schema and wrapped it with web service operations that
use bits and pieces from DMI. We also use the DMI standards for the
messages that will be exchanged by the WS and its client.

Regards,
Gerson

On Thu, Jun 18, 2009 at 9:44 PM, Mario Antonioletti <
mario at epcc.ed.ac.uk> wrote:


Hi,

	 

	Hi Mario, think 8am GMT Monday June 22 would be 5pm AEST Monday
June 22. This seems generous on your part, we could certainly
accommodate a later start - if we say 8:30am GMT you get a little more
sleep at little cost to us :-)

 

I'm usually in well before 8 so let's leave it at that. My UK colleagues
saidthey could make it....

	 

	Could also do later.
	
	I've added little to the briefing doc since we last communicated
- had no time. Happy to provide it as it currently is - does not include
anything of the work undertaken since the UK visit. Gerson and Alex have
been making progress. I'll ask then how best to provide access to their
stuff.

 

Could you guys come up with an agenda as to what needs to be discussed
and I'll email the list - hopefully today or tomorrow. If the briefing
doc is fit to show to the public then we could just distribute that.


Let me know,

Mario


+-----------------------------------------------------------------------
+
|Mario Antonioletti:EPCC,JCMB,The King's Buildings,Edinburgh EH9 3JZ.
|
|Tel:0131 650 5141|mario at epcc.ed.ac.uk|http://www.epcc.ed.ac.uk/~mario/
<http://www.epcc.ed.ac.uk/%7Emario/>  |
+-----------------------------------------------------------------------
+
-- 

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

 


-- 
Scanned by iCritical.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ogf.org/pipermail/ogsa-dmi-wg/attachments/20090619/58534df2/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: examples.zip
Type: application/x-zip-compressed
Size: 4348 bytes
Desc: examples.zip
Url : http://www.ogf.org/pipermail/ogsa-dmi-wg/attachments/20090619/58534df2/attachment-0001.bin