[jsdl-wg] DataStaging concerns

[Peter G.Lane]

>> 1) There's still controversy over whether staging should or should 
>> not be integrated into a DRM.  As far as I can tell, for example, the 
>> BES doesn't have any plans to implement staging.  DRMMA makes this 
>> optional.  If BES ends up using JSDL, wouldn't this be a violation of 
>> the spec which requires each element to be supported in some way?
>
> "Supported" has a very particular meaning within a JSDL context, and 
> the
> effective meaning could include a definite response "I don't know how 
> to
> do data staging, man!"

Great, thanks for that explanation.  I had kind of assumed something of 
the sort, but I wasn't sure.

>  We discussed data staging quite a few time
> (around a year ago IIRC) and what we came up with is a minimum to allow
> processing of jobs on a number of different systems including domains
> like cross-cluster deployment where everything has to be shipped in 
> first.
>
> If we'd had a proper workflow language too, we'd have done data staging
> differently. But there wasn't something suitable already existing (BPEL
> does something else) and if we'd have had to develop our own, we'd 
> still
> be arguing about it now.
>
>> 3) I don't particularly like that the DataStaging sections include an 
>> option to remove the file at the end of the job.  If I'm staging out 
>> data then this doesn't make a whole lot of sense.  I'd much prefer a 
>> separate section which explicitly lists all the files that are to be 
>> removed from the submission machine after the job has been completed. 
>>  This would also cover the case of data that is created rather than 
>> staged in but still needs to be removed after job completion.
>
> The "remove data at the end of the job" applies after staging it out
> (it'd be a bit silly otherwise).

Right. I was thinking that it would mean "remove from target".  I 
understand the semantics better now, so this makes sense.

> It is also the case that you can list a
> file in the data staging section and not have it staged in or out, but
> just deleted at end-of-job.

Great!  I hadn't considered this possibility.

>> 4) Based on the current GRAM incarnation, it would be nice to let 
>> RFT's transfer request description extend a base staging schema and 
>> then use that in the JSDL document rather than adding a bunch of 
>> extensions to DataStaging.  This is similar to how I'd want to go 
>> about using POSIXApplication.
>
> I don't fully grasp what you mean here. The following is legal (modulo
> namespaces) according to draft-18:
>
>   <jsdl:DataStaging>
>      <jsdl:FileName>example</jsdl:FileName>
>      <!-- I think we agreed to default this next element -->
>      <jsdl:CreationFlag>jsdl:overwrite</jsdl:CreationFlag>
>      <jsdl:Source>
>        <!-- I've no idea what your RFT schema might look like -->
>        <rft:SynchFileWithSomewhere>...</rft:SynchFileWithSomewhere>
>      </jsdl:Source>
>   </jsdl:DataStaging>
>
> (Hmm, the non-normative examples in the data-staging section of d18 
> seem
> out of step. Bother.)
>
> Given that the above is legal, what's the problem?

Actually, this clarifies things for me.  This misunderstanding was also 
due to my incorrect interpretation of what Source and Target meant.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2782 bytes
Desc: not available
Url : http://www.ogf.org/pipermail/jsdl-wg/attachments/20050520/8e8cc375/attachment.bin