[DRMAA-WG] Torque/PBS DRMAA - file transfer routines

[Rajic, Hrabri]

Thanks Andre,

That could be a good starting point for us.

File transfer resolution is a complex issue, that would require a
different approach today than 5 years ago, and possible yet another
approach 5 years from now.  Is it part of the current spec or another
one could be also a good question.

One litmus test for making it a standard would be when implementations
start adding it as optional attributes.

If this issue is not already recorded as a tracker we need to add it.

For now this issue does not impact the current DRMAA-R candidate
document.

Regards

	Hrabri



>-----Original Message-----
>From: drmaa-wg-bounces at ogf.org [mailto:drmaa-wg-bounces at ogf.org] On
Behalf
>Of Andre Merzky
>Sent: Saturday, May 05, 2007 9:40 AM
>To: Peter Troeger
>Cc: DRMAA Working Group
>Subject: Re: [DRMAA-WG] Torque/PBS DRMAA - file transfer routines
>
>For what its worth, the SAGA group also stumbled over this
>problem.  We finally opted to adopt a LSF like syntax,
>following a suggestion from Chris Smith (no surpise, ey?  ;-).
>
>That lsyntax ism in our opinion, generic enough to easily
>map to the various backends.  The syntax basically is
>(quoting the SAGA spec):
>
>-----------------------------------------------------------------
>  File Transfer Specifications
>  ----------------------------
>
>  The syntax of a file transfer directive for the job
>  description is modeled on the LSF syntax {(LSF stands for
>  Load Sharing Facility, a commercial job scheduler by
>  Platform Computing), and has the general syntax:
>
>    local_file operator remote_file
>
>  Both the local_file and the remote_file can be URLs.  If
>  they are not URLs, but full or relative pathnames, then
>  the local_file is relative to the host where the
>  submission is executed, and the remote_file is evaluated
>  on the execution host of the job.
>
>  The operator is one of the following four:
>
>      '>'  copies the local file to the remote file
>           before the job starts. Overwrites the remote
>           file if it exists.
>
>      '>>' copies the local file to the remote file before
>           the job starts. Appends to the remote file if
>           it exists.
>
>      '<'  copies the remote file to the local file after
>           the job finishes. Overwrites the local file if
>           it exists.
>
>      '<<' copies the remote file to the local file after
>           the job finishes. Appends to the local file if
>           it exists.
>-----------------------------------------------------------------
>
>That mechanism is also used for stdin/stdout/stderr.  I
>don't want to say that DRMAA should use that - there are
>certainly limitations - but might be worth to look at at
>some point...
>
>Cheers, Andre.
>
>
>Quoting [Peter Troeger] (May 04 2007):
>> From: Peter Troeger <peter.troeger at hpi.uni-potsdam.de>
>> To: DRMAA Working Group <drmaa-wg at ogf.org>
>> Subject: Re: [DRMAA-WG] Torque/PBS DRMAA - file transfer routines
>>
>> > Issue:
>> >
>> > File transfer routines are desired by many users and currently they
>> > are
>> > only limited to standard input, output and error streams.
>> > ---------
>> >
>> > Omitting file transfer capability was done on purpose in version
>> > one to
>> > put all the major DRM systems at the same footing.
>> >
>> > On of the ambitious DRMAA goals was to influence DRM system vendors
or
>> > providers to standardize (one more time) the DRM systems and add
>> > missing
>> > capabilities as deemed necessary by the user community, so this is
an
>> > ambitious goal - in contrast to the request.
>>
>> Maybe this is out of scope for DRMAA. Even the three available file
>> transfer attributes are implemented in a different manner in Condor,
>> SGE and GridWay. Specifying a common file transfer API is also
>> already handled by other working groups.
>>
>> Peter.
>>
>>
>> >
>> > -Hrabri
>> > --
>> >   drmaa-wg mailing list
>> >   drmaa-wg at ogf.org
>> >   http://www.ogf.org/mailman/listinfo/drmaa-wg
>--
>"XML is like violence: if it does not help, use more."
>
>--
>  drmaa-wg mailing list
>  drmaa-wg at ogf.org
>  http://www.ogf.org/mailman/listinfo/drmaa-wg