[DRMAA-WG] Torque/PBS DRMAA - file transfer routines

[Andre Merzky]

For what its worth, the SAGA group also stumbled over this
problem.  We finally opted to adopt a LSF like syntax,
following a suggestion from Chris Smith (no surpise, ey?  ;-).

That lsyntax ism in our opinion, generic enough to easily
map to the various backends.  The syntax basically is
(quoting the SAGA spec):

-----------------------------------------------------------------
  File Transfer Specifications
  ----------------------------
  
  The syntax of a file transfer directive for the job
  description is modeled on the LSF syntax {(LSF stands for
  Load Sharing Facility, a commercial job scheduler by
  Platform Computing), and has the general syntax:

    local_file operator remote_file

  Both the local_file and the remote_file can be URLs.  If
  they are not URLs, but full or relative pathnames, then
  the local_file is relative to the host where the
  submission is executed, and the remote_file is evaluated
  on the execution host of the job.

  The operator is one of the following four:

      '>'  copies the local file to the remote file
           before the job starts. Overwrites the remote 
           file if it exists.

      '>>' copies the local file to the remote file before
           the job starts. Appends to the remote file if
           it exists.

      '<'  copies the remote file to the local file after
           the job finishes. Overwrites the local file if
           it exists.

      '<<' copies the remote file to the local file after
           the job finishes. Appends to the local file if 
           it exists.
-----------------------------------------------------------------

That mechanism is also used for stdin/stdout/stderr.  I
don't want to say that DRMAA should use that - there are
certainly limitations - but might be worth to look at at
some point...

Cheers, Andre.


Quoting [Peter Troeger] (May 04 2007):
> From: Peter Troeger <peter.troeger at hpi.uni-potsdam.de>
> To: DRMAA Working Group <drmaa-wg at ogf.org>
> Subject: Re: [DRMAA-WG] Torque/PBS DRMAA - file transfer routines
> 
> > Issue:
> >
> > File transfer routines are desired by many users and currently they  
> > are
> > only limited to standard input, output and error streams.
> > ---------
> >
> > Omitting file transfer capability was done on purpose in version  
> > one to
> > put all the major DRM systems at the same footing.
> >
> > On of the ambitious DRMAA goals was to influence DRM system vendors or
> > providers to standardize (one more time) the DRM systems and add  
> > missing
> > capabilities as deemed necessary by the user community, so this is an
> > ambitious goal - in contrast to the request.
> 
> Maybe this is out of scope for DRMAA. Even the three available file  
> transfer attributes are implemented in a different manner in Condor,  
> SGE and GridWay. Specifying a common file transfer API is also  
> already handled by other working groups.
> 
> Peter.
> 
> 
> >
> > -Hrabri
> > --
> >   drmaa-wg mailing list
> >   drmaa-wg at ogf.org
> >   http://www.ogf.org/mailman/listinfo/drmaa-wg
-- 
"XML is like violence: if it does not help, use more."