[jsdl-wg] JSDL: Updates

[Donal K. Fellows]

Ali Anjomshoaa wrote:
> Remember, this week's the big push. Next week we're closing down the
> issues for v1.0. PLEASE ALL READ THE SPEC. Many thanks in advance.

What follows is a discussion of agenda issues 11, 12 and 14. If anyone
has any problems with my suggestion, drop me a line ASAP. Otherwise I'll
write it up into a set of modifications to the spec later this week.

The major issues left (that I'm at all involved with, that is) are to do
with the coupling of filesystems to the components of POSIXApplication.
We need to get this:
   a) done (without a mechanism, we can't connect the resource
      information relating to the filesystem without hard-coding the
      location of files, and that forces jsdl documents to be
      non-relocatable)
   b) correct (well duh!)
   c) comprehensible (if we come up with too complicated a solution,
      people are going to have trouble implementing it and that will hurt
      adoption)

The following sub-elements of POSIXApplication need to be optionally
related to a (virtual) filesystem:

   Executable
   Argument
   Input
   Output
   Error
   WorkingDirectory
   Environment

The relation of an element to a filesystem should be done by
interpreting the element's contents as a filename relative to the root
of the filesystem (which amounts to a simple prefixing operation). The
interpretation should apply exactly when a filesystemName attribute is
present on the element. When no filesystem is supplied, the element
should not be interpreted as relative to anything at the JSDL processing
level (leaving the underlying system free to interpret it however it
wants; e.g. looking up on the PATH if it is an Executable, passing
directly to the program if it is an Argument or Environment,
interpreting as a file/dir relative to the current WD for the others).

[EXAMPLE]
I could specify the location of my mailbox to be relative to my home
directory like this:

   <jsdl-posix:Environment name="MAIL" filesystemName="HOME">
       mailboxes/INBOX
   </jsdl-posix:Environment>

This would (on one particular system) set the MAIL environment variable
to /home/donal/mailboxes/INBOX
[END EXAMPLE]

As a side effect, it becomes possible to drop the clause from the
FileSystem/MountPoint element that talks about environment variables
because I could do this to set the SCRATCH environment variable to the
root of the SCRATCH filesystem:

   <jsdl-posix:Environment name="SCRATCH" filesystemName="SCRATCH"/>

All that remains to be done is define a default set of filesystems. Each
of these, unless the user uses an explicit FileSystem element, should be
declared like this:

   <FileSystem name="HOME">
     <Description>
       The user's home directory.
     </Description>
     <FileSystemType> normal </FileSystemType>
   </FileSystem>

   <FileSystem name="ROOT">
     <Description>
       The root of the main system filesystem. Not safe to assume that it
       is writeable.
     </Description>
     <FileSystemType> normal </FileSystemType>
   </FileSystem>

   <FileSystem name="SCRATCH">
     <Description>
       Temporary space that persists some time after the job completes
       and which is capable of taking relatively large files.
     </Description>
     <FileSystemType> spool </FileSystemType>
   </FileSystem>

   <FileSystem name="TMP">
     <Description>
       Temporary space that does not necessarily persist after the job
       terminates and which might not be shared between resources, but
       which will be fast.
     </Description>
     <FileSystemType> tmp </FileSystemType>
   </FileSystem>

Notes: None of these have any space, mount-point or mount-source
constraints. We're just establishing the general existence of the
filesystems. They may in practice be on the same physical filesystem
(and I would actually expect that to be the case with many uses of
SCRATCH and TMP, of course, though I have a requirement for having them
separate so that I can model our supercomputer cluster configuration).
The actual consuming systems should in-fact have defined locations for
these and reject a document if it demands that they be a value which
clashes.

If people prefer SPOOL to SCRATCH as a name, that's cool with me. :^)

The names HOME, ROOT, SCRATCH, TMP do not pollute the environment
variable space in my scheme (though HOME and TMP are quite possibly set
already). They have to be explicitly coupled and the only space that is
actually polluted is the space of filesystem names, which is a concept
that is internal to JSDL.

In schema terms, we'd just be adding a few attributes to a few elements.

(Question: should the "tmp" filesystem type be "temporary"?)
(Spec Bug: FileSystem examples do not match pseudoschema; names are
NCName not QName or anyURI).
(Question: should DataStaging/FileSystemName be QName or NCName?)

Donal.