[jsdl-wg] Coupling PosixApplication to DataStaging

[Donal K. Fellows]

Karl Czajkowski wrote:
> This seems as reasonable as the other magical approaches.  It only
> prevents arbitrary string splicing like we have in GRAM... we use a
> ${VARIABLE} substitution rule in string fields like args and
> environment settings.

Yeah, that's what ant and a few other tools do too. I seem to recall
hearing/reading that XML hackers hate it as an idiom. :^)

> The only thing more general might be some attribute to enable such
> a rewrite rule on a particular string element field?  E.g. by the
> ever faithful URI or QName "extensible enumeration type" to indicate
> how such a filesystem string name is merged into the target element.

In general that's a good idea, but it's much more complex and we can do
90% of what we need with only 10% of the effort. As a bonus, my scheme
also lets us avoid having to define what a directory separator looks
like, at least for basic use. :^)

> Could this be handled using a more open content model?  I am still not
> sure I understand the use case, e.g. who defines the virtual labels
> and their meaning.  Could they ever be created dynamically by some
> external negotiation or stateful process, meaning the label really
> only means something to the client and for the provider it is just an
> arbitrary space in a pool?

The virtual labels are a separate thing that we need to pin down,
especially as they're needed elsewhere too. I suggest that we probably
want a basic set like these (of course, users and jsdl consumers can
create their own):

   HOME - Executing user's home directory

   TEMP (or TMP) - Temporary directory not guaranteed to last past the
      end of staging out, but relatively fast. Might not be shared across
      a cluster.

   SCRATCH - Temporary directory that can hold "large" temporary files
      with longer lifespan than TEMP where it is distinct (I know of a
      few systems where these are quite separate, and where it is shared
      cluster-wide.)

I suppose we could add the job working directory to that list, but if
anyone is doing cunning stuff there they should probably do it by hand.

Creating new kinds of these things through negotiation or other stateful
processing? Sounds fine to me.

Donal.