[jsdl-wg] Questions and potential changes to JSDL, as seen from HPC Profile point-of-view

[Donal K. Fellows]

Marvin Theimer wrote:
> [MARVIN] The main thing I'm after is that the behavior of trying to step
> "outside" the mountpoint by cd'ing out the top must be either (a)
> prohibited or (b) explicitly marked as undefined in its behavior, with
> an error fault potentially being generated.  This is because in the
> Windows world I can imagine that a mountpoint definition might map to
> setting up a drive letter and you can't cd up out of a drive letter.

As I said, forbidding ".." from anywhere in *any* path in a HPC-profiled
JSDL document would be reasonable, and forbidding interoperable jobs
from doing a chdir() or equivalent at all is also reasonable (in both
cases, if people do those things then we can say that we don't define
what happens; it's up to vendors, with faulting being a nice strategy).
Also, using drive letter mappings for JSDL virtual paths on Windows is a
perfectly reasonable implementation strategy (but not something we
should mandate at either specification or profile level, of course).

Looks to me like we're in 100% agreement on this.

> [MARVIN] Yes, but these references don't require the BES service to
> parse the mountpoint and mountsource elements or do something like
> execute a stat operation on the mountpoint to confirm that it properly
> exists.  Excluding these elements from the HPC profile base case doesn't
> mean that an implicitly required mountpoint doesn't exist, merely that
> the compliant (BES) service is allowed to be oblivious to it.

The best way forward is probably to just define a small set of VFSes
that should/must be supported. That will scope the interop problem
nicely. I think (based on UNICORE experience) that the key locations for
a job are:

   Working Directory - main place for the job to work; may be isolated
        from other jobs (though that's really a quality-of-service
        feature that will be good for some jobs, bad for others, and
        neutral for the rest)

   Fast Local Temporary Directory - basically /tmp on Unix; need not be
        shared across a cluster; no isolation guarantees

   Large Temporary Directory - place for things like staged in BLAST
        databases, intermediate weather model dumps, etc. Should be
        shared across the cluster, should have a longer-term delete
        policy than /tmp if different from it. May be the same as FLTD;
        no isolation guarantees

   User's Home Directory - for application settings, definitely long term
        persistence (backups strongly recommended!) but might not permit
        very large files. Definitely possible to access outside the scope
        of the job. Need not be isolated from other jobs, and isolation
        from other users is dependent on user and site policy.

In theory, all could be the same directory, but that would be very odd.

All those FSes should have standardized names, and we should state that
profile-compliant jobs will not refer to any other filesystem. (And that
punts loads of complexity right out of the ballpark!)

(Any features of the above FS types I've missed anyone? Any preferred
names? I'm happy to call them John, Paul, George and Ringo, but that
might be confusing to others...)

> [MARVIN] I think the HPC profile should say that specifying ROOT will
> result in undefined behavior: the compliant service is free to accept
> the specification, ignore it, or raise a fault.  Similarly, if an
> activity is created then it must be prepared for undefined behavior if
> it tries to use ROOT in a file name that it opens.

Since I've not put ROOT in the basic set above, I'd argue that if the
job refers to ROOT, it's not in the space that should be defined by the
HPC profile. And so we don't need to solve the problem of how to handle
it on multi-rooted operating systems. :-)

Donal.