[ogsa-bes-wg] Re: [jsdl-wg] Questions and potential changes to JSDL, as seen from HPC Profile point-of-view

[Andre Merzky]

Hi, 

Quoting [Donal K. Fellows] (Jun 12 2006):
> 
> Marvin Theimer wrote:
> >If we narrow the definitions of mountpoint and mountsource enough and 
> >precisely describe their semantics then we might arrive at something 
> >that could be fairly widely used.  I???m thinking of things like saying 
> >that you can???t navigate ???out??? of a file system via ???cd ..???, etc. 
> >This is definitely something to explore. 
> 
> Change "can't" to "shouldn't" and I'd agree. I don't regard the mount
> stuff as being a way of describing security enforcement points. Systems
> can do it that way, but at least some won't.
> 
> In fact, I'd be happy enough with the profile stating that paths in JSDL
> documents should not contain either the "." or the ".." elements at all.
> That's a fairly strong requirement and guarantees that the job won't
> fail on systems where your style of semantics are enforced.


The GFS group (naturally) did some analysis on file systems,
directory navigation etc, and on related issues in Grids.
It might be useful input, and prevent you from re-doing that
job.  

You might want to browse their meeting material section on
forge
(https://forge.gridforum.org/sf/docman/do/listDocuments/projects.gfs-wg/docman.root.meeting_materials) for some overview material.

Hope that helps, 

  Andre.


> 
> >Since the HPC profile base case treats data staging as being 
> >out-of-scope, the base interface profile will exclude these; but that 
> >can be done independently of anything else.  (And, of course, the data 
> >staging extension to the HPC profile will need to deal with this subject 
> >in any case, even if it???s ignored in the base case.)
> 
> Perhaps. Virtual FS references also sneak into arguments, stdio
> redirection, working directory specs, and environment variables, all of
> which may well need to be defined with respect to some VFS root. One way
> of dealing with this is to define a minimal set of VFSes in the profile
> (probably based on the JSDL set?) and state that profile-conforming job
> submissions will only use those.
> 
> >The HPC profile (and BES) /have/ to deal with the issue of describing 
> >available resources.  So, one way or the other, the subject will get 
> >addressed this summer.  As much as possible, I???d like to avoid 
> >duplicating the work done in JSDL for that ??? if for no other reason than 
> >that users will likely be unhappy if they have to learn two different 
> >ways of describing what they will perceive as being variations of the 
> >core concept, namely resource description ??? both required and available.
> 
> Sure, but you're using the elements to have subtly different meanings.
> Not that I'm objecting; just pointing it out. :-) The best bet I'd guess
> is to state that the published elements represent a description of the
> maximal job (maximal capacity, most specific capability) supported by
> the container.
> 
> You'll also need to manufacture a way of describing the set of apps (and
> other related things, like libraries) supported. I don't think that the
> way JSDL does this (even under the "maximal" interpretation outlined
> above) is going to work in a sane-enough way.
> 
> >Any advice on this subject would be greatly appreciated.  As I said 
> >above, I have to deal with this subject one way or the other and would 
> >prefer to do so with the minimum of feather-ruffling (while still making 
> >progress that results in a usable HPC profile by the end of the summer).
> 
> You've got a mandate to ruffle feathers. You're writing a profile. JSDL
> isn't a profile, and so we (with my JSDL hat on) need to be a little bit
> more circumspect, if only to keep the noise down. :-)
> 
> >If you allow for the notion of file systems and mount points to be in 
> >the core spec then I would argue that you are implicitly buying into 
> >systems that also support the notion of current working directory (some 
> >jobs may of course not use it).
> 
> Good argument. :-)
> 
> >I would argue that one not specifying allowed/disallowed behaviors is a 
> >bad approach when interoperability is at issue.  (I???m talking about 
> >disallowing ???cd ..??? out the top, not disallowing change-directory 
> >within the subtree specified by a file system element.)
> 
> I'd certainly say that it is a feature nobody has to implement, and
> which no interoperable job may make use of. (I don't see any way of
> preventing the job itself from trying to do a 'cd ..' internally, nor
> even any way to properly analyse whether the job will try to do it. Any
> security checks required by a VFS interpretation must still be enforced
> at runtime anyway.)
> 
> [re root filesystems]
> >Again, from an interop point-of-view, this seems dangerous.
> 
> This seems to me like an argument in favour of not including ROOT in the
> HPC profile. That doesn't cause me problems, though I suspect that
> everyone building a BES on top of unix will add it. (Who knows, perhaps
> it would be in a POSIX-HPC sub-profile?)
> 
> >Regarding your reactions to my straw man proposal, it seems like you 
> >pretty much agree with everything except the following:
> >
> >·        You???re not convinced how universal the posix extension elements 
> >for things like command-line arguments and working directory are.  My 
> >response is that I think they are at least as universal as the data 
> >staging elements.
> 
> I'm likely agree with you there.
> 
> >·        You don???t want to move the data staging section out of the core 
> >specification.  For the HPC profile base case, data staging elements 
> >will be prohibited since they are out-of-scope for the base case.  The 
> >HPC profile extension for data staging will allow the JSDL data staging 
> >elements.  Whether or not these should be in a separate JSDL extension 
> >or whether they can be generalized to cover a wide(r) range of systems 
> >is a topic for future discussion.
> 
> Sounds fair enough. The inclusion of data staging was so that JSDL would
> be relevant in cases that don't really match the HPC case, such as
> clusters where everything has to be staged into the machines for
> anything at all to work, and where no real workflow coordinator was
> available. (Myself, I prefer the data staging to be in a separate
> workflow step, but workflows are not really in scope of any GGF working
> group at the moment.) IIRC, it was Darren Pulsipher who had this use case.
> 
> >·        You???re leery of tackling the resource description problem.  
> >Understandable, although the HPC profile working group will have to and 
> >will be seeking guidance from the JSDL and other communities on how to 
> >do so.
> 
> I think it's hard and some people have very entrenched views on this.
> But I also think that you can get something workable for the 90% case
> together in short order here. I suggest that the "homogenous container" 
> model under the maximal resource interpretation is good enough. :-)
> 
> >Is that a fair characterization of your position?
> 
> Pretty fair. I can't speak for whether it is a characterisation of
> anyone else's position. :-)
> 
> Donal.



-- 
"So much time, so little to do..."  -- Garfield