[ogsa-wg] Re: [ogsa-rss-wg] Re:[RSS Architecture Discussion]

[Karl Czajkowski]

On Dec 14, Soonwook Hwang modulated:
...
> I am also wondering how these different job types will affect the design 
> of EPS/CSG interface and protocol. The way to describe resource 
> requirements for job execution may be different depending on the type
> of jobs. Is the resource element part of JSDL v1 spec sufficient to be 
> used to describe requirements of jobs of different types?  
> I suspect that depending on job types, the parameters used for ranking 
> function/mechamism is likely to be different, and that we might end up 
> having different EPS/CSG interfaces for each job type. 
> 

I think you could express an MPI job in JSDL v1 resource language, and
have a co-allocation aware selection service return different possible
mappings where some of the homogeneous resources come from different
managers.  E.g. a "simple" JSDL job is exploded into a set of
concurrent JSDL jobs to be directed to different execution sites.

However, I think you would quickly want a more rich resource language
extension---beyond JSDL v1---where you could describe network
connectivity requirements (possibly hierarchically) and heterogeneous
resource requirements.  For example, describing a job where you need a
certain number of "large memory" nodes with a good interconnect, a
certain number of "fast CPU" nodes with a separate good interconnect,
and a less stringent WAN connection between the two node sets for some
coupled numerical code.  This is definitely beyond the scope of the
JSDL v1 resource language, though a nice extended language could
encapsulate multiple v1 resource clauses, one for each homogeneous
node set...

I think the big debates here will always return to which way such
natural job/resource graphs will be normalized into a document tree.
Should there be one job with complex resource requirements?  Or some
complex workflow with lots of simple (sub-)job documents?

In the general case, there can be lots of cross-cutting relationships,
e.g. correlating different executable specifications or data-staging
requirements with different node types (or node identities), having a
mixture of localized per-node, per-nodeset, and global resource
allocation requirements, etc.  Different normalized document trees
will emphasize one job regime over another, making that one convenient
to specify using nice, lexically scoped properties while either
blocking other kinds of job or at least making them use less
convenient cross-references and such to encode the natural graph.

Of course, the debates are not just about syntactic style, but assumed
document processing algorithms.  Out of necessity, people are
implementing restricted hueristics; out of convenience, I think they
are also relying on document syntax to help enforce representation
invariants for their implemented hueristics.  I haven't seen anybody
favoring a unified "big mess blob in/big mess blob out" interface
where all the implementation-specific constraints must be checked
internally while mapping to some more prosaic internal format.  I have
often wondered, and am not sure this would lead to better or worse
interop than a bunch of more specialized interface standards...


karl

-- 
Karl Czajkowski
karlcz at univa.com