[ogsa-bes-wg] Questions and potential changes to BES, as seen from HPC Profile point-of-view

[Marvin Theimer]

Hi;

If we go the way you're suggesting then the HPC profile will need to
include BES, JSDL, and as-yet ill-defined specs for an information
service infrastructure and resource selection services.  That implies
that the HPC profile WG will end up defining a simple version of such as
a specification that is separate from everything else currently going on
in GGF.  That's OK by me, and in fact, my original email starting this
thread proposed the creation of a QueryResources operation that would
supply the requisite information.  I included it in my straw man
proposal for changes to BES, but would be happy to push it to a separate
interface (including the associated resource information model).

I agree that we should stay away from general constraint models (at
least for now) and certainly should stay away from having different
systems employ non-standard profile interpretations.  My inclination
would be to start with your suggest for a set of JSDL descriptions as
the way to expose execution systems such as clusters, either as a
replacement for the current BES information model or as a separate
QueryResources interface that BES services would export.

Marvin.

-----Original Message-----
From: Donal K. Fellows [mailto:donal.k.fellows at manchester.ac.uk] 
Sent: Tuesday, June 06, 2006 7:15 AM
To: Karl Czajkowski
Cc: Peter Lane; Marvin Theimer; ogsa-bes-wg at ggf.org; Ed Lassettre
Subject: Re: [ogsa-bes-wg] Questions and potential changes to BES, as
seen from HPC Profile point-of-view

Karl Czajkowski wrote:
> The main concern, I think, is that JSDL is "so close" to having the
> right terminology for both levels.  It would be a shame to lose focus
> and have to repeat the effort again later.  I do not think the
> heterogeneous solution is a simple "bag of JSDL documents"...

I think this is getting rapidly off-topic for BES. ;-)

We want to have at most a simple model for basic execution. Or even no
resource model at all. There's no particular reason for BES to publish
any information about the resources that it is in front of; that is the
responsibility of the information service infrastructure and (on the
consumption side) resource selection services. Requiring the use of a
published information model (let alone one as complex as you're
proposing) precludes many possible implementations for what are
effectively non-functional reasons.

And most (probably way over 99%) of the activity on the grid is, and
will continue to be, simple jobs/activities. Too much effort, not enough
return.

> I think a more general composition of the core concept, e.g. "memory",
> and a scoping construct, e.g. "per resource," "sum over resource
> array," or "sum over heterogeneous aggregate" would be better. I
> haven't thought about the coupled cases much, so I do not embarrass
> myself with short English phrases for them :-)

What about clusters that provide a shared memory (OpenMP) model; do we
have to introduce another modifier to deal with that? What will the term
mean when applied to the other resources? And the next extra thing that
wasn't originally thought of? What happens if a resource may be a member
of multiple heterogenous aggregates simultaneously, with some defined by
the user's VO?

Can we get away without opening this high-pressure can of worms? :-)

> The idea I am advocating is to have one general constraint model that
> is hopefully neutral, such that it can be mapped to the local
> constraint model of various schedulers or meta-schedulers.  No single
> service would necessarily implement the fully general model, but would
> rather support profiled subsets of this model. The benefit is that the
> different profiles are all related by a common model, and it is
> feasible to consider more general solvers/profiles being introduced
> over time.

I don't think it is a practical proposal for interoperability. Either
there's a common profile with no significant extensions to it or there's
a standard semantics for reconciling terms from one part of the
term-space with terms from another part. The first is pretty much
equivalent to ignoring your proposal (except with more matching
complexity! Yay!) and the second is extremely difficult. The other
alternative (that I do not want) is for there to be systems out there
that claim to support the same standards, but which profile them in such
different ways that interoperation is nigh-on impossible. That's where
we are right now, in effect.

[I was writing a much longer response to your message, but decided that
a total point-by-point rebuttal was foolish. And this is the wrong place
to make a fool of myself.]

Donal.