[ogsa-bes-wg] Questions and potential changes to BES, as seen from HPC Profile point-of-view

[Marvin Theimer]

Hi;

I agree with Chris' characterization of the most common case.  I think
the JSDL community will eventually want to support additional elements
that allow specification of the sorts of things that have come up in
this email thread -- especially since there is no standard,
interoperable definition of what queues should be supported in a larger
system that spans multiple job schedulers and multiple compute clusters.
Luckily, it seems (to me) that most of these extensions can be added in
a very localized manner that is transparent to most of the grid
infrastructure.

I do believe that the HPC profile work will require a richer resource
description capability for describing available resources than the
current one in BES.  I think Donal's suggestion of an array of jsdl
documents seems like a good starting point for discussion of such a
topic.

Marvin.

-----Original Message-----
From: owner-ogsa-bes-wg at ggf.org [mailto:owner-ogsa-bes-wg at ggf.org] On
Behalf Of Christopher Smith
Sent: Wednesday, June 07, 2006 10:44 AM
To: Donal K. Fellows; Peter Lane
Cc: ogsa-bes-wg at ggf.org; Ed Lassettre
Subject: Re: [ogsa-bes-wg] Questions and potential changes to BES, as
seen from HPC Profile point-of-view

Much like Donal, I don't see this requirement overmuch in our user base
(squarely in the "HPC Profile" space). Mostly users are submitting jobs
that
can either utilize any architecture in their environment (the admins
help
"make this so"), or they work within a homogeneous set (x86_64 running
Linux), so the JSDL document is good enough for a lot of these cases.
They'll also use queues and "host groups" to makes these kinds of
selections
for them. Even the parallel jobs are almost always constrained such that
all
tasks are running on a single architecture.

I realize that JSDL has some limitations in describing resources at this
time. I was also under the impression that some work was being done to
define some kind of information model based on generic name/value pairs
(where some names could be standardized by convention), but so that you
could represent any type of attribute. This would go along with some
kind of
grammar to put stuff together into expressions that could be evaluated
for
the purposes of resource requirement evaluation. I'm willing to live
with
the JSDL v1.0 limitations, given that there will be some changes in the
near
future if we work at it.

-- Chris


On 07/6/06 01:23, "Donal K. Fellows" <donal.k.fellows at manchester.ac.uk>
wrote:

> Peter Lane wrote:
>> Right, I meant something a little different when I mentioned JSDL.
JSDL
>> doesn't allow for describing complex requirements. For example,  one
>> might need a complex set of resources to run a distributed
application
>> on a cluster. The best JSDL can do, IIRC, is to allocate  N
homogenous
>> resources. There's no way you can say, for example,  "give me two
IA64
>> machines, two x86_64 machines, and two i386  machines". This is a
very
>> real requirement by users of GRAM.
> 
> Hmm, that's not something I've ever seen in our job flows. On the
other
> hand, we've found that our users aren't interested in specifying what
> sort of processor they use at all. They focus on the application
instead
> and if that's part of the set described as supported by the container
> (ignoring questions of how this is discovered for the moment) then it
> hardly matters what the underlying CPU or OS is. (OK, both of those
can
> certainly make a difference to the relevant performance metrics, but
> we'd rather state that we support a certain level of performance
instead
> of providing some information that people can use to try to infer what
> they're really interested in.)
> 
> The only real use for matching on CPU and OS is if you're staging in
the
> binary executable to run as part of the job. Physicists seem to like
to
> do that; seems to be a peculiarity of that community. We don't see the
> same thing to anything like as great an extent among other scientists
> and engineers (and we have very little CS in our usage profile, though
> if we did I'd expect them to be similar to the physicists). On the
other
> hand, the physicists seem to prefer to buy their own clusters too.
> 
> Oh, you're supporting physicists? :-)
> 
> Donal.
>