[ogsa-bes-wg] Questions and potential changes to BES, as seen from HPC Profile point-of-view

[Marvin Theimer]

Hi;

I agree that exact matching on cpu and os isn't the most important
issue.  What IS important is being able to say "all the same": this is
really important for various kinds of MPI applications.

I suspect you'll see more specification of binary executable as time
goes on.  As people purchase multiple clusters with somewhat different
hardware and as they add new hardware to existing clusters they end up
with heterogeneous environments where the exact choice of hardware
really does matter.  For example, running numerical libraries that have
been optimized for Intel or AMD on the opposite chip type can yield
substantial performance penalties.

Marvin.

-----Original Message-----
From: owner-ogsa-bes-wg at ggf.org [mailto:owner-ogsa-bes-wg at ggf.org] On
Behalf Of Donal K. Fellows
Sent: Wednesday, June 07, 2006 1:24 AM
To: Peter Lane
Cc: ogsa-bes-wg at ggf.org; Ed Lassettre
Subject: Re: [ogsa-bes-wg] Questions and potential changes to BES, as
seen from HPC Profile point-of-view

Peter Lane wrote:
> Right, I meant something a little different when I mentioned JSDL.
JSDL 
> doesn't allow for describing complex requirements. For example,  one 
> might need a complex set of resources to run a distributed
application 
> on a cluster. The best JSDL can do, IIRC, is to allocate  N homogenous

> resources. There's no way you can say, for example,  "give me two IA64

> machines, two x86_64 machines, and two i386  machines". This is a very

> real requirement by users of GRAM.

Hmm, that's not something I've ever seen in our job flows. On the other
hand, we've found that our users aren't interested in specifying what
sort of processor they use at all. They focus on the application instead
and if that's part of the set described as supported by the container
(ignoring questions of how this is discovered for the moment) then it
hardly matters what the underlying CPU or OS is. (OK, both of those can
certainly make a difference to the relevant performance metrics, but
we'd rather state that we support a certain level of performance instead
of providing some information that people can use to try to infer what
they're really interested in.)

The only real use for matching on CPU and OS is if you're staging in the
binary executable to run as part of the job. Physicists seem to like to
do that; seems to be a peculiarity of that community. We don't see the
same thing to anything like as great an extent among other scientists
and engineers (and we have very little CS in our usage profile, though
if we did I'd expect them to be similar to the physicists). On the other
hand, the physicists seem to prefer to buy their own clusters too.

Oh, you're supporting physicists? :-)

Donal.