[Pgi-wg] OGF PGI Requirements - Flexibility and clarity versus Rigidity and confusion

[Andrew Grimshaw]

Aleksandr,
Referring to your sentence/paragraph

"Such "simple" job is very far from being "typical". At least in NorduGrid
world AFAIK."

Could you elaborate. I see in my work basically two "types" of jobs that
dominate - sets of HTC "parameter space" jobs, and true parallel MPI jobs.
In both cases the "job" is basically a command line - either an mpiexec/run,
and application with parameters, or a script with parameters. The job has
known inputs and outputs, or a directory tree location where it needs to
run. The jobs runs to completion, or it fails, in either case there are
output files and result codes. Sometimes the job is a workflow, but when you
pick that apart it turns into jobs that have inputs and outputs along with a
workflow engine orchestrating it all.

What is a typical job that you see? When I say "typical" I mean covers 80%
of the jobs.

A

-----Original Message-----
From: Aleksandr Konstantinov [mailto:aleksandr.konstantinov at fys.uio.no] 
Sent: Sunday, May 02, 2010 3:36 PM
To: pgi-wg at ogf.org
Cc: Andrew Grimshaw; 'Oxana Smirnova'; 'Etienne URBAH'; 'David SNELLING';
lodygens at lal.in2p3.fr
Subject: Re: [Pgi-wg] OGF PGI Requirements - Flexibility and clarity versus
Rigidity and confusion

Hello,

I agree that problem is to difficult to solve. One should take into 
account that initially task was different. Originally AFAIR it was
an attempt of few grid project to make a common interface suitable
for them. Later those were forced to OGF and problem upgraded
to almost unsolvable.

Andrew Grimshaw at Saturday 01 May 2010 15:36 wrote:
> Oxana,
> Well said.
>
> I would add that I fear we may be trying too much to solve all problems
the
> first time around - to "boil the ocean". To completely solve the whole
> problem is a daunting task indeed as there are so many issues.
>
> I personally believe we will make more progress if we solve the minimum
> problem first, e.g., securely run a simple job from
infrastructure/sw-stack
> A on infrastructure/sw-stack B.

This problem is already solved. And it was done in few ways. 
1. Client stacks supporting multiple service stacks
2. BES + GSI
3. Other combinations currently in use
And none is fully suitable for real production. So unless task of PGI 
is considered to be purely theoretical this approach would become 
equal to one more delay.

>
> "Infrastructure/sw-stack A" means a set of resources (e.g., true
> parallel-Jugene, clusters, sets of desktops) running a middleware stack
> (e.g., Unicore 6 or Arc) configured a particular way. In the European
> context this might mean an NGI such as D-Grid with Unicore 6 running a job
> on NorduGrid running Arc. (Please forgive me if I have the particulars of
> the NGIs wrong.)
>
> "Simple job" means a job that is typical, not special. This is not to say
> that its resource requirements are simple, it may have very particular
> requirements (cores per socket, interconnect, memory), rather I mean that
> the job processing required is simple: run w/o staging, simple staging,

Such "simple" job is very far from being "typical". At least in NorduGrid
world AFAIK.

> perhaps client interaction with the session directory pre, post, and
during
> execution.
> Try to avoid complex job state models that will be hard to agree
> on, and difficult to implement in some environments.
>
> "Securely" means sufficient authentication information required at B is
> provided to B in a form it will accept from a policy perspective. Further,
> that we try as much as possible to avoid a delegation definition that
> extends inwards beyond the outer boundary of a particular

I'm lost. Is is delegation or definition which extends?

> infrastructure/sw-stack. (The last sentence is a bit awkward, I personally
> think that we will need to have two models of authentication and
delegation
> - a legacy transport layer mechanism, and a message layer mechanism based
> on SAML, and that inside of a software stack we cannot expect sw-stacks to
> change their internal delegation mechanism.)
>
> I believe authentication/delegation is the most critical item: if we
cannot
> get the authentication/delegation issues solved, the rest is moot with
> respect to a PRODUCTION environment. We may be able to do demo's and
stunts
> while punting on authentication/delegation, but we will not integrate
> production systems.)

Wasn't delegation voted no during last review?


A.K.