[Pgi-wg] Notes from PGI f2f Workshop in Amsterdam
Johannes Watzl
watzl at nm.ifi.lmu.de
Mon May 3 08:51:35 CDT 2010
PGI f2f Workshop Amsterdam, 29-30 April 2010
Meeting notes
participants in Amsterdam:
Morris, Shabhaz, Balazs, Etienne, Luigi, Johannes
on the call:
Steve Crouch, Tim Parkinson, Andrew, Mark, Emmanouil, Oxana, David
Wallom, David Snelling
Start: 9:55
list in wiki with ~210 requirements
condensed list created over the last days in gridForge
dropped items marked as "dropped" in details
questions?
Andrew: should use "raise hand" functionality
Andrew: any correlation between numbers/ids of old and new version
Etienne:
numbers of spreadsheet
Morris:
screensharing - raise hand
start at the beginning of the list
Morris:
look at the process
review
David S:
progress on refining requirements
next steps, outcome of this workshop: agreement
mutual understanding on requirments
Andrew:
some requirmenents got dropped, want to bring some of them back
Morris:
outcome: agreed list of requirements in scope of PGI
Balazs:
if we want to make decisions - want to see the process of decision making
Morris:
should talk about
David S:
recommend to quickly go through the requirements
general agreement
Balazs:
one objection is enough to do it in the second round
Morris:
comments to the process?
Dave S:
everybody has to understand it first and then quickly accept or reject it
once you have a requirement what to do next
what is the next step in the process?
Morris:
everybody can come up with a strawman
Balasz:
forward the list to BES, JSDL,... groups
David S:
call for suggestions?
Andrew:
need some criteria to select
David S:
how long do people have to put together strawman proposals
Johannes:
collect by OGF29
select at OGF30
Morris:
problems with this timeline?
David W:
should work on strawmen at OGF29
Morris:
don't have to be completed totally but in a shape to be able to discuss them
Andrew:
more important to get good starter strawmen from different directions
Dave S:
by end of OGF29 you need to have chosen one
Morris:
would be tough
David:
go through the process
on the screensharing
Morris:
(1)
yes - no triage quick
list of condensed list pdf - re-proposed ha
everybody agrees to say yes and keep them
quickly
(2)
when we have the requirements ...
any strawman that answers as many requirements as possible
might be numerous ones
(3)
Strawman collected At OGF29 jUne 20th
Decided on
(4)
Select strawman by OGF30
David S:
hos does this fit with other pojects?
Morris:
strawman from european perspective
October is a little bit late but ok
David S:
strawman decided by beginning of August
make a decision before OGF30
Morris:
mabe no collection at OGF29
another f2f in August to support the decision process
to be able to present the decision at OGF30
what about others in the group?
US?
Andrew:
busy til mid of July
continue the work in GIN
timeline works
vacation time may be a problem
Morris:
then just two weeks to decide for you
OGF29 for discussions of strawmen drafts
do you think two weeks are enough?
Andrew:
negotiation would take more than two weeks
people can come to positions in two weeks
Morris:
aim: save time
Balazs:
no decision can be made before august, unrealistic to have a decision
before august
Morris:
end of August deadline?
David W:
indroducing delays
global effort!
keep 31 July
David S:
if it is important enough, people will increase the pace
Morris:
increase the pace now
or
accept the delays
David S:
decision mid of July solve problems with european holidays
Morris:
problem in the US
Andrew:
we will find a way
Morris:
instead of 31 August -> mid of July
Luigi:
EC has an explicit requirement
we need an agreement, specification, cannot wait too much time
we need it before August
project starts 1 May
Morris:
Luigi pointed out even mid of July too late
Luigi:
we have to finalize the list and then go on
Balazs:
commitment
need 2 months of dedicated work for PGI
Dave S:
decision as early as possible!
mid of July seems possible
right track
push mid July decision
focus on requirements
Balazs:
dedicated effort needed
Luigi:
roadmap: we will have strawman
Morris:
how to note down the process?
proposal: Etienne note down "yes" or "no"
Etienne:
would like to trace the information in the wiki
Balazs:
desaster, too slow!
google document
David W:
in this session go through requirments which were dropped
if anybody has concerns or wants discuss dropped requ.
in the next session go through
two groups on the call who have a number of dropped requirements they
want to have discussed
Morris:
we will lose a lot of time
follow Dave S. suggestion to go quickly through the list
David W:
ok
Balazs:
take google doc and use DFN screensharing
David W:
so let's start
Morris:
Everybody ready?
Andrew:
yes
David:
yes
Emannouil:
yes
starting with going through the list.
Morris:
when we are done with quickly going through, we do the second run
mark every requirement with yes, no, or unclear
Requirement 13,14,15,16
everybody agrees on message level
17: no
18: same as 12
19:
David W: definition of good practice, not really a requirement
Andrew: maybe no
authorization is out of scope
Etienne:
in scope of GLUE model
when client submits job it is accepted or not by policy
24:
David W:
restriction on must?
Andrew:
if you are going to require you should take one at least
at some point in security looking at production grids
maybe this is a no
25:
Oxana: mutual exclusive with 22
Andrew:
25 is too strong - should say no to 25
26:
Luigi: no
Andrew: for the US a hard requirement
Shibboleth may be the answer
Etienne:
understand from user point of view but not from service point of view
27:
Andrew:
against, because it is telling how to implement things
Etienne:
it is a no
28:
Andrew:
method interpreted as mechanism?
-> yes
Oxana:
what is the use of this requirement
29:
David W:
reject
32:
must?
Andrew:
out of scope
Oxana:
authorized means authorization is granted
37:
Etienne:
"compute" is useless here
Andrew:
strong requirement from user community to have logs
independant of accounting and security
41:
Andrew:
should be a no
not a requirement
45:
Andrew:
unclear?
48:
Andrew:
no
David:
when it is moved it becomes new activity
----
going through rejected
Andrew: NF13 as new requirement 160
also requirement from Etienne and Luigi
Etienne has a list with reproposed requirements
everybody else agrees with the rejected requirements.
from Etienne:
new requirement 161
NF6 - 162
David W:
specific
IS2 - 163
IS8 - 164
Steve:
what is the difference between this and IS3
IS9 - 165
IS13 - 166
AC0 - 167
Andrew: non scalable requirement
added 168, 169, 170, 171
setting up dimdim screensharing
try first 5 unclear ones
going through unclear requirements
3
Etienne: vector limits
we must have one separate endpoint for each group of operations which
have the same vector limits
Andrew:
we haven't defined what an endpoint is
why do we have to split them to different endpoints
change text
Balazs:
in the inhomogenous case more endpoints
Andrew:
if we send JSDL document, leave it to BES to handle it
David W:
maybe will become impossible to manage
change to *no*
###
5:
change text
Balazs:
in GLUE a model cannot do anything with storage
Etienne:
there is a global pointer to storage
the execution service must only provide details about computing entities
and no details about storage entities
change to *yes*
###
8:
Morris:
scalability issue
Etienne:
better description form the wiki
changed to *yes*
###
11:
rewritten but after timeout still unclear
###
13:
Mark:
don't seem like a requirement
Morris:
they are options
Luigi:
there is written "may"
Morris:
looking at 13-16, they are all the same thing
Balazs:
suggestion to have a dedicated security session
we need to have a global agreement of authentication/authorization in PGI
Morris:
we need a specific security working task
###
18:
Andrew:
believe should not be a requirement
Morris:
example?
Tim:
many institutions will not allow non institution members to access
Etienne:
if there is an anonymous request: no authorization
Andrew:
by saying it must require security credentials
will allow anybody to use a machine
certain policies will not be allowed
clear, changed to *no*
###
19:
Etienne:
for authorization someone can send package
I am manager of certain VO user in other VO and has special account on a
certain Grid
A man in the middle should not be able to get one of these
David:
about how the authorization is actually handled
Etienne:
you can put several SAML assertion into one single header
Andrew:
with SAML
all you know is some attributes from somebody you trust
different words to describe it
Oxana:
how can this prevented by packaging
Etienne:
solution: global signature
Morris:
we all should design it a way that we all can say yes
changed to *yes*
###
26:
Andrew:
requirement from funding agency
we need to support delegation and trust federation
Morris:
single sign-on
Etienne:
not understanding the last sentence
Andrew:
incommon is one of the providers in the united states for research and
government
agreement on SAML document
Morris:
Move to Shibboleth in Europe too
change to *no*
###
29:
change to *yes*
###
31:
Luigi:
whenever a user may need to manage a lifecycle of an activity
when certificate expired activity is terminated
Balazs:
on the service you have credentials somewhere
Luigi:
you associate activity with credentials
rephrasing
change to *yes*
###
32:
Luigi:
we need a mechanism to authorize a user to manage an activity from
another user
Etienne:
better explanation on the wiki
Andrew:
just add "must exist a mechanism"
concerned with focus on GLUE entities
information model
does not concretely say anything
change to *yes*
close of day one.
participants online: Tim, Steve, Mark Andrew, Emannouil, Aleksandr,
Oxana, David
participants amsterdam: Balazs, Shabhaz, Morris, Johannes, Luigi, Etienne
go through unclear requirements
10 min per requirement
stop 5pm
34:
Andrew: 34 and 35 coupled
Etienne:
details on wiki page
Andrew:
take them out
Steve:
can run a grid without this
Etienne:
in desktop grids is mandatory
Balazs:
JSDL hook
assume the execution service is able to deal with it
Morris:
maybe have it without allication repositories
changed text
change to *yes*
###
35:
Andrew:
merge 34 and 35
Etienne:
global namespace separate
Morris, Balazs:
example
change to *yes* and duplicate to 34
###
36:
Andrew:
TeraGrid uses UR for accounting
is it is necessary to extend UR to network
Etienne:
compare to 167
Morris:
but we understand it?
Balazs:
profiling on usage record
changing text
Etienne:
accounting records for activities
change to *yes*
###
37:
Andrew:
sequence of things that happend to job, tracking
Luigi:
maybe more generic
Etienne:
local system logs
Balazs:
activity level?
what do you want to log?
Etienne:
changes of job states
Andrew:
fault codes
Morris:
security audits, user job log
change to *yes*
###
38:
Etienne:
proposal to mark as out of scope
changing text
Luigi:
request operation exposing logs on the portType level
Andrew:
some service
change to *no*
###
39:
duplicate of 38
David W:
change the area to accounting and logging
Etienne:
clearly not compute and not accounting
change to *yes*
###
40:
Etienne:
change to accounting and logging
Balazs:
why different to 39?
Etienene:
here: should support complex query languages
Balazs:
different "query" in 39 and 40
change text
at least one complex query language
change to *no*
###
48:
Andrew:
you need to be able to access the information; done over the execution
service?
Balazs:
also managing the job?
Andrew:
yes
change to *no*
###
49:
Andrew:
endpoint
URI or epr
why 49 is a requirement
Etienne:
the way how clients submit request is part of the interface
still unclear after 10 min discussion
an illustration will help
###
50:
Morris:
what is activity id? different understanding
Andrew:
suppose activity id is an epr
should be able to extract the address field from it
Aleksandr:
you cannot contact epr, only the service represented by epr
change text
David W:
what is the reason for this requirement?
makes it a lot harder for the group
Luigi:
consider activity id as an epr - flexible structure
Oxana:
more than one requirement in this text
checking wiki page with original text
still unclear after 10 min
###
51:
change text
Andrew:
if we are operating in a WS world and strictly going with opaque ids;
you need a name service to bind
noticeable slower
Oxana:
last sencence must be removed: it is exactly the same as in requ 50
Andrew:
we understand the requirements, discuss later
Etienne:
how can any client contact any server
Morris:
clarify this but not now, maybe with illustration
change 50 and 51 to *no*
###
52:
change to *no*
###
56:
Andrew:
clear to me, but reword it
one way to achieve are vector operations
will vote no
change to *no*
###
57:
Morris:
bulk operation limit
change to *no*
###
58:
Balazs:
clients should not assume that esxecution service comes with this
frunctionality
Morris:
we have a requirement that defines something out of scope
change text
Mark:
pretty much everybody should vote no either in or out of scope
rephrase the requirement
change to *no*
###
62:
change to *yes* duplicate of 37
###
63:
change to *no*
###
64:
change to *no*
###
69:
Mark:
vote no, activity ids not defined
Balazs:
we don't know details of activity id - general object
Etienne:
session directory embedded in activity id?
change to *no*
###
72:
Mark:
difference between state and statuas information?
Etienne, Balazs:
here the same
Etienne:
replace multiple by extended
Balazs:
different schemas
for backwards compatibility
change text
Etienne:
Must support different models simultanously
Johannes:
you can just have the PGI model
Morris, Balazs:
yes, so MAY
Mark:
status information about activities
change to *no*
###
73:
David W:
GLUE model instead of GLUE2
Morris, Etienne:
wiki: LB6
example there
Shabhaz:
here query language, not in BES
Morris:
BES little bit extended with filters
change to *no*
###
78:
Morris:
lot of things related to this
David:
pending state
it's an optional substate? should be
Balazs:
pending state coupled to the batch system
Etienne:
accepted by the execution service
Luigi:
delegated state
mapping between pending and delegated
Morris:
the mechanism must be there
change to *no*
###
84:
Luigi:
want to do management
change to *no*
###
85:
change to *no*
###
87:
Morris:
lease
submitted it, never herad of it, do it once?
Luigi:
you create the lease instance, negotiate the time to live
define maximum lease time
it is a timer
David:
don't understand
Luigi, Balazs:
avoid zombie jobs
Morris:
can't get rid of a job
Balazs:
cleanup over the zombie hjobs
Luigi:
network prroblems
David:
simple redescription
Morris:
rephrase final state to any state
David:
clearer but a no
change to *no*
###
88, 89:
postponed
###
90:
Tim:
"will be" is this expressing a requirement
Balazs:
will define a set of validation steps
you can postpone this validation
Morris:
manual data staging
kick off requirement
complete knowledge if validated and executed
Etienne:
replace while by but
Balazs:
validation can be very heavy and time consuming
some of these steps can be done later
incomplete validation in the beginning and do the other steps later
Etienne:
execution should perform validation
David:
intensly complicated
change to *no*
###
91:
Balazs:
include full validation
activity creation
change to *no*
###
92:
Morris:
somebody who cannot understand this?
David:
clear
change to *no*
###
94:
Tim:
purge works only on final state
Oxana:
pending not a final state
Tim:
so you have to kill it
Oxana:
what is the definition of purge
Morris:
remove activity completely
change text
change to *yes*
###
96:
Mark:
activity gets migrated to another executionservice, the new one becomse
the manager and the old one can forget about it
use excution service and not BES
Morris:
example
Mark:
so ok
support migration
for efficiency reasons the original BES must not be longer responsible
for it
change to *no*
###
97:
Mark:
wait for Andrew
postponed
###
99:
Mark:
execution service start and stop accepting requests
Etienne:
opposite of 84
related
change to *no*
###
102:
Etienne:
automatic resubmission
what is not clear here?
Balazs:
which layer?
retry
Etienne:
clear or not?
change to *yes*
###
104:
David:
should be a yes but rework the language of the requirement
Oxana:
job description should allow for alternative data sources
David:
an input task may neet data to be staged from different stages
Morris:
JSDL does all or nothing
change text
Shabhaz:
no because of multiple sources for one of the data sources
change to *no*
###
106:
Balazs:
one data file you want to stage, more sources for this file
David:
you have to take the input from other people
Etienne:
alternate sources for the same file
Balazs:
text ok?
Oxana:
use case
created some data file
upload it
target
separate target
maybe more edifferent copies
please copy to at least two
should be able to specify in job description how many are needed
Morris:
data staging with and/or
still unclear after 30 min
###
105:
Etienne:
no because of no manual data staging
change to *no*
###
110:
Dave:
no, becasue flexibility ranges give
change to *no*
###
112:
Oxana:
unclear because you cannot "describe a description"
Morris:
change description to resource
Balazs:
in the draft specification there are more examples
Morris:
believe it is well understood
David:
yes, but a no
already in several places
Balazs:
job description is expressing what you want
resource requirement and not application requirement
clearly separation
a mess in current JSDL
Oxana:
memory consumption can also come from other not BES related jobs
Balazs:
no agreement
Oxana:
consider virtual machine
change to *no*
###
113:
Balazs:
structure already recommended in the draft specification
Morris:
again separation
path as one element and the executable as the other element
David:
why?
still unclear after 10 min
###
114:
David:
should be yes
clear
Oxana:
"favorite" requirement
change to *yes*
###
115:
David:
Oxana and myself understand but vote for no
Balazs:
why
David:
badly written
Aleksandr:
please propose your description
David:
include scale time required
Oxana:
new structure for job duration in units of benchmark
Balazs:
time requirement no benchmark requirement
Etienne:
if you cannot express time in seconds no time
Balazs:
structure containing time and benchmark
composed
name, value, time
David:
says which benchmarks?
Balazs:
not hardcoded benchmarks
Oxana:
somebody else has to scale time
Balazs:
you specify time as well
NO agreement
change to *no*
116:
Etienne:
totally unclear
Balazs:
in current JSDL data staging element
not defined if it is a directory or a file
idea is to identify if it is directory or file
David:
just copy data elemet
change the wording
change to *yes*
###
117:
David:
clear but no
Balazs:
for anybody unclear?
###
about 30 "unclear"
about 80 "no"
Morris:
1st step:
go through the unclears
Balazs:
at least two meetings for unclears
Morris:
looking on NOs
prioritisation approach
downgrade these to 30 or 40 which are very important
Mark:
some of the nos multiple telecons
Morris:
what do you think of prioritisation
David:
you just need to have it clearly stated how this should work
David:
two parallel groups to start with
would be a good thing and double the speed
Balazs:
next thing voting - decision making
Oxana:
if one group makes a decision it is final
Balazs:
who will vote?
Mark:
everybody has a vote
David:
voting is wrong
split up and work on nos and unclears and clear them
Mark:
OGF has already solved this
Luigi:
not possible to split
nobody else from gLite
Morris:
fine with trying this
David:
remove this voting
if somebody does not show up it is their problem
Balazs:
problem with finding two persons in gLite
David:
just continue with one group
going through the unclears
Mark:
will channel Andrew
we would find people for parallel groups
we could do parallel
Luigi:
maybe gLite can find a second or third person
Morris:
setting up teams in the google doc
Team 1, Team 2, Middleware, Reserve
Morris, Shahbaz, UNICORE, Shiraz
Balazs, Aleksandr, ARC, Oxana
Luigi, ???, gLite, Paolo
Etienne, ???, EDGeS, ???
Emannouil, Steve Brewer? , Globus, ???
Steve Crouch, Tim, OMI-UK, David
???, ???, GENESIS, Andrew, John
???, ???, NAREGI, ???
--
_ _ _ _ _ _ Johannes Watzl
|\/| |\ | |\/| Institut für Informatik / Dept. of CS
| | | \| | | Ludwig-Maximilians-Universität München
======= TEAM ======= Oettingenstr. 67, 80538 Munich, Germany
Room E 005, Phone +49-89-2180-9162
Munich Network Management Team Email: watzl at nm.ifi.lmu.de
Münchner Netz-Management Team http://www.nm.ifi.lmu.de/~watzl
More information about the Pgi-wg
mailing list