[saga-rg] SAGA conference call
Andre Merzky
andre at merzky.net
Thu Nov 24 07:13:11 CST 2005
Well well, the iisue list did not feel like attaching
yesterday, so lets see how it works out this time...
Cu soon,
Andre.
Quoting [Andre Merzky] (Nov 23 2005):
>
> Hi all,
>
> I scheduled the call for tomorrow, here are the details:
>
> Date: 24 November 2005
> Time: 3:00pm CET
> Number: 0844.888.8888 (UK)
> 01805.004.102 (Germany)
> +44.870.088.5706 (Others )
> Code: 808044
>
> This should be:
>
> +--------------+-------------+-------------+-------------+-------------+
> | Tokyo London | London | Amsterdam | New York | New Orleans |
> +--------------+-------------+-------------+-------------+-------------+
> | Thu 11:00 PM | Thu 2:00 PM | Thu 3:00 PM | Thu 9:00 AM | Thu 8:00 AM |
> +--------------+-------------+-------------+-------------+-------------+
>
> As for an agenda, I propose we start iteration of the issue
> list again - I attach the current version.
>
> Cheers, Andre.
>
>
>
> Quoting [Tom Goodale] (Nov 21 2005):
> >
> > Hi,
> >
> > does anyone wish to resume having regular SAGA conference calls ?
> >
> > Cheers,
> >
> > Tom
--
+-----------------------------------------------------------------+
| Andre Merzky | phon: +31 - 20 - 598 - 7759 |
| Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 |
| Dept. of Computer Science | mail: merzky at cs.vu.nl |
| De Boelelaan 1083a | www: http://www.merzky.net |
| 1081 HV Amsterdam, Netherlands | |
+-----------------------------------------------------------------+
-------------- next part --------------
+-------------------------------------------------------------+
#####
# # ##### ###### # #
# # # # # ## #
# # # # ##### # # #
# # ##### # # # #
# # # # # ##
##### # ###### # #
###
# #### #### # # ###### ####
# # # # # # #
# #### #### # # ##### ####
# # # # # # #
# # # # # # # # # #
### #### #### #### ###### ####
+-------------------------------------------------------------+
Peter Kunszt <Peter.Kunszt at cern.ch>
Kalman Kovari <Kalman.Kovari at cern.ch>
=====================================
1) The examples are sometimes more misleading than useful...
- DONE ANDRE: clearify, discuss on list
- why?
2) The error handling and error semantics are not defined well
enough. This is essential in order to assure proper
interoperability.
- TODO TOM: create proposal for procedural and OO
- compile list of error codes, check for consistency
3) There is a lack of structure in terms of interface
layering. High level interfaces (like JobManagement) and
low level interfaces (like Stream) are not distinguished,
motivated, or put into context. E.g. why is there a need
of a networkstream-level interface for a grid application?
- DONE ANDRE: CLEARIFY in intro
- intentional, see use cases!
4) Example (P18): OO structured, not like the iface.
(eg. i = dir.getNumEntries() instead of
dir.getNumEntries(a) ? )
- DONE ANDRE: cleanup, also session handle etc.
5) Example (P29): an exact copy of Ex. 1.
- DONE ANDRE
- check
6) Description - Streams (P62): close or destroyStream? maybe
close, but not consistent.
- DONE ANDRE: use <constructor> or so. - check SIDL
==> not specified in SIDL,
proposal: use "CONSTRUCTOR" and "destructor"
- see Notes in Streams
7) General: sometimes bitwise OR-s are used for multiple
options (Stream/ActivityType, P59), sometimes arrays of
values (NameSpace/CopyFlags, P9). Why no convention?
- DONE ANDRE: check for languages, fix
- I agree, bitwise OR is simplier
8) Interface NSDir: what are the semantics of the copy method?
of getName and getURL?
- DONE ANDRE: ask for clearification, sync with GFS
Thorsten Sch"utt:
-----------------
9) isFile -> isEntry?
- DONE ANDRE: ask mailing list
- no feedback - I think we should, for clearer naming
and structure.
- should we revive NSEntry (more reasons?)?
10) SIDL docs on wiki!
- DONE ANDRE
11) fix all examples in respect to session and context
- DONE ANDRE.
Use a default session handle if none is specified
-> examples are still valid :-D
12) flags
- DONE ANDRE: see above
13) Attribute -> AttributeSet
- TODO TOM: leave naming, but better describe idea
DONE ANDRE
Steven Newhouse:
----------------
14) Inconsistent use of security! Its only in the stream
section, nowhere else. "Not specifying a security system
makes sense, but I think all of these methods should have
a security token being passed into them or it should be an
argument in the relevant object constructor."
- DONE ANDRE
- should be fixed by session and context
15) I assume the URL can support a number of protocols -
http/https/file/gridftp. I see no way to register plugins
in the interface (may be this is an 'internal' interface
as opposed to a user funtion. Maybe there needs to be aet
of interfaces to help the developers, e.g. register
protocol plugins.
- OPEN
- plugin: implementation level
- url support: util class? any://? -> language specific
- need good error codes
16) State example language each time! Do examples in multiple
languages!
- OPEN
17) How are you going to handle partial functionality
implementations? Should _all_ operations support a
NotImplementedException? Should there be a standard static
method in each section to discover what is implemented?
e.g. supported protocols & supported methods.
- DONE ANDRE
- methods need ALL to be implemented.
- protocols etc: hmmm...
- discussed at GGF, DONE
- was discussed at GGF...
- there WILL be partial implementation, but they are not
SAGA conform then.
- necessary (session, context, ...)
- sufficient (one other subsys)
- CAN implement packages -> complete packages (partial
compliant)
- OPEN
- should we allow query of implemented packages?
18) You have an 'LSF' schema for >>, >, < & <<. But in the
file/directory area you use a series of attribute flags. I
think you should carry these attribute flags forward into
the job definition area - either way I think you should
have one model not two!
- DONE ANDRE: mark as known issue
19) I. General Model.
Language mapping.
Standard attribute model.
Security.
Tasks
II. Tier 1 Interfaces.
III. Tier 2 and above Interfaces.
- II and III on same level!
- DONE
20) What is to come in version 2?
- DONE ANDRE
- put into intro
- we have no explicit roadmap!!
- steering and monitoring
- possibly combining logical/physical files (read on
logical files)
- advert service
- GridRPC
- GridCPR
- Task dependencies (simple work flows and batches)
- extensions to existing classes
Others:
=======
21) 'The URL Problem': should we approach it?
- DONE ANDRE:
- explain problem, leave it open
- an implementation MAY be able to translate, or use
any://
- MUST throw good exception otherwise
22) advert service: easy: NS + Attribute + find on attribute.
Why not do it?
- DONE ANDRE: proposal to the mailing list.
23) find in name spaces?
- DONE ANDRE: proposal to the mailing list.
- DONE ANDRE: put into spec
- why not, at least for names!?
But over attributes?
What about query language then?
+-------------------------------------------------------------+
Andre:
======
24) list etc. only on dir - why not on dir_entry?
OPEN
25) go from getSize to real stat
OPEN
26) namespace ops on file names - why not on shell wildcards?
OPEN
27) should callbacks be introduced for some things? Best on
tasks, but also for streams (is factory!) Necessary for
monitoring/streaming, so best introduce in general...
- TODO ANDRE
- has been discussed in Boston
- very desirable for job and task states
- simple callbacks are sufficient
- impl. differences don't matter
- do it soon
28) integrate GridRPC / GridCPR / JSDL? What else?
Use Cases?!
- OPEN
- integrate to what extend?
29) check if bulk operations can be done with task containers
(e.g. explicit support needed/wanted for argument list
generation?)
- OPEN
30) ACLs!
- OPEN
- Later, after we get input from the security area
31) Authorization (call outs?)
- OPEN
32) task = ftf.read (1, buffer, &n);
task.run ();
When are 'n' and 'buffer' available? I would say after
wait (state = finished or cancelled).
Should we have a state 'failed'? For simplicity -
otherwise you have to check every job which 'finish'ed if
it actually finished successfully... (was discussion
feedback at VU)
- DONE ANDRE
33) into intro:
default is to define semantics everywhere.
point out parts where semantics is up to the implementation
- OPEN
GGF14:
======
34) send UC doc to editors!
- DONE SHANTENU:
- DONE SHANTENU: follow up
+-------------------------------------------------------------+
Phone calls:
============
35) relation to OGSA into intro
- DONE ANDRE:
36) - example are not normative for language binding
- DONE ANDRE
- state in intro
- provide one examples in various languages
- DONE ANDRE: C, C++, Perl, Java (DONE ROB)
- TODO TOM: Fortran
- TODO HARTMUT: Python
37) better state consistency model in intro:
- DONE ANDRE.
+-------------------------------------------------------------+
Various:
========
38) add myproxy context type
- DONE ANDRE
39) check Nathans CPR implementation
- DONE ANDRE
40) put JSDL, GridRPC etc specs onto WIKI
- DONE ANDRE
41) how to query LAST job state before STOPPED? History
possible? how to query reason for job error condition?
- DONE CHRIS/ANDRE
42) job states: PRESTAGING POSTAGING
- DONE CHRIS
43) jobs: what about sandboxing?
- DONE ANDRE
+-------------------------------------------------------------+
44) email use case authors to map Strawman to Use Case,
propose AG or conf call
- DONE ANDRE
45) map strawman to GridLab UC
- DONE ANDRE
46) add unix permission like ownership etc
- DONE ANDRE
47) check JSDL compliance
- TODO CHRIS/ANDRE
48) document realation of other GGF APIs and pieces
- DONE ANDRE - take from SC05 paper...
49) do a list of examples on the WIKI
- DONE ANDRE
50) a context should get added only once to a session.
- DONE ANDRE
51) wait on tasks: return immediatly if task is already done,
only throw on cancelled (why even then?)
- DONE ANDRE
52) getContext -- listContexts (consistency)
- DONE ANDRE
53) getState on TaskContainer, giving vec of states?
- DONE ANDRE
54) link/copy/move/remove: default use pwd? if yes, what is
state after move/remove?
- DONE ANDRE: don't allow (check POSIX) - BadParameter
- DONE ANDRE: introduce stale handle exception
55) check strawman for references
56) cancel blocking?
- DONE ANDRE: mailed to list
- DONE ANDRE
- should cancel be allowed to return immediately, w/o definite
successfully cancelling the task?
57) saga::job self = getSelf (); ?
- DONE ANDRE: added to jobService.getJob () semantics
58) IncorrectSession: e.g. to be thrown on method calls, where
the caller is attached to session_1, and an argument is
attached to session_2.
- DONE ANDRE
59) add attributes to stream:
- TODO ANDRE
- "bufsize", "1024"
- "timeout", "0.0010"
- "blocking", "yes"
- "nodelay", "yes"
- "compression", "yes"
- "reliable", "yes" (not streams, only messages)
60) add attributes to others? consistency model might be an
option for replication, blocking/nonblocking for file I/O
- TODO ANDRE
61) job: can we specify what files to clean up after job was running
(security!)
- TODO ??? : ask mailing list
62) tex/pdf handles some subsection underlinings incorrectly
- TODO ANDRE
63) should there be a method:
- OPEN
task.get_task_factory ()
which returns the tf which created the task? GridRPC has a
similar method call, and it seems to make it much easier to keep
track of tasks. E.g. one could start a remote read task, and
when that is finished start another read:
saga::file f (url);
saga::file::task_factory ftf = file.get_task_factory ();
saga::task t = ftf.read (100, out);
t.run ();
... // do may things
if ( saga::task::Done = t.get_status () )
{
// ok, do another read on that one...
saga::file::task_factory ftf = t.get_task_factory ();
ftf.read (100, out2);
}
That would be VERY useful for task containers, e.g. if 1 of n
tasks finishes, and requires a follow up call on the same task
factory. Right now, the user needs to maintain a mapping from
task to the respective task factory (which is not tooo difficult
though, just possibly cumbersome).
However: the problem with that approach is, that the return type
for the call differs for the various task types!! So it would
either require:
- have specific task types, and have task container
etc. work on a base class of them (ugh?)
- have a base task_factory type which gets returned and casted
(ugh?)
REVISED: we don't have task factories anymore, so the question
is if there should be the ability to get the task
creating object from a task...
64) should the wait on the task container have an option allowing to
wait for any/all tasks?
- OPEN
65) all constructors need additional optional session (describe
verbose)
- TODO ANDRE
66) add update_location to logical file?
- DONE ANDRE
67) remove use case sections (should go into req. doc)
- DONE ANDRE
68) wait on task containers: more possibilities such as in GridRPC?
- OPEN
69) give a SIDL intro (keywords)
- OPEN
70) describe task model 4 verbosely in the spec, leave details to
language bindings.
- TODO ANDRE
More information about the saga-rg
mailing list