[saga-rg] Re: comment on SAGA strawman doc.

Tue May 31 11:22:25 CDT 2005

Dear Peter, Kalman, 

many thanks for your detailed comments!  We will try to
answer them all, but it will take us some time.  After a
first glance, they look very useful to me!

I forward the mail to the SAGA list, for archiving and
comments.

Cheers, Andre.

Quoting [Peter Kunszt] (May 31 2005):
> Subject: comment on SAGA strawman doc.
> Date:    Tue, 31 May 2005 18:08:50 +0200
> From:    "Peter Kunszt" <Peter.Kunszt at cern.ch>
> To:      "EGEE Middleware Engineering Management Team" <project-eu-egee-middleware-emt at cern.ch>
> Cc:      "EGEE Middleware Design Team" <project-eu-egee-middleware-design at cern.ch>,
> 	       "Kalman Kovari" <Kalman.Kovari at cern.ch>, 
>          <andre at merzky.net>
> 
> hi all
> 
> following up on our task to be interoperable with
> standards and to see how we fit into the large world
> of GGF, we have looked at the strawman SAGA API doc .
> http://wiki.cct.lsu.edu/saga/space/start/strawman-api.pdf
> we focussed on the data management side, to see whether we
> could implement these interfaces 'easily' or not.
> 
> some of the comments are a bit sketchy.. glancing
> at the SAGA mailinglist, some may have been discussed already.
> 
> so we have only looked at the APIs relating to data management.
> 
> the good news is that it seems that with some minor exceptions it would
> be quite straightforward
> to map the SAGA API to the existing Data Management APIs of the Fireman,
> glite-io and the metadata catalog. it is not a surprise that the SAGA
> API is much more sketchy and seems at times inconsistent and incomplete
> compared with our
> APIs since they don't have a real implementation behind them yet.
> 
> the only 'issue' is the introduction of a stream-based network
> API in SAGA. in our architecture / design we decided to go with the
> managed async data transfer model (FTS), not with direct synchronous
> streaming. the
> reason is obvious: we need to manage the network resource efficiently.
> 
> unfortunately the SAGA doc does not motivate the interfaces to an
> extent where we would be able to deduce an 'architecture'...
> we might also argue that some interfaces (like replica location, storage
> space management,
> lifetime management, quotas, etc) are missing...
> 
> cheers
> peter
> 
> ps. kalman kovari [cc'd] did the actual work, i am just echoing and
> commenting
> his findings. i also cc'd andre merzky of the SAGA-WG.
> 
> --------------
> 
> some detailed doc comments:
> 
>  - The interfaces in the document have some self-inconsistencies,
> however, none of them is critical. 
>    We expect these will be cleaned whenever someone starts to implement
> them.
> 
>  - The examples are sometimes more misleading than useful...
> 
>  - The error handling and error semantics are not defined well enough.
> This is essential
>    in order to assure proper interoperability.
> 
>  - There is a lack of structure in terms of interface layering. 
>    High level interfaces (like JobManagement) and low level interfaces
> (like
>    Stream) are not distinguished, motivated, or put into context.
>    E.g. why is there a need of a networkstream-level interface for a
> grid application?
> 
>  - Example (P18): OO structured, not like the iface. (eg. i =
> dir.getNumEntries() 
>    instead of dir.getNumEntries(a) ? )
> 
>  - Example (P29): an exact copy of Ex. 1.
> 
>  - Description - Streams (P62): close or destroyStream? maybe close, but
> not consistent.
> 
>  - General: sometimes bitwise OR-s are used for multiple options
>       (Stream/ActivityType, P59), sometimes arrays of values
> (NameSpace/CopyFlags, P9). Why no convention?
> 
>  - Description: Stream/read: Throws: IncorrectState if stream not in
> "NotConnected" state. typo.
> 
>  - Description: Stream/write: Same as above. C&P.
> 
>  - Description: StreamServer/create,destroy: create/createStreamServer,
> destroy/destroyStreamServer?
> 
>  - Interface NSDir: what are the semantics of the copy method? of
> getName and getURL?
> 
> -----------------
> 
> some details concerning implementation mappings.
> below we mention the classes to be implemented, and their methods:
> (an 'ok' is put to methods that are trivial to be implemented using the
> existing APIs. probably just
> name and parameter translations are required in that cases)
> 
> 
> iface NSDir - except for copy(), everything can be implemented using the
> fireman-catalog.
> 
>  changeDir: clientside manipulation+validity check in the catalog.
>  list: we only implement it with limits -- some default limits are
> required.
>  readlink: ok.
>  exists: getlfnstat+check.
>  isDir: getlfnstat+check.
>  isFile: getlfnstat+check.
>  isLink: getlfnstat+check.
>  getNumEntries: we have no efficient implementation, possible with
> readDir(), but slow.
>  copy: What do they mean saying "copy"? Needs clarification.
>  link: ok.
>  move: ok.
>  remove: unlink().
>  makeDir: ok.
>  
>  getURL, getName: undocd. Needs to be clarified what they mean here, but
> probably only clientside manipulation.
> 
> iface Directory extends NSDir - internal scheme to implement directory
> services. we already have a working catalog,
> 			so we only need to make an interface for it.
>  getSize: fireman::getlfnstat.
>  openDir: clientside consturctor+check in the catalog if name is valid.
>  open: open/create a file. glite_io::open() and for creating some
> fireman::create, too.
> 
> iface File - implementable through glite_io.
>  read, write, seek: ok.
> 
> iface LogicalDirectory extends NSDir - in the implementation scheme we
> have no difference
> 				between Directory and LogicalDirectory
> 
> iface LogicalFile
>  addLocation, removeLocation, listLocations: addReplica, removeReplica,
> listReplicas. ok.
>  replicate: addReplica
> 
> iface Stream, StreamServer - low level, seemingly managing network
> streams.
> [see comments above]
> 
> iface Attributes - usual key/value pairs, with support for
> vectorAttributes.
> 
> not looked at:
> iface Task, TaskContainer
> iface JobInfo, Job, JobService

-- 
+-----------------------------------------------------------------+
| Andre Merzky                      | phon: +31 - 20 - 598 - 7759 |
| Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 |
| Dept. of Computer Science         | mail: merzky at cs.vu.nl       |
| De Boelelaan 1083a                | www:  http://www.merzky.net |
| 1081 HV Amsterdam, Netherlands    |                             |
+-----------------------------------------------------------------+