[saga-rg] Telecon reminder
Andre Merzky
andre at merzky.net
Wed Dec 8 01:14:31 CST 2004
Quoting [Tom Goodale] (Dec 08 2004):
>
> Hi,
>
> so for the late reminder, we are going to have a telecon to discuss the
> design team meeting tomorrow (Wednesday 8th of Dec) at 8AM Pacific = 10AM
> Central = 16:00 UTC = 17:00 central europe.
Hartmut, Gab and I are at the GridLab workshop in Poland - we
don't know if we manage to join, but will try to. In case we
can't, here are some comments for the listed items:
> 0) Agenda bashing
> 1) Discussion of previous meeting (notes below)
> 2) Coordination of Postdam and CCT design tream fragments
Hartmut organised the meeting in Golm, Potsdam, but the
participant rate is low (may be a good thing, though) - we are
GridLabbers only: Hartmut, Andrei, Michael, me. Do you know of
anybody else we did not hear from?
We have the AG node daily from 4pm CET.
> 3) Coordination of CCT and other USA design team participants
> 4) Finalise participants lists
> 4) Discuss focus areas
see comments below.
> 5) Preliminary discussion of security issues (if sufficient interest)
>
> Design team meeting must
>
> o identify and review existing APIS
> - what is the status of the API list document Andre merzky and Gabrielle
> Allen are working on ?
The result of that is attached, and on
http://cs.cct.lsu.edu/saga/space/Related+Grid+APIs
The atteched text is obviously incomplete and not polished -
thats why we have been hesitant to send it earlier. Any help in
getting that done with soon would be appreciated.
> o identify where we can coordinate with these APIs
From a first glance, it seems not to make sense to just create a
mixture/superset of these APIs
- the Look and Feel is too diverse
- the same problem space is solved in different (sometimes
inconsistent) ways
But of course the are common paradigms. The most strong ones are:
- notion of directories for files/replicas
- provide remote data accesss via 'files'
- good, verbose error handling (different approaches)
I personally would focus on those points first, since they also
seem the most well understood ones, but if I understood
correctly, the F2F at SC04 already provided a (different?) list?
(Can you send the notes, please?).
Also, it seems useful to discuss general paradigms as Tasks, Bulk
Requests, Workflow, etc as general design points.
> o identify where the existing APIs don't interact well with each other
>
> o come up and work on an 'atomic core set' of APIs.
That is something you could derive from the attached doc as it is
now, that's why I do attach it. Hope that helps for an initial
set of APIs.
> We need to document what we find out and make sure it is widely
> disseminated. In particular we should have a two page summary document
> listing the use cases we looked at and the other APIs we reviewed.
>
> There was worry about how to coordinate the two design teams.
IIRC, the original proposal was to work semi-independently:
start the work in Europe, have a 2-4 hour overlap with the US and
hand over results/proposals/documents, and then vice versa.
Problem will be the handover back to Europe - we are missing an
Asian proxy *sigh*...
Best regards,
Andre.
> I am working on getting notes from the face to face out as well, however a
> lot of that discussion focussed on the use cases and focus areas, which I
> have previously summarised in a mail, and then went into a discussion of
> how to deal with security aspects of the API, for which we decided that we
> should actively engage security experts to work with us.
>
> I hope to be able to speak to many of you on the call tomorrow.
>
> Cheers,
>
> Tom
--
+-----------------------------------------------------------------+
| Andre Merzky | phon: +31 - 20 - 444 - 7759 |
| Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 444 - 7653 |
| Dept. of Computer Science | mail: merzky at cs.vu.nl |
| De Boelelaan 1083a | www: http://www.merzky.net |
| 1081 HV Amsterdam, Netherlands | |
+-----------------------------------------------------------------+
-------------- next part --------------
--------------------------------------------------------------------
Frameworks considered from within GGF:
--------------------------------------
- GGF - DRMAA - Distributed Resource Management Application API (WG):
DRMAA API Specification (GGF-DRMAA-API.pdf)
DRMAA Job Status Scheme (GGF-DRMAA-JobStatus.png}
- GGF - JSDL - Job Submission Description Language (WG):
JSDL Terms (GGF-JSDL-Terms.pdf)
- GGF - GFS - Grid File Systems (WG):
GFS Architecture (GGF-GFS-Arch.pdf)
GFS Specification (GGF-GFS-Spec.pdf)
- GGF - SRM - Storage Resource Management (WG):
SRM Specfification (GGF-SRM-SpecV2.pdf)
- GGF - GRPC - Grid Remote Procedure Calls (WG):
GRPC Specification (GGF-GridRPC-API.pdf)
GRPS Specification Ext. (GGF-GridRPC-API-ext.pdf)
- GGF - GCPR - Grid Checkpoint and Recovery(WG):
No Documents available as of yet
- GGF - DAIS - OGSA Data Services and Integration (WG):
OGSA-DAI Interfaces (GGF-OGSA-DAI.pdf)
OGSA-Data Services (GGF-OGSA-DATA.pdf)
- GGF - OGSA/OGSI (WG):
OGSA Specification (GGF-OGSA.pdf)
OGSI Specification (GGF-OGSI.pdf)
OGSI Primer (GGF-OGSI-Primer.pdf)
- GGF - OREP - OGSA Data Replcation Services (WG):
- no relevant documents -
- GGF - PA - Persistent Archives (RG):
Persistent Archive Basic
Components (GGF-PA-1.pdf)
- GGF - DFDL - Data Format and Description Language
- no relevant documents -
- GGF - ACE - Advanced Collaborative Environments (RG)
- no relevant documents -
--------------------------------------------------------------------
Frameworks considered from within SAGA Community:
-------------------------------------------------
- Globus - Commodity Grid (CoG):
URL: (http://www.cogkit.org/)
Globus CoG API v4 beta (Globus-CoG-API.pdf)
Globus GoG Object Model (Globus-CoG-ObjectModel.png)
- EGEE - Grid Abstraction API Draft:
URL:
EGEEE GridAPI Doc (EGEE-GridAPI-API.pdf)
EGEEE Arch Doc (EGEE-GridAPI-Arch.pdf)
- pV3 - Remote Visualization System:
System Description (pv3-1.ps)
--------------------------------------------------------------------
Frameworks considered from delivered use cases:
-----------------------------------------------
CoreGrid
- UC-1 - CM - Cyber Infrastructure for Coastal Modeling:
- no relevant documents -
- UC-2 - DiVA - Distributed Visualization Architecture:
Visualization Viewpoints (use-case-DiVA-1.pdf)
- UC-3 - DRMAA - Distributed Resource Management Application API:
DRMAA API Specification (GGF-DRMAA-API.pdf)
DRMAA Job Status Scheme (GGF-DRMAA-JobStatus.png}
- UC-4 - GridLab - Grid Application Toolkit (GAT):
GridLab GAT API Spec (use-case-GridLab-2.pdf)
- UC-5 - KoDaVis - Collaborative visualization of atmospherical data:
VisIt User Manual (use-case-KoDaVis-1.pdf)
- UC-6 - MedicalImaging - Iterative Image Reconstruction:
GEMSS...
- UC-7 - RealityGrid - RelityGrid Steering API:
RG Steering API (use-case-RealityGrid-1.pdf)
- UC-8 - SuperScalar - GRID SuperScalar:
SuperScalar Manual (use_case_SuperScalar_1.pdf)
SuperScalar Paper (use_case_SuperScalar_2.pdf)
- UC-9 - Visit - Computational Steering of a Ground Water Pollution Simulation:
GEMSS...
- UC-10 - VizService - Visualization Service for the Grid
Grid Viz API (use-case-VizService-1.pdf)
--------------------------------------------------------------------
Feature matrix for frameworks:
------------------------------
+-------------------+-------+-------+-------+-------+-------+-------+
| | DRMAA | JSDL | GFS | SRM | GRPC | GCPR |
+-------------------+-------+-------+-------+-------+-------+-------+
| API/Interface | x | - | - | - | x | ? |
| Protocol | - | - | - | - | - | - |
| Language | - | x | - | - | - | - |
| Service | - | - | x | x | - | - |
| Architecture | - | - | x | x | - | ? |
+-------------------+-------+-------+-------+-------+-------+-------+
| Resource | x | x | - | x | - | - |
| Data Access | - | - | x | x | - | - |
| Data Management | x | x | x | x | - | - |
| Data Bases | - | - | - | - | - | - |
| Data Stream | - | - | - | - | - | - |
| Logical Files | x | x | x | x | - | - |
| Events/Steering | x | - | - | - | x | x |
| Information | - | - | - | - | - | - |
| Communication | - | - | - | - | - | - |
+-------------------+-------+-------+-------+-------+-------+-------+
| Error | x | - | x | x | x | x |
| Audit | - | - | - | - | - | - |
| Transactions | - | - | - | - | - | - |
| Workflow | - | - | - | - | - | - |
| Tasks | - | - | - | - | - | - |
| Bulk | x | - | - | - | - | - |
| QoS | - | - | - | - | - | - |
+-------------------+-------+-------+-------+-------+-------+-------+
+-------------------+-------+-------+-------+-------+-------+-------+
| | DAIS | OGSA/I| OREP | PA | DFDL | ACE |
+-------------------+-------+-------+-------+-------+-------+-------+
| API/Interface | x | x | - | - | x | - |
| Protocol | - | - | - | - | - | - |
| Language | - | - | - | x | x | - |
| Service | x | x | x | - | - | - |
| Architecture | - | x | x | x? | - | - |
+-------------------+-------+-------+-------+-------+-------+-------+
| Resource | - | x | - | - | - | x |
| Data Access | x | - | - | x | x | x |
| Data Management | x | - | x | x | - | x |
| Data Bases | x | -? | - | - | - | - |
| Data Stream | - | - | - | - | - | x |
| Logical Files | - | - | x | x | - | - |
| Events/Steering | - | x | - | - | - | x |
| Information | - | x | - | x | - | x |
| Communication | - | x | - | - | - | x |
+-------------------+-------+-------+-------+-------+-------+-------+
| Error | x | x | x | x | - | - |
| Audit | - | - | - | x | - | - |
| Transactions | x | - | - | - | - | - |
| Workflow | - | -? | - | - | - | - |
| Tasks | - | x | - | - | - | - |
| Bulk | - | - | - | - | - | - |
| QoS | - | - | - | - | - | - |
+-------------------+-------+-------+-------+-------+-------+-------+
+-------------------+-------+-------+-------+
| | CoG | EGEE | pV-3 |
+-------------------+-------+-------+-------+
| API/Interface | x | x | x |
| Protocol | - | - | x |
| Language | - | - | - |
| Service | - | x | x |
| Architecture | - | x | x |
+-------------------+-------+-------+-------+
| Resource | x | x | - |
| Data Access | x | x | x |
| Data Management | x | x | - |
| Data Bases | - | x | - |
| Data Stream | x | - | x |
| Logical Files | x | x | - |
| Events/Steering | x | - | x |
| Information | - | x | - |
| Communication | - | - | x |
+-------------------+-------+-------+-------+
| Error | x | x | x |
| Audit | - | x | - |
| Transactions | - | - | - |
| Workflow | x | - | - |
| Tasks | x | - | - |
| Bulk | x? | - | - |
| QoS | - | - | - |
+-------------------+-------+-------+-------+
+-------------------+-------+-------+-------+-------+-------+-------+
| | UC-1 | UC-2 | UC-3 | UC-4 | UC-5 | UC-6 |
+-------------------+-------+-------+-------+-------+-------+-------+
| API/Interface | x | x | x | x | x | x |
| Protocol | - | - | - | - | - | - |
| Language | - | - | - | - | - | - |
| Service | - | - | - | - | - | x |
| Architecture | - | x | - | - | x | - |
+-------------------+-------+-------+-------+-------+-------+-------+
| Resource | x | x | x | x | - | x |
| Data Access | - | x | x | x | x | x |
| Data Management | x | x | x | x | - | x |
| Data Bases | - | - | - | - | - | - |
| Data Stream | - | x | x | x | x | - |
| Logical Files | x | x | x | x | - | - |
| Events/Steering | x | x | x | x | x | x |
| Information | - | x | x | x | x | x |
| Communication | - | x | x | x | x | - |
+-------------------+-------+-------+-------+-------+-------+-------+
| Error | - | x | x | x | - | x |
| Audit | - | - | x | x | - | x |
| Transactions | - | - | - | - | - | - |
| Workflow | x | x | - | - | - | - |
| Tasks | - | x | - | - | - | - |
| Bulk | - | - | x | - | - | - |
| QoS | - | x | - | - | - | + |
+-------------------+-------+-------+-------+-------+-------+-------+
+-------------------+-------+-------+-------+-------+
| | UC-7 | UC-8 | UC-9 | UC-10 |
+-------------------+-------+-------+-------+-------+
| API/Interface | x | x | x | x |
| Protocol | - | - | - | - |
| Language | - | - | - | - |
| Service | - | - | - | x |
| Architecture | - | - | x | x |
+-------------------+-------+-------+-------+-------+
| Resource | - | x | x | x |
| Data Access | - | x | x | x |
| Data Management | - | x | - | x |
| Data Bases | - | - | - | x |
| Data Stream | x | - | x | x |
| Logical Files | - | - | - | - |
| Events/Steering | x | x | x | x |
| Information | - | x | x | x |
| Communication | x | - | - | - |
+-------------------+-------+-------+-------+-------+
| Error | x | x | x | x |
| Audit | - | - | x | - |
| Transactions | - | - | - | - |
| Workflow | - | x | - | - |
| Tasks | - | - | - | - |
| Bulk | - | - | - | - |
| QoS | - | - | x | - |
+-------------------+-------+-------+-------+-------+
--------------------------------------------------------------------
Remarks:
--------
RealityGrid - Computational Steering API:
- High Level API
- FOCUS:
- remote steering (parameters, IO, CPR)
- remote data access (streaming, push)
- job status information
- simplem communication (msg)
- UTILITY:
- debug
- discover/inquire
- IMPLEMENTATION:
- in C
- target: C
- one implementation
- on top of own Grid infrastructure
GridLab - Grid Application Toolkit (GAT):
- high level API
- FOCUS:
- Files/Streams/Pipes
- Events/Monitoring
- Resources/Jobs/Brokers
- 'Advertisables' (objects + meta data)
- UTILITY:
- security context
- GATSelf (THIS job)
- location/URL
- preferences
- status (auditing/tracing/debugging)
- time/time period
- table, string, list, buffer --> map to native if possible
- IMPLEMENTATION:
- C, Java
- wrappers in C++, Python
- on top of GridLab infrastructure, Globus, Unicore, others
(implementation is thin layer, 'adaptors' bind to middleware
available - dynamic swapping of adaptors)
Globus - Commodity Grid (CoG):
- medium to high level API
(for applications, but historically reflects middleware (globus) 1:1,
add additional layers: task, graph, queue, workflow, portal)
- FOCUS
- File/Directory
- Job submission
- Task/Graph/Workflow
- UTILITY
- Task/Graph/Workflow
- Status
- IMPLEMENTATION
- on top of GT-2.4, GT-3.0, GT-3.2, GT-4.x (later), or SSH
- potentially more - implement new 'handlers'
GridLab - Grid Application Toolkit (GAT)
- hight level API
- FOCUS:
- Files
- File access
- Logical Files
- Communicaion channels
- Adverts
- Rsources
- Resource Broker
- Jobs
- Events/Monitoring
- UTILITY
- error traces
- some Debug
- Context (session)
- URL
- Time/TimePeriod
--------------------------------------------------------------------
GGF - Distributed Resource Management Application API (DRMAA)
- high level API
- FOCUS:
- job submission
- job monoitoring
- job control (not steering)
- UTILITY:
- error tracing
- IMPLEMENTATION
- in SUN Grid Engine
- C, Java under discussion (prototype exists)
GGF - Open Grid Service Architecture/Infrastructure (OGSA/OGSI)
- Grid: everything is a service
- Services: permanent, transient
- OGSA: management level for services
- creation/destruction/lifetime
- discovery
- interface description
- handshake/bootstrap
- security
EGEE - Middleware Prototype
- high level API
- FOCUS:
- Access Service (job submission and control)
- Security
- information service (consumer, producer)
- (gatekeeper
- (compute element)
- (storage element)
- (workload management)
- (data scheduling)
- file catalog
- meta data catalog
- UTILITY:
- error tracing
- ACL
- time, time period, frequency
- IMPLEMENTATION:
- None
GGF - Persistent Archive - Concepts
- describes concept space for PA community/applications
- FOCUS:
- persistens storage for large data
- logical files
- attributes
- data manupulation
- synchronization
- locking
- staging
- data access
GGF - Advanced Collaborative Environments
- describe ACEs
- security requirements
x
- scheduling requirements
GGF - OGSA Replication Services (OREP)
- describes services, interfaces and architectures
- interface: see egee API
GGF - Data Format and Description Language (DFDL)
GGF - Transaction Mgmt Research Group (TM)
- describes generic transaction service
GGF - Job Submission Description Language (JSDL)
- describes terms and language to be used for describing jobs to be
submitted to a resource management system.
Grid Computing Environments (GCE-RG)
- considers Grid Application Environments in geneal
- should review SAGA API
More information about the saga-rg
mailing list