[gin-info] [Fwd: Re: Memo on GIN-info discussions on June 20]

[Yuji Saeki]

I forward mails among Satoshi, Laura and Laurence. (4)

Yuji

-------- Original Message --------
Subject: Re: Memo on GIN-info discussions on June 20
Date: Mon, 26 Jun 2006 10:14:36 -0700
From: Laura Pearlman <laura at ISI.EDU>
To: Laurence Field <Laurence.Field at cern.ch>
CC: matsu at is.titech.ac.jp, saga at grid.nii.ac.jp,
ysaeki at grid.nii.ac.jp,hitoshi.sato at is.titech.ac.jp
References: <000001c69409$7c57e770$bb1cbb88 at npc13>
<20060620173425.3B5E.MATSU at is.titech.ac.jp>
<7CDDCC4ADBB6504993824620A385CA0B780649 at cernxchg14.cern.ch>
<7CDDCC4ADBB6504993824620A385CA0B780665 at cernxchg14.cern.ch>

>
> We need to clarify our objective, is it the minimal set of attributes that we need for "Hello World" job submission? A useful VO job? Full service discovery or full schema interoperation? I assume that for SC06 it is the "Hello World" job.
I agree, although I'm not sure what "Hello World" means in this case.  I 
would like to see this clarified to, perhaps, a set of simple queries 
that we should be able to ask any of the monitoring systems (e.g., "find 
all clusters across the participating grids that have at least N nodes 
with processor type X").
> We should already have mapping documents between ARC/Glue and Glue/Naregi. From these, we should highlight the minimal attributes from each schema that are require for "Hello World" job submission.
My understanding from last week's call is that this is somewhat 
complicated for CIM -- apparently CIM expresses information at a finer 
granularity than the minimal attribute set, so while it seems to be 
fairly straightforward to map CIM data to the minimal attribute set 
(using aggregation operations such as counting numbers of jobs or in 
some cases adding or multiplying numeric-valued attributes), it's more 
difficult to map the minimal attribute set onto the CIM schema.

                   -- Laura

Laurence Field wrote:
> Hi,
>
> I am sorry for the delay, here is my response to the memo
>
> Deliverable 1:
>
> We need to clarify our objective, is it the minimal set of attributes that we need for "Hello World" job submission? A useful VO job? Full service discovery or full schema interoperation? I assume that for SC06 it is the "Hello World" job. We should also point out that we are only interested in the three schema, ARC, GLUE and the Naregi Extensions to CIM.  
>
> We should already have mapping documents between ARC/Glue and Glue/Naregi. From these, we should highlight the minimal attributes from each schema that are require for "Hello World" job submission.
>
> Phase 1
>
> Usually resource selection is done via the VO flag in the information system. Therefore, we should be able to use production sites and just add support for the GIN VO. 
>
> Phase 2
>
> I don't think we should deploy two systems. Deploying the prototype and evolving it into a production system should be fine.
>
> As for discussing the quality of the prototype .......
>
> All code should be simple. Simple code works, complex code doesn't. If code is complex, the problem has not been understood.  All code should be extensible, robust and maintainable and this is especially true for prototypes as you will want to evolve them. :)
>
> I am not sure that we discussed the idea of a production system.   I thought the idea was that the gateways would be maintained by the infrastructure that needed them. For example EGEE would maintain the NAREGI to EGEE gateway and NAREGI would maintain the EGEE to NAREGI gateway. This way the people doing the maintenance have the motivation to keep them up and running. As the providers are unique to the task I don't think that there needs to be any standard providers, they should be custom made for the task. Also I feel that writing providers in Java is a bad idea, it is like using Java for command line tools, the time for the JVM to load and the foot print does not make it suitable for the task.  
>
> Having standard translators might be a good idea but I think that the effort it takes might not be worth it over custom translators.
>
> I think the rest of the memo describes generic solution for service discovery. This was not discussed before, however, if this is the direction you want to go, I am all for it.  The information system is currently used for two main purposes, service discovery, finding the endpoints of service and service meta data for those services, in particular the CE and SE.  If we forget about the metadata, which in theory we could get by contacting the service directly and not using the information system, we have the problem of service discovery. Within EGEE we are currently trying to roll out this as as specific use of the information system.  We already have the GlueService entry/table/object which defines the attributes needed for service discovery.  The GIP can already provide this information and we are currently ensuring that this is done for all services.  We also have an API for Service Discovery.  This has been implemented with plugable backends so that it can use either BD!
 II or R-GMA, however this could be extended to use MDS4, Cell Domains, ARC MDS, etc.  We have already spoken to OSG about this and will be comparing APIs.
>
> For hacking together "Hello World" jobs for SC06,  prototype gateways are the way to go.  If we are talking about doing service discovery properly then I am all for it but it wasn't what we were talking about before.  This does reduce the scope a great deal, which I think is good, and focuses on the fundamentals.  We would then need to do the following.
>
> Agree on the API (EGEE and OSG already have one, not sure about Naregi)
> Agree on the Schema for Service Discovery. (It is already in Glue)
> Ensure that all Grids are publishing this information.
> Create plugins for the API for the different systems.
>
> Let me know what you think
>
> Laurence
>