[glue-wg] How to Work with Glue! WRT wLCG Vo Views

[Laurence Field]

Hi Owen,

The main aim of the Glue Working Group, which is defined in the Working 
Groups charter, is to consolidate the existing schema already in use 
into a commonly agreed schema which meets the existing use cases. The 
plan is following an aggressive time scale to which we are hopefully 
reaching the end. The schema reflects the current architecture both in 
topology and tooling already in use across the production 
infrastructures. It is not the aim of the group to define the 
architecture for grid computing but just encapsulate it into an 
information model.

However, I am very interested in your ideas and would like to discuss 
them further but it might be easier on the phone.

Speak to you soon.

Laurence



owen.synge at desy.de wrote:
> On Thu, 24 Apr 2008 12:17:17 +0200
> Laurence Field <Laurence.Field at cern.ch> wrote:
>
>   
>> Hi Owen
>>
>> I am not sure what you a referring to as VO service objects, please could you explain what you mean by this in more detail.
>>     
>
> Hello Laurence and All Glue people.
>
> All definitions are an attempt within Glue's perspective. 
>
> Definition
> ~~~~~~~~~~
>
> A VO Service object is the cross product of the concepts VO and
> Service and I suggest the fundamental Object that all things in Cloud
> computing or a Grid are Identified as.
>
> This implies a common understanding of the concepts of VO and Service.
>
> Principle Definitions
> ~~~~~~~~~~~~~~~~~~~~~
>
> A VO is more than this it is an abstract collections of users bound to
> an administrative domain that acquires a legal standing. For the
> purposes of Glue Specification management I believe it should be a
> grouping of user requirements. 
>
> Most members of most grids are members of multiple VO's for example by
> virtue of my Job, I can be a member of dteam, wlcg, ops, dgrid, sa3,
> "Eu funded", "Germany Funded" and academic.
>
> A service is an abstract catch all for any software that can be
> operated via a network interface and wishes to advertise its self to a
> collection of VO's.
>
> Clarification
> ~~~~~~~~~~~~~
>
> A VO Service object is the cross product of the concepts VO and
> Service, it sole purpose is as a service that is advertised to a VO as
> potentially useful and exploitable.
>
> To try and narrow down a VO service object in more solid form, it
> would probably have the following attributes.
>
> Unique Identifier
> Site Unique Identifier
> URI (Uniform Resource Identifier)
> VO (That it is advertising to)
> Type (Unique Identifier to the service Type)
> Implementation (Unique Identifier to Implementation of "Type")
> Version (Unique Identifier to Version of "Implementation")
>
> And maybe a few more mandatory attributes that don't spring to mind, but
> looking at Glue Service Objects will help. I expect a few optional
> arguments as well. 
>
>   
>> The Glue Schema just defines an abstract vocabulary for describing grid entities. Further markup of the Glue schema during deployment is envisaged and expected. There are various placeholders in existing entries for additional attributes and vendor specific extensions can always be defined. As most of the attributes are optional in Glue, it is not expected that everything will be published, only what is really needed to meet the specific use cases. To ensure interoperation between infrastructures, an interoperability profile needs to be defined which expresses what attributes are mandatory for the cross infrastructure use cases. 
>>     
>
> Im pleased but feel that having much more than VO Service Objects, as a
> base class and in effect inheriting from them. As a minimum level of
> clarity is imperative.
>
>   
>> I completely understand the need for VO based information and the ability to annotate the information. This is a real use case which is already in use for wLCG as you pointed out.
>>     
>
> I know, but I feel that many people, do not realise that VO's can
> publish. This has led to Glue becoming more complex than absolutely
> necessary.
>
> VO's such as discovery, monitoring and modeling are freely mixing in a
> single schema where they could more profitably be decomposed but that
> is future work for Glue 3.0. 
>
>   
>> However,this has more to do with the information system implementation rather than the schema. Essentially there are two different views of the grid, an infrastructure centric view and a VO centric view. The current version of the schema and information system does in take an infrastructure centric view. 
>>     
>
> I should like a decomposition of Requirements by VO, each VO having a
> separate schema. I believe this would lead to the following VO's
>
> Discovery (What is a service? computers? storage?)
> Site (What is in a Site? What Grids does it advertise to support?)
> Monitoring (reliability,resources)
> Grid (Grid management, wLCG, NorduGrid, OSG, dGrid) 
>
> And therefore Schema. I may have missed a Schema out.
>
>   
>> The Glue schema is a response to meet the existing demand for such a solution based on the current way of thinking. As grid infrastructures evolve and grid initiatives like EGI have more influence, we might see a paradigm shift which will require a change in Glue. However, before we started defining Glue 2.0, there were many discussions, including alignment with the OGF reference model, all of which have an infrastructure centric view. 
>>     
>
> Good point, but I see the need to define a index of services to
> facilitate derived schema, and consider resolving Glue into a series of
> Derived Schema from VO Service objects.
>
> Our infrastructure centric view is just one of the views we are taking
> in Glue. Therefore I feel Glue better expressed for future uses as three
> standards at least. With the VO Service Object as the class common to
> all schema. Glue at this moment is not just infrastructure centric, but
> is also monitoring and modeling.
>
>   
>> The problem highlighted is that VOs have there own vocabulary, naming and semantics that is important to them eg the wLCG Tier structures.
>>     
>
> Yes I agree with you Laurence, and suggest that VO's such as wLCG and
> Nordu Grid should put their Monitoring Requirements in a separate
> schema and define how service discovery should be unified by, sharing a
> common VO-Object with the Schema for Service Discovery, but potentially
> not bound between schema.
>
> I would see Monitoring as an area for a much richer schema than is
> provided by Glue currently. This is for good engineering reasons and to
> make things interoperable. Hopefully in time Monitoring service Summary
> output could be standardised to include availability metrics for a
> variety of service types and implementations.  
>
>   
>> However, this is specific to the VO and is something that does not need agreement. The VO is free to define there own vocabulary. The real problem is a technical one on how this information is used within an information system. For EGEE we have have evolved various mechanisms to do this, such as the FCR mechanism. 
>>     
>
> I was unaware of the FCR mechanism. For people who have not looked up
> what FCR mechanism is its a way so VO's can publish information.
>
> Unfortunately Grid wide Monitoring within Glue is potentially
> changeable, this is why I want VO-Service Objects, and a clear statement
> that all services are expressed through VO-Service Objects. I feel that
> transforming Glue to a set of schema that share a common VO-Service
> Object will massively simplify the FCR mechanism for users and reduce
> potential tight coupling, and increase interoperability.
>
> The danger of not embracing VO-Service objects is that we become unable
> to send users of Glue a clear way to describe resources consistently and
> briefly in their own schema as inherited objects as not all services
> are represented in a consistent way.
>
> I fear the consequence of inconsistency of representation of physical
> entities with in Glue 1.0 and potentially Glue 2.0 will lead to
> dependent representations suffering greatly when accounting objects are
> redefined for example. This use case occurred between Glue 1.2 and Glue
> 1.3. 
>
>   
>> I am really happy that you bring up the idea of Glue 3.0! As I said in my previous mail Glue 2.0 is not the end but just the being. As we are currently finalizing the specification, this is the wrong time to revisit the discussions on the fundamental model. Over time and through the influence of EGI etc, we may see a shift in the fundamental model to which this group must respond. 
>>
>>     
>
> I also share Laurence's fears about such a fundamental change as an
> entity relationship change, but to me the worry is that without
> re factoring to make the expression of an instance of a Storage element
> identical to that of the CE (which is I already a VO Service
> Object derivative) we will make work for users of the FCR and other
> Schema publishing methods that are used on Grids. If people are aware
> that the VO Service Object is the identifier of all services it, may
> well allow smoother progression to multiple Grid providing services. 
>
> Like Laurence and Sergio, but maybe for different reasons I want to
> focus on keeping Glue simple, for me this is best expressed as a series
> of sub schema all referencing a single VO Service Object Entity.
>
> I hope Glue 3.0 can be at least 3 schema sharing the common concept of a
> VO-Service Object and specifying instances of services in terms of
> VO-Service Objects, for the benefit of interoperability and FCR users.
>
> I agree this is too lofty a goal for Glue 2.0.
>
> I feel that changing the schema's entity relations in that area has
> been done so recently its a shame the idea of presenting services in a
> consistent way has not been received clearly enough from me.
>
> Since Glue is such a large effort, and has so many potential Grids as a
> target audience I feel Glue 2.0 could be improved immensely by
> simplification by this entity change and facilitate smother progression
> to Glue 3.0 with FCR users and similar communities.
>
> Regards 
>
> Owen
>