[glue-wg] How to Work with Glue! WRT wLCG Vo Views

[owen.synge at desy.de]

On Thu, 24 Apr 2008 12:17:17 +0200
Laurence Field <Laurence.Field at cern.ch> wrote:

> Hi Owen
> 
> I am not sure what you a referring to as VO service objects, please could you explain what you mean by this in more detail.

Hello Laurence and All Glue people.

All definitions are an attempt within Glue's perspective. 

Definition
~~~~~~~~~~

A VO Service object is the cross product of the concepts VO and
Service and I suggest the fundamental Object that all things in Cloud
computing or a Grid are Identified as.

This implies a common understanding of the concepts of VO and Service.

Principle Definitions
~~~~~~~~~~~~~~~~~~~~~

A VO is more than this it is an abstract collections of users bound to
an administrative domain that acquires a legal standing. For the
purposes of Glue Specification management I believe it should be a
grouping of user requirements. 

Most members of most grids are members of multiple VO's for example by
virtue of my Job, I can be a member of dteam, wlcg, ops, dgrid, sa3,
"Eu funded", "Germany Funded" and academic.

A service is an abstract catch all for any software that can be
operated via a network interface and wishes to advertise its self to a
collection of VO's.

Clarification
~~~~~~~~~~~~~

A VO Service object is the cross product of the concepts VO and
Service, it sole purpose is as a service that is advertised to a VO as
potentially useful and exploitable.

To try and narrow down a VO service object in more solid form, it
would probably have the following attributes.

Unique Identifier
Site Unique Identifier
URI (Uniform Resource Identifier)
VO (That it is advertising to)
Type (Unique Identifier to the service Type)
Implementation (Unique Identifier to Implementation of "Type")
Version (Unique Identifier to Version of "Implementation")

And maybe a few more mandatory attributes that don't spring to mind, but
looking at Glue Service Objects will help. I expect a few optional
arguments as well. 

> The Glue Schema just defines an abstract vocabulary for describing grid entities. Further markup of the Glue schema during deployment is envisaged and expected. There are various placeholders in existing entries for additional attributes and vendor specific extensions can always be defined. As most of the attributes are optional in Glue, it is not expected that everything will be published, only what is really needed to meet the specific use cases. To ensure interoperation between infrastructures, an interoperability profile needs to be defined which expresses what attributes are mandatory for the cross infrastructure use cases. 

Im pleased but feel that having much more than VO Service Objects, as a
base class and in effect inheriting from them. As a minimum level of
clarity is imperative.

> I completely understand the need for VO based information and the ability to annotate the information. This is a real use case which is already in use for wLCG as you pointed out.

I know, but I feel that many people, do not realise that VO's can
publish. This has led to Glue becoming more complex than absolutely
necessary.

VO's such as discovery, monitoring and modeling are freely mixing in a
single schema where they could more profitably be decomposed but that
is future work for Glue 3.0. 

> However,this has more to do with the information system implementation rather than the schema. Essentially there are two different views of the grid, an infrastructure centric view and a VO centric view. The current version of the schema and information system does in take an infrastructure centric view. 

I should like a decomposition of Requirements by VO, each VO having a
separate schema. I believe this would lead to the following VO's

Discovery (What is a service? computers? storage?)
Site (What is in a Site? What Grids does it advertise to support?)
Monitoring (reliability,resources)
Grid (Grid management, wLCG, NorduGrid, OSG, dGrid) 

And therefore Schema. I may have missed a Schema out.

> The Glue schema is a response to meet the existing demand for such a solution based on the current way of thinking. As grid infrastructures evolve and grid initiatives like EGI have more influence, we might see a paradigm shift which will require a change in Glue. However, before we started defining Glue 2.0, there were many discussions, including alignment with the OGF reference model, all of which have an infrastructure centric view. 

Good point, but I see the need to define a index of services to
facilitate derived schema, and consider resolving Glue into a series of
Derived Schema from VO Service objects.

Our infrastructure centric view is just one of the views we are taking
in Glue. Therefore I feel Glue better expressed for future uses as three
standards at least. With the VO Service Object as the class common to
all schema. Glue at this moment is not just infrastructure centric, but
is also monitoring and modeling.

> The problem highlighted is that VOs have there own vocabulary, naming and semantics that is important to them eg the wLCG Tier structures.

Yes I agree with you Laurence, and suggest that VO's such as wLCG and
Nordu Grid should put their Monitoring Requirements in a separate
schema and define how service discovery should be unified by, sharing a
common VO-Object with the Schema for Service Discovery, but potentially
not bound between schema.

I would see Monitoring as an area for a much richer schema than is
provided by Glue currently. This is for good engineering reasons and to
make things interoperable. Hopefully in time Monitoring service Summary
output could be standardised to include availability metrics for a
variety of service types and implementations.  

> However, this is specific to the VO and is something that does not need agreement. The VO is free to define there own vocabulary. The real problem is a technical one on how this information is used within an information system. For EGEE we have have evolved various mechanisms to do this, such as the FCR mechanism. 

I was unaware of the FCR mechanism. For people who have not looked up
what FCR mechanism is its a way so VO's can publish information.

Unfortunately Grid wide Monitoring within Glue is potentially
changeable, this is why I want VO-Service Objects, and a clear statement
that all services are expressed through VO-Service Objects. I feel that
transforming Glue to a set of schema that share a common VO-Service
Object will massively simplify the FCR mechanism for users and reduce
potential tight coupling, and increase interoperability.

The danger of not embracing VO-Service objects is that we become unable
to send users of Glue a clear way to describe resources consistently and
briefly in their own schema as inherited objects as not all services
are represented in a consistent way.

I fear the consequence of inconsistency of representation of physical
entities with in Glue 1.0 and potentially Glue 2.0 will lead to
dependent representations suffering greatly when accounting objects are
redefined for example. This use case occurred between Glue 1.2 and Glue
1.3. 

> I am really happy that you bring up the idea of Glue 3.0! As I said in my previous mail Glue 2.0 is not the end but just the being. As we are currently finalizing the specification, this is the wrong time to revisit the discussions on the fundamental model. Over time and through the influence of EGI etc, we may see a shift in the fundamental model to which this group must respond. 
> 

I also share Laurence's fears about such a fundamental change as an
entity relationship change, but to me the worry is that without
re factoring to make the expression of an instance of a Storage element
identical to that of the CE (which is I already a VO Service
Object derivative) we will make work for users of the FCR and other
Schema publishing methods that are used on Grids. If people are aware
that the VO Service Object is the identifier of all services it, may
well allow smoother progression to multiple Grid providing services. 

Like Laurence and Sergio, but maybe for different reasons I want to
focus on keeping Glue simple, for me this is best expressed as a series
of sub schema all referencing a single VO Service Object Entity.

I hope Glue 3.0 can be at least 3 schema sharing the common concept of a
VO-Service Object and specifying instances of services in terms of
VO-Service Objects, for the benefit of interoperability and FCR users.

I agree this is too lofty a goal for Glue 2.0.

I feel that changing the schema's entity relations in that area has
been done so recently its a shame the idea of presenting services in a
consistent way has not been received clearly enough from me.

Since Glue is such a large effort, and has so many potential Grids as a
target audience I feel Glue 2.0 could be improved immensely by
simplification by this entity change and facilitate smother progression
to Glue 3.0 with FCR users and similar communities.

Regards 

Owen