[glue-wg] v1.3 documentation, please!

[Paul Millar]

Hi Stephen,

Thanks for the comments.  My comments are interleaved:

On Wednesday 07 May 2008 19:09:27 Burke, S (Stephen) wrote:
> > [...] nuggets of information, but none of them seem complete or
> > authoritative.
>
> Bear in mind that GLUE and the gip live in different domains, so nothing
> is authoritative for both (and the LDAP schema is different again).

Yes, indeed.  To my mind there are four layers that should be documentation:
  a. GLUE,
  b. GLUE/LDAP binding,
  c. EGEE/(W)LCG's usage of GLUE,
  d. how to get information into GIP.

Each layer builds on the previous one, so GLUE/LDAP documentation should 
(naturally) refer the reader to the relevant GLUE documentation for 
definitions of attribute values.  Because of this, the GLUE/LDAP 
documentation can (and should) be terse, but I think it is still needed.

I guess not all of this can be fixed in this forum: the last two are not 
really OSG GLUE WG issues; so perhaps documenting WLCG's usage and publishing 
through GIP should be an off-line discussion (from this mailing list).

> The 
> GSSD wiki is at yet another level, it's only for LCG and is only a
> recommendation - and personally I think is wrong in at least one respect
> (see below).

This is a management issue within WLCG.  It's probably not helpful discussing 
this further here, although it's something I feel should be discussed.

> For Glue 1.3, which is presumably what you're currently 
> implementing, OGF is not directly relevant although the web site does
> have various useful documents.

True, but OGF-GLUE-WG does provide the GLUE/LDAP binding, which I feel 
currently is lacking some documentation.

> > Second, I could also find no authoritative source of information on
> > Glue/LDAP binding for v1.3. 
>
> Fundamentally there is only one authoritative source, namely the actual
> LDAP schema in cvs! (as Maarten already pointed out.)

Ah!  At the risk of pointing out (at length) the obvious:
	validity != correctness
(disclaimer, my usage of the term "correct" here).

A document (e.g., an LDIF file) is valid iff it satisfies a set of validation 
criteria, which are usually written in some validation language.  With XML 
there are several validation languages (DTD, XML Schema, Relax NG, 
Schematron, ...), for LDAP there's the LDAP Schema language, etc.

A document is correct iff it contains all the necessary information in the 
prescribed fashion, allowing the information to be accessed.  I believe we 
are striving for correctness here, so people can use the information 
published; this is a much stronger requirement than validity, but also one 
that's harder to check, beyond applying validation checks.

In an ideal world, all valid documents would also be structurally correct; 
however, due to limitations in the validation languages, one cannot always 
write validation tests to fully check a document is structurally correct.

A specific example: GlueChunkKey is described in the LDAP schema as taking 
attribute values with the 1.3.6.1.4.1.1466.115.121.1.26 format, which is IA5 
(e.k.a ISO 646 or IRA5): a 7-bit-clean sequence of 8-bit bytes (roughly ASCII 
without $ or ~).  So a GlueChunkKey can take (more or less) any ASCII text as 
a valid attribute value.  However, from the notes, this is clearly not what 
is expected, hence validity != correctness.

So, for these reasons, one cannot (reasonably) claim that the schema is the 
single authoritative source as schemata can only validate a document, which 
is a lesser requirement than correctness.

>   There are some CNAF notes[3] [...]
> > [3]	http://glueschema.forge.cnaf.infn.it/SpecV13/LDAP
>
> Persoanally I think that whole page should be deprecated

Personally, I think the page should be deleted.  It contains incorrect 
information and (apparently) isn't being maintained (sorry Sergio, that's how 
it looks.)

However, as a proposal, I feel this WG should have an authoritative document 
(which could be a wiki page) that describes GLUE/LDAP for v1.3 GLUE.  This 
document should refer the reader to the LDAP-Schema in CVS, but is should 
contain the extra "additional" information, like how to construct a valid 
GlueChunkKey, where they might be necessary, etc.

> [...] Most of the other things are in the schema document anyway -

Most things are, but the GlueChunkKey (as one example) isn't.  There may be 
other information that is LDAP-specific but not captured by the validation 
tests (LDAP Schema).

> specifically, the old GlueService URI and URL attributes have been removed
> from my service publisher, which is just in the process of being released,
> and the CE renaming of CPUs to Slots is true but we still publish CPUs as
> well. 

Yes, but these are EGEE/WLCG issues, right?

[As an aside: does EGEE/WLCG document that GlueService -URI / -URL are no 
longer required?  Is there anyway of being notification when these 
requirements change?]

> The mds-vo-name part varies depending on where you query: = resource at
> [...]

Ta!

> As for the proposal, it suggests that all space tokens of a given type
> should be merged into a single SA, but personally I still think that we
> should publish one SA per token - this is e.g. how Jens' castor provider
> works and I find it extremely useful to keep track of free and used
> space.

Again, I guess this is a EGEE/WLCG issue; not that it isn't interesting, but 
I'm trying to limit myself to just OGF-GLUE-WG issues.

> However, we should discuss this more offline as this is not an 
> OGF/Glue 2 issue.

Agreed.  Which forum within EGEE (or WLCG) discusses its adoption of GLUE?

> >[How to publish to GIP]
> Well, it's LDIF conforming to the 1.3 schema, basically what you get
> from ldapsearch. As above, info providers need to publish their objects
> with DNs ending "mds-vo-name=resource,o=grid".

Aye, so "any old" LDAP isn't sufficient, it must be child objects under 
the "mds-vo-name=resource,o=grid" object.

> It may also be worth 
> pointing out that you can publish in two different ways: info providers
> publish entire objects, whereas gip plugins just publish the changes
> relative to a static file created by YAIM.

OK.

> Laurence will no doubt say more, but this is the wiki page:
>
> https://twiki.cern.ch/twiki//bin/view/EGEE/InformationSystem

Thanks!  That page helps greatly.

Cheers,

Paul.