[glue-wg] New Endpoint and Service types

[stephen.burke at stfc.ac.uk]

Paul Millar [mailto:paul.millar at desy.de] said:
> Possibly, but then perhaps DPM and dCache also provide sufficiently
> different storage behaviour to quality.  It's difficult without a
> benchmark to decide.

I think this is the only case where we have multiple service implementations which are largely interoperable, so it probably needs to be decided on its own merits. It may be that we just don't have a relevant use-case - I still think the only likely need to make a distinction would be between dcache, dpm, storm, castor and bestman as a group and other things like standalone xrootd and gridftp. Despite your protestations of differences the first group are used more-or-less interchangeably by LCG and EGI, we have had quite significant efforts (with varying degrees of success) to make them interoperable and we have a set of tools (GFAL/lcg-utils) which allow them to be used in a uniform way.

> Other than that, I don't think there's much that's similar: they're
> rather different implementations, with different design choices.

But how does that affect the users? More specifically, what do you regard as a *contract* with your users beyond the specifications of the various protocols (and features which are already covered by GLUE attributes), i.e. documented behaviour that users can rely on? If dcache is a Type it needs a definition, which is presumably not just "whatever dcache happens to do right now" ...

> I think the problem with Type is in deciding the use-case for querying
> it.  When would they query Type rather than, say, Manager.ProductName?
> AFAIK, we don't have concrete examples where this information is useful.

ServiceType is only ever going to be useful in that sense where there is a need to identify sets of services which are bigger than a single implementation and smaller than the universe of services with a given set of endpoint types, or which are anyway identified as Computing or Storage Services (or any other specialisation). It seems possible that we have no instances at all where that's the case!

> > What protocol name do you recognise in e.g. a getTURL operation to
> > return an xroot TURL?
> 
> Currently it's 'root://'

That isn't what I meant:

https://sdm.lbl.gov/srm-wg/doc/SRM.v2.2.html#_Toc241633071

typedef                   struct {                   TAccessPattern                                                     accessPattern, 

                                                      TConnectionType                              connectionType,

                                                      string[]                                                                        arrayOfClientNetworks,

                                                      string[]                                                                        arrayOfTransferProtocols

} TTransferParameters

"arrayOfTransferProtocols" is a list of names of access protocols, but I don't believe that the SRM spec defines anywhere what those names should be (you'll see that it's only typed as "string[]" despite intrinsically being an enumerated type), hence each SE has its own internal list, which you can get with the GetTransferProtocols method:

https://sdm.lbl.gov/srm-wg/doc/SRM.v2.2.html#_Toc241633127

but that doesn't of course tell you explicitly what the names refer to. I tried a number of times to get some kind of official definition of them, but the best I ever got was a verbal agreement to follow the GLUE definition - which in the past was largely true (apart from the use of "rfio" in DPM which should have been "gsirfio" since it's completely different to the "rfio" known to Castor). However we're now getting support for new protocols, and I have no idea what names the various implementations are using for them.

>  > Does it match what DPM and StoRM use?
> 
> I couldn't say: you would need to ask DPM and StoRM people.

This seems to be a basic problem - we're trying to reach an agreement on standards between groups who don't talk to each other! I can see three main possibilities: a) all the storage implementors talk to each other and reach a common agreement; b) the most prevalent implementation (DPM) decides and everyone else follows; c) we give up and accept that we don't have interoperable standards.

> Yes, but please bear in mind that there are many extensions that build
> on top of HTTP and that an endpoint may support many (into
> double-digits) of them concurrently.  Publishing an endpoint for each
> results in (excessive?) duplication.

Maybe, but you'd only need to do it for cases where there is a practical need. Personally I don't know enough about http extensions to know what things people would commonly want to distinguish and query for.

Stephen

-- 
Scanned by iCritical.