[glue-wg] feedback on storage entities

[owen.synge at desy.de]

On Sun, 30 Mar 2008 05:51:00 +0200
<Maarten.Litmaath at cern.ch> wrote:

> Whichever way, we would like to publish such back-end implementation
> names and versions explicitly.  Flavia has a use case: the name and
> version of the back-end storage system should be available, such that
> it can be deduced from the information system which sites are likely
> to suffer from which open issues.  This is important information for
> debugging operational problems in WLCG (and other grids).


Dear all,

Could some one please explain why back-end storage system is important
for Glue?

The backend tape system which is only applicable to a handful of sites
(less than 14 sites I would guess) does not need to be discovered as we
tend to hand code special provision for such sites in our data flow
models. We have a white board in desy where we track the deployment of
all tier 1's. The data is not so large that it demands discovery for
bug fixing.

For me Glue is mostly about Tier 2/3 sites endpoint discovery which
have low resources and even lower manpower and yet will be expected to
fill in some (not all) of this relational information by hand. I feel
that modeling a Tier 1 or a tier 0 is exactly what we should not be
focusing upon. Since these sites are already known by name, have FTE's
who can fill in Glue schema values by hand (and our Grid specific
extensions). Tier 2/3 sites should ideally do less hand coding as
manpower is an issue their.

I'm in favour of a catalogue mapping service so catalogue
synchronisation can occur which if such was added to the LCG frame work
would make the tiny amount of "monitoring" data in the Glue useless as
the catalogues would know what data was used at each site and far more
usefully could delete files rather than just know they need to be
deleted.

Patrick has told me dCache position is to accept all requests we can
deliver and fight against all we cant do. Paul now represents dCache
for Glue, I just drop and email in now and again.

My comments are not dCache related, I just fear we are missing a rare
opportunity to remove attributes from an inter grid standard. I am much
happier we extend the Glue to resolve missing services for
LCG/OSG/Nordugrid within our grids and MOU's, but to add these as
fields all Grids should use seems bloated and bad form, when their
function is just to resolve architectural floors in our current grids
ability to mange its cataloges. Its consequence is to prevent us
focusing on missing data and syncronisation services, make inter
operation harder by adding what I see as non use cases, such as data we
should just know (backend tape system is not a job selection issue),
and dynamic trivia such as poor quality monitoring (that does not
include enough data to resolve running out of space anyway which seems
the only use case), but that's a longer email. Remember all fields (and
worse relations) we add to Glue wont go away for years and the costs
are most apparent with tier 2/3 sites. I feel we are not being hard
enough on people bringing use cases and looking at alternative
solutions. I feel that we could resolve all the SE service discovery
with just two (or three to be consistent with the rest of GLue)
entities if we removed what I currently see as counter productive use
cases.

Since I fear I am in the minority, I will stop this email now.

Owen

Opinions expressed here I mine, and not necessarily shared by Paul or
Patrick or the dCache team.