[glue-wg] When is data stale?

Tue Apr 21 05:53:38 EDT 2015

Hi all,

I also have the feeling the discussion is becoming a bit sterile. We can
make the GLUE2 spec better but I hardly understand how Paul definitions
without using the actual terms we want to define could help.

I might sound harsh somewhere down but is not my intention, I am trying
to get to the definition Paul would like to have. I apologise if the
tones sound bad, I really don't want to!

Few comments down:

On 2015-04-20 19:46, Paul Millar wrote:
> [...]
>>> Florido: "No. CreationTime refers to the record related to the
>>> entity described. Only the resource provider that creates the
>>> datastructure can know such time. BDII levels above resource just
>>> copy the data." Again, straw-man fallacy as it requires GLUE 2 to
>>> distinguish between resource- and higher-level BDIIs.
>>
>> I don't understand your point at all here. Let's try again: the
>> creation time is the time the *information* represented in a GLUE
>> object is created.
> 
> Again, a circular (or semantically null) defn: you're defining
> CreationTime using the phrase "the time [something] is created".
> 
> Put another way, the concept of 'information being created' is too loose
> a term: it could mean almost anything, so defines nothing.
> 

Well, this is a rhetorical game and not a scientific discussion anymore
IMHO. I understand you want a definition out of the practical
implementation, and since you seem to like riddles, I will avoid the
words creation and time (at this point a mere exercise of wording)
here it is:

The CreationTime is the number of seconds elapsed since the Epoch
(00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970)
formatted as described in the GLUE2 document when BOTH these two are true:
1) the GLUE2 record for a GLUE2 entity is being generated
2) the data contained in the record, that is, the data that describes
the entity the record refers to, is being collected.

I see no fallacy nor circularity. It's a definition. It does NOT require
the knowledge of provider, resource- whatever-BDII

Of course, if you want to be really picky there is a time drift between
1) and 2) because a Turing machine is sequential. But we can avoid this
discussion I hope...

I can provide a similar definition for Validity if you like... but I
will shift to Stephen's suggestion that this is community-driven, but
it's not because of the model, it's because what is "Valid" is community
driven, and by experience I can tell it will be even if you try to
define it otherwise!

Maybe the only real outcome of this discussion is Jens' comment that
'Validity' was a bad name! :D

>> That information may be copied, translated or stored in many
>> different formats, none of which has anything to do with the
>> information itself, i.e. the values of the various attributes.
> 
> How do you distinguish between information being copied from some other
> BDII and being copied from an info-provider?
> 
> In fact, these are basically the same.  BDII even treats them the same:
> they are just two potential sources of information.
> 
> The only distinction between being a resource-, site- or top-level BDII
> is where it fetches its information.
> 
>>> However, the difficulty in describing the exact semantics of
>>> CreationTime suggests (to me) that GLUE 2 _does_ include the
>>> concept of aggregation, at least because it has CreationTime with
>>> different processing models depending on whether the agent is a
>>> primary data source (e.g., Resource BDII) or an aggregating source
>>> (Site- or Top- BDII).
>>
>> The thing which sets the CreationTime is not the BDII at any level,
>> it's the information provider.
> 
> "BDII" and "information provider" are not defined in GLUE 2 (except in
> Appendix A.).  Therefore, cannot contribute towards the definition of
> CreationTime.
> 
> In case it isn't obvious, I agree that CreationTime should be set by the
> info-provider and not modified by any BDII.
> 
> What is interesting is that (apparently) one cannot describe this
> desired behaviour rigorously in GLUE-2, despite everyone being agreed
> this is the desired behaviour and the other behaviour is plain wrong.
> 
> To me, this points to a deficiency in GLUE 2.
> 
> [...]
> 
>

I do not see the needs to describing it in the model. One describes that
in an implementation of a hierarchical information system (today only
BDII and maybe EMIR, which nobody uses)

Otherwise we need a model that takes into account hierarchical
propagation of information (as mentioned before, an aggregation model)

But for me having the above in the GLUE2 model sounds like if physicist
should describe the Standard Model in terms of the pieces of paper,
emails, research papers, people, historical events needed to describe
the physics in it...

>> [...]
>> I don't see that this is different to any attribute - what you
>> publish needs to be driven by the use cases. It wouldn't be
>> especially difficult to publish a different Validity for each object
>> type, or even for e.g. different batch systems, but unless you have
>> something to specify the use there's nothing to motivate such a
>> varying choice.
> 
> My use-case was what you might expect: allowing detection of a
> particular failure mode.  Specifically, the information publishing "got
> stuck" at one site.  The details don't matter, but the result was old
> ("stale") data continued to be re-published.
> 

In ARC, we decided long time ago that the information system should NOT
be used as a monitor for the information system itself. If one does that
it does it at his own risk; the reason lies behind the fact that the
information system is more like a business card. It presents services to
users. It might fake some of the information to please the
users/communities needs, or to hide faults
in the system in a way that the overall system still works (and this is
what actually happens!)

Using the information system as a monitoring tool requires a different
approach, namely, the information system itself must be able to
self-diagnose. Apart from the philosophical question if this is even
possible, for ARC this is difficult because the information system is
part of/triggered by other parts of the middleware: if the middleware
dies the infosys dies with it. This is not up to GLUE2 to define, and is
not part of most current architectures, and to me it indicates that
proper monitoring should be done with third party tools. As a matter of
fact that claim applies to most software.

So if you want to know if the information publishing "got stuck" you'd
better be a good sysadmin and use a decent process monitoring tool, let
it be Nagios or a simple cronjob that sends emails...

Cheers,
Florido
-- 
==================================================
 Florido Paganelli
   ARC Middleware Developer - NorduGrid Collaboration
   System Administrator
 Lund University
 Department of Physics
 Division of Particle Physics
 BOX118
 221 00 Lund
 Office Location: Fysikum, Hus B, Rum B313
 Office Tel: 046-2220272
 Email: florido.paganelli at REMOVE_THIShep.lu.se
 Homepage: http://www.hep.lu.se/staff/paganelli
==================================================