[UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET

Tue Feb 28 05:24:09 EST 2012

Hi John,

I think there were at least two of us that understood that some sort of 
integral was needed. In fact after the discussion on integral vs 
sampling we moved on the discussion of timeduration vs validduration.
For me it was now clear that we moved in this direction.
This allows a single record to give all the information we need on the 
storage usage.

Andrea

On 02/28/2012 10:58 AM, john.gordon at stfc.ac.uk wrote:
> Mike, I think you are way out of line here. As long ago as Munich OGF we discussed time integrals for UR and I thought we reached a consensus (well maybe everyone except you) that we could not ask storage systems to start evaluating integrals of storage usage for the use of UR. They don't do it and we would be asking them to implement journaling or to recalculate usage at every file interaction (or at least write/modify/delete). Just not on.
>
> I thought we all understood that this is a compromise for the pragmatic purpose of getting a UR soon that can be implemented in the real world. You seem to enjoy the tautological discussion of what accounting should be in the theoretical world. I just want to start writing records tomorrow, if not today.
>
> In the longer term maybe we make requirements against storage systems that they implement integrals as well as user level and i/o accounting.
>
> For now can we concentrate on agreeing something that can be implemented.
>
> John
>
> PS I'll try to call in. I'm in Taipei.
>
>> -----Original Message-----
>> From: Mike Jones [mailto:mike.jones at manchester.ac.uk]
>> Sent: 21 February 2012 18:48
>> To: Gordon, John (STFC,RAL,ESC)
>> Cc: andrea.cristofori at cnaf.infn.it; j.k.nilsen at fys.uio.no; ur-
>> wg at ogf.org
>> Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012,
>> 15:00 CET
>>
>>
>> John,
>>
>> I always come back to time integration because once the record is
>> written it
>> cannot be misinterpreted; it is by definition correct even if it is not
>> accurate [that is important].  I understand the inaccuracies of
>> sampling, but
>> the rules of said sampling are specific to the resource, are probably
>> restricted by tooling and must ultimately form part of their service
>> description.
>>
>>> Once one has a collection of records then different grids, projects,
>>> infrastructures may apply different procedures.
>> It is precisely this variation that I am trying to eliminate. Resource
>> consumers and providers are the primary entities concerned with
>> generation and
>> consumption of URs. Grids, brokering systems, infrastructures need not
>> ever be
>> present and as such may make only secondary use of these records, if
>> permitted.  It should certainly not be these secondary relying parties
>> that
>> interpret these data with assumptions, for instance, on how regularly a
>> record
>> was cut.
>>
>> <deeper>
>> If you insist on having usage records only define an instantaneous
>> value then
>> it becomes difficult to understand how much the resource was used:
>> You then have 3 choices to retrieve useful information:
>> 1, to requires an infinite set of instantaneous URs [clearly stupid]
>> 2, to gain the set of all URs produced and assumes static between each
>> [requires complex safeguards],
>> 3, rely on the grid infrastructure, that it has some universal way of
>> interpreting this [wrong domain: assumes infrastructure policy, e.g.
>> set by
>> EGI not OGF].
>>
>> In conclusion,
>> UR must be able to cope with both sampled data and known (continuous)
>> data
>> usage.  It must be expressed within the usage record so it cannot in
>> itself be
>> taken out of context.  It must therefore contain integrated usage data.
>>
>> I have, up until now, attempted realise what I think you require as
>> incidental
>> information extra to usage but I am rapidly coming to the conclusion
>> that what
>> you really want is an allocation record not a usage record.  An
>> allocation
>> record would allow you to subsequently generate a usage record given a
>> set of
>> sampling rules.
>>
>> Mike
>>
>> On Tuesday 21 February 2012 14:07:07 john.gordon at stfc.ac.uk wrote:
>>> Mike, you keep coming back to time integration. I think that once we
>>> decided on sampling rather than evaluating integrals then all we have
>> is a
>>> series of observations and there are a variety of conclusions we can
>> draw
>>> from them.  None of them contain the word ‘always’ since we can never
>> know
>>> what happens between observations.
>>>
>>> Once one has a collection of records then different grids, projects,
>>> infrastructures may apply different procedures.  I am not against the
>>> durationtime/elapsedtime argument but I won’t be attempting to
>> construct
>>> time integrals from it. I will not be treating periods of time not
>> covered
>>> by records as unused or empty space. I will be calculating
>> min/max/mean
>>> over a time period with perhaps some weighting if the end-user
>> projects or
>>> sites feel that this isn’t the full story. I might come back and
>> propose a
>>> summary UR for passing this information around. I think that for many
>>> people desperate for storage accounting these simple metrics will be
>>> enough. If they are not then we can reconvene and change/extend the
>> UR.
>>> John
>>>
>>> From: Mike Jones via mobile [mailto:mike.jones at manchester.ac.uk]
>>> Sent: 21 February 2012 10:56
>>> To: Gordon, John (STFC,RAL,ESC); andrea.cristofori at cnaf.infn.it;
>>> j.k.nilsen at fys.uio.no Cc: ur-wg at ogf.org
>>> Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012,
>> 15:00
>>> CET
>>>
>>> To follow up on John's WLCG usecase: I agree that VO space allocated
>> over a
>>> period of time _is_ a usage. BUT VO allocation at any instant is not.
>> That
>>> said, the dynamics of any allocation/usage needs to be recorded to be
>> able
>>> to answer the question "over period X did the allocation always
>> satisfy:
>>> instantaneous_storage( used+available)>  N (dimension:bytes)?" John,
>> does
>>> that last clause satisfy WLCG?
>>>
>>> Mike
>>> --
>>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>>> john.gordon at stfc.ac.uk wrote:
>>> I’ll be in Taiwan next week UTC+8 so I may or may not attend. Depends
>> if
>>> anything official is happening that evening and if I can stay awake
>> and if
>>> Skype works.
>>>
>>> I’ll add the WLCG storage use case to the wiki before then. I must
>> stress
>>> that this is important to WLCG and that we consider space allocated
>> to a
>>> VO as ‘usage’ as it is not available to others and is to be recorded
>> just
>>> like space occupied by files. i.e. Not monitoring any more than
>> recording
>>> the existence of files in monitoring.
>>>
>>> John
>>>
>>>
>>> From: ur-wg-bounces at ogf.org [mailto:ur-wg-bounces at ogf.org] On Behalf
>> Of
>>> Andrea Cristofori Sent: 21 February 2012 08:15
>>> To: Jon Kerr Nilsen
>>> Cc: ur-wg at ogf.org
>>> Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012,
>> 15:00
>>> CET
>>>
>>>
>>> Dear all,
>>>
>>> Sorry for the short notice but neither I or Jon can make it to today
>> phone
>>> conference. We would like then to postpone it to next week (same day,
>> same
>>> time). About today actions Jon already published the document and I
>> got an
>>> answer for the slot at OGF:
>>>
>>> http://www.ogf.org/gf/event_schedule/index.php?id=2413
>>>
>>> Again apologies.
>>>
>>> Cheers,
>>> Andrea
>>>
>>>
>>>
>>>
>>> On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote:
>>> Hi all,
>>>
>>> As John AK just reminded me, the next phone meeting is today. We'll
>> go
>>> through the actions mentioned below and possibly make some new ones.
>> Just
>>> to try out something new, I suggest we try the Skype conference room
>> in
>>> Lund (which is starting to get stable now). So you can call in to
>>>
>>> Place: Skype
>>> Skype contact: lund.room.1
>>> Phone bridge: +46462884608
>>>
>>> cheers,
>>> Jon
>>>
>>> On 7. feb. 2012, at 17:32, Andrea Cristofori wrote:
>>>
>>> Dear all,
>>>
>>> Most of the phone conference has been focused on the comments on the
>> use
>>> cases provided by John AK. We commented directly inline on the wiki.
>> Feel
>>> free to add more Use cases if not present. We will comment them
>> during
>>> next phone conference. We also agreed in asking for a second time
>> slot
>>> during OGF (if possible the same day) and divide the agenda in two
>> parts
>>> and JKN will look for information on how to make the document he
>>> circulated the 25th public. The actions for next meeting are the
>>> following:
>>>
>>> - AC: Ask for another session the same day at OGF and update the
>> agenda on
>>> OGF website - JKN: make public the document
>>> - All: write some examples for the different use cases (minima, grid,
>>> local, etc.) (Ralph local, JAK Grid/dCache)
>>>
>>>
>>>
>>> Especially for John Gordon: can you please comment on those two use
>> cases:
>>>    1.  As a project I would like to be able to view the used and
>> unused
>>> storage space that I have on a storage resource. - Thus I can see how
>> much
>>> headroom I have.
>>> All: seems to be more monitoring Comments from John are required
>>>
>>>    1.  As a project I would like to be able to view the requested
>> storage I
>>> have on a specific resource and the allocated/reserved resources I
>> have on
>>> that resource. - Thus I can see I asked for 100TB and I currently
>> have
>>> only 80TB at my disposal (of which i am using 50TB). All: seems to be
>> more
>>> monitoring Comments from John are required
>>>
>>>
>>> Cheers
>>> Andrea
>>> --
>>>   ur-wg mailing list
>>>   ur-wg at ogf.org<mailto:ur-wg at ogf.org>
>>>   https://www.ogf.org/mailman/listinfo/ur-wg
>>>
>>>
>>>
>>>
>>> --
>>>
>>>    ur-wg mailing list
>>>
>>>    ur-wg at ogf.org<mailto:ur-wg at ogf.org>
>>>
>>>    https://www.ogf.org/mailman/listinfo/ur-wg
>>>
>>>
>>>
>>> --
>>>
>>> Andrea Cristofori
>>>
>>> INFN-CNAF
>>>
>>> Viale Berti Pichat 6/2
>>>
>>> 40127 Bologna
>>>
>>> Italy
>>>
>>> Tel. : +39-051-6092920
>>>
>>> Skype: andrea-cnaf
>>>
>>>
>>> --
>>> Scanned by iCritical.

-- 
Andrea Cristofori
INFN-CNAF
Viale Berti Pichat 6/2
40127 Bologna
Italy
Tel. : +39-051-6092920
Skype: andrea-cnaf