[UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET

Tue Feb 28 05:02:25 EST 2012

John, thanks for your support. Your proposal is fine with me.

I am happy to include a time period and my preference is for the time period since the last published measurement.  One could then treat the measurement as an average over that time period. One could them combine records over a longer time period and rewrite the time period in a new record.

I don’t think we can ask storage systems to deliver integrals. This has to be capable of use for a simple filesystem.

John
From: john alan kennedy [mailto:jkennedy at rzg.mpg.de]
Sent: 22 February 2012 09:35
To: Gordon, John (STFC,RAL,ESC)
Cc: mike.jones at manchester.ac.uk; andrea.cristofori at cnaf.infn.it; j.k.nilsen at fys.uio.no; ur-wg at ogf.org
Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET

Hi John,

I understand why you want the allocated resources and personally have nothing against us having an optional field for it.

<ur:StorageResourceCapacityUsed>14728</ur:StorageResourceCapacityUsed>
<ur:StorageResourceCapacityAllocated>50000</ur:StorageResourceCapacityAllocated>

and I agree that this should be in the same record.

I do however feel that a UR should cover a period of time and not just one point in time.

I would argue that it is the job of the sensor or even better the storage system to be able to provide this information.
The UR is there primarily to describe this information which can then be used for accounting/billing.

We talked briefly in the last tele-con about a possible mis-usage of the record for a time duration of "0" which would then be a point in time record. I am starting to think we may need to actually tag the record as Point-in-Time (PiT) or Time-Integral(TI), but that's a secondary thought. The main point is that we could possibly cover both with the same record format... (or we may need two URs.). To be honest i need to think about it some more.

The integral approach just seems more natural if we see accounting/billing as the end goal.
* It fits better with the Compute UR (point-in-time doesn't fit at all).

* You shouldn't need to do any mathematical gymnastics after the fact (no need to consider policies for min/max/avg usage) etc.

* Missing records is something that you may have in Compute URs as well.
I agree that in compute you are in a better situation since the missing records can often (if not always) be generated later since the batch system allow this - the storage systems are just weak here.

It should be the job of the sensor software or storage system to be able to plug any gaps. That's what you do with Compute.

Anyway, I think it's best to have your use-case written and then we can all discuss a bit more - prob best during the tele-con.

cheers
johnk

On 02/21/2012 03:07 PM, john.gordon at stfc.ac.uk<mailto:john.gordon at stfc.ac.uk> wrote:
Mike, you keep coming back to time integration. I think that once we decided on sampling rather than evaluating integrals then all we have is a series of observations and there are a variety of conclusions we can draw from them.  None of them contain the word ‘always’ since we can never know what happens between observations.

Once one has a collection of records then different grids, projects, infrastructures may apply different procedures.  I am not against the durationtime/elapsedtime argument but I won’t be attempting to construct time integrals from it. I will not be treating periods of time not covered by records as unused or empty space. I will be calculating min/max/mean over a time period with perhaps some weighting if the end-user projects or sites feel that this isn’t the full story. I might come back and propose a summary UR for passing this information around. I think that for many people desperate for storage accounting these simple metrics will be enough. If they are not then we can reconvene and change/extend the UR.

John

From: Mike Jones via mobile [mailto:mike.jones at manchester.ac.uk]
Sent: 21 February 2012 10:56
To: Gordon, John (STFC,RAL,ESC); andrea.cristofori at cnaf.infn.it<mailto:andrea.cristofori at cnaf.infn.it>; j.k.nilsen at fys.uio.no<mailto:j.k.nilsen at fys.uio.no>
Cc: ur-wg at ogf.org<mailto:ur-wg at ogf.org>
Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET

To follow up on John's WLCG usecase: I agree that VO space allocated over a period of time _is_ a usage. BUT VO allocation at any instant is not. That said, the dynamics of any allocation/usage needs to be recorded to be able to answer the question "over period X did the allocation always satisfy: instantaneous_storage( used+available) > N (dimension:bytes)?"
John, does that last clause satisfy WLCG?

Mike
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
john.gordon at stfc.ac.uk<mailto:john.gordon at stfc.ac.uk> wrote:
I’ll be in Taiwan next week UTC+8 so I may or may not attend. Depends if anything official is happening that evening and if I can stay awake and if Skype works.

I’ll add the WLCG storage use case to the wiki before then. I must stress that this is important to WLCG and that we consider space allocated to a VO as ‘usage’ as it is not available to others and is to be recorded just like space occupied by files. i.e. Not monitoring any more than recording the existence of files in monitoring.

John

From: ur-wg-bounces at ogf.org<mailto:ur-wg-bounces at ogf.org> [mailto:ur-wg-bounces at ogf.org] On Behalf Of Andrea Cristofori
Sent: 21 February 2012 08:15
To: Jon Kerr Nilsen
Cc: ur-wg at ogf.org<mailto:ur-wg at ogf.org>
Subject: Re: [UR-WG] Actions for next phone meeting on 21.02.2012, 15:00 CET

Dear all,

Sorry for the short notice but neither I or Jon can make it to today phone conference. We would like then to postpone it to next week (same day, same time).
About today actions Jon already published the document and I got an answer for the slot at OGF:

http://www.ogf.org/gf/event_schedule/index.php?id=2413

Again apologies.

Cheers,
Andrea

On 02/21/2012 09:08 AM, Jon Kerr Nilsen wrote:
Hi all,

As John AK just reminded me, the next phone meeting is today. We'll go through the actions mentioned below and possibly make some new ones. Just to try out something new, I suggest we try the Skype conference room in Lund (which is starting to get stable now). So you can call in to

Place: Skype
Skype contact: lund.room.1
Phone bridge: +46462884608

cheers,
Jon

On 7. feb. 2012, at 17:32, Andrea Cristofori wrote:

Dear all,

Most of the phone conference has been focused on the comments on the use cases provided by John AK. We commented directly inline on the wiki. Feel free to add more Use cases if not present. We will comment them during next phone conference. We also agreed in asking for a second time slot during OGF (if possible the same day) and divide the agenda in two parts and JKN will look for information on how to make the document he circulated the 25th public. The actions for next meeting are the following:

- AC: Ask for another session the same day at OGF and update the agenda on OGF website
- JKN: make public the document
- All: write some examples for the different use cases (minima, grid, local, etc.) (Ralph local, JAK Grid/dCache)

Especially for John Gordon: can you please comment on those two use cases:

  1.  As a project I would like to be able to view the used and unused storage space that I have on a storage resource.
- Thus I can see how much headroom I have.
All: seems to be more monitoring Comments from John are required

  1.  As a project I would like to be able to view the requested storage I have on a specific resource and the allocated/reserved resources I have on that resource.
- Thus I can see I asked for 100TB and I currently have only 80TB at my disposal (of which i am using 50TB).
All: seems to be more monitoring Comments from John are required

Cheers
Andrea
--
 ur-wg mailing list
 ur-wg at ogf.org<mailto:ur-wg at ogf.org>
 https://www.ogf.org/mailman/listinfo/ur-wg

--

  ur-wg mailing list

  ur-wg at ogf.org<mailto:ur-wg at ogf.org>

  https://www.ogf.org/mailman/listinfo/ur-wg

--

Andrea Cristofori

INFN-CNAF

Viale Berti Pichat 6/2

40127 Bologna

Italy

Tel. : +39-051-6092920

Skype: andrea-cnaf

--
Scanned by iCritical.

--

  ur-wg mailing list

  ur-wg at ogf.org<mailto:ur-wg at ogf.org>

  https://www.ogf.org/mailman/listinfo/ur-wg

--

+------------------------------------------------------------+

|Dr. John Alan Kennedy          Rechenzentrum Garching (RZG) |

|Mail:  jkennedy at rzg.mpg.de<mailto:jkennedy at rzg.mpg.de>     Boltzmannstrasse 2           |

|Phone: +49 89 3299 2694        85748 Garching               |

|Fax:   +49 89 3299 1301                                     |

+------------------------------------------------------------+
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ogf.org/pipermail/ur-wg/attachments/20120228/26b04dc2/attachment-0001.html>