[UR-WG] Ideas for storage accounting record

[Jon Kerr Nilsen]

Hi,

> AFAIK EMI is very interested in having something usuable very soon (this 
> year) for performing storage accounting, so this is a fairly urgent topic 
> if OGF wants to have something to say. However there is likely to be a 
> certain overlap between the two groups.

Just to confirm that EMI is indeed interested in having a usable definition of a storage accounting record before January as we have promised EU to have definition ready by then. This definition, "reflecting practical, financial and legal requirements of storage location, usage and space and data flow", is then planned to be implemented in DPM, StoRM, dCache, gLite FTS and UNICORE FTS before the end of EMI.

The EMI data group is currently in the process of collecting requirements from user communities and storage providers, so not much input from us yet, but I'm certainly hoping for overlap between OGF and EMI work on this topic.

best regards,
Jon
Storage accounting task leader
EMI data group

On 23. sep. 2010, at 13.16, Henrik Thostrup Jensen wrote:

> Hi
> 
> Here are some preliminary ideas for fields in a storage accounting record. 
> The first target for NDGF is dCache, however we also want be able to do 
> storage accounting on plain unix directories (SGAS has a special client 
> called Bart, which does job accounting on LRMS-level, and we have users 
> who's primary usage for SGAS is regular (non-grid) HPC).
> 
> * record metainformation:
> 
> recordId	unique
> createTime	iso timestamp for the record
> 
> * identity block:
> 
> global uid	global user identity (properly optional, vo is often more interesting)
> vo_issuer
> vo_type
> vo_name (or just vo)
> vo_group
> vo_role
> ## complex vo data? (multiple groups/roles)
> ## having multiple groups/roles for data is somewhat messy
> 
> * storage information:
> 
> sitename	FQDN / Unique sitename
> poolname?	A site will often have several places to store (should probably have another name)
> storagesystem	which storage system is used
> storagetype	ssd / platter disk / tape
> storageclass	pinned / deletable / precious / archival / whatever
> storageurl	url where the storage is located at
> 
> * time period information:
> 
> starttime	startime for when the record holds
> endtime		endtime for when the record holds
> 
> * space usage
> 
> spaceused	space used by the identity (bytes is probably best)
> spacefree	optional (number is not always available)
> spacereserved	space reserved for the identity block (optional)
> 
> 
> Design issues:
> 
> should everything be split into seperate records (say one for disk, one 
> for tape, and so on), or should be possible to specify multiple usage 
> sections in storage accounting record for different classes of stored 
> data.
> 
> AFAIK EMI is very interested in having something usuable very soon (this 
> year) for performing storage accounting, so this is a fairly urgent topic 
> if OGF wants to have something to say. However there is likely to be a 
> certain overlap between the two groups.
> 
> 
>     Best regards, Henrik
> 
>  Software Developer, Henrik Thostrup Jensen <htj at ndgf.org>
>  Nordic Data Grid Facility. WWW: www.ndgf.org
> --
>  ur-wg mailing list
>  ur-wg at ogf.org
>  http://www.ogf.org/mailman/listinfo/ur-wg