[UR-WG] Storage Accounting Discussion

[Andrea Cristofori]

Hi John,

some comment


Il 29/11/2011 11:52, john.gordon at stfc.ac.uk ha scritto:
> I was actioned to provide some text for discussion about the issue I raised at the Lyon UR-WG meeting. I am having problems logging in to GridForge right now so I am sending this round so that people see it before today's call. I am travelling this afternoon so I will need to find wireless to use Skype unless someone can advise me on calling in from a physical phone.
>
> Regards,
>
> John
>
> Publishing Allocated Space:
>
> The current StAR proposal only records the 'usage' of storage - ie the space taken up by user files. I propose that it should also contain a field containing the total size of the storage area under consideration. To support this I draw a parallel with the cpu UR where both cpu and wallclock are recorded. The record thus shows the effect of a job on the resource - how long it occupied the node - and the actual usage it made during this time.

The parallel with CPU for me is more like CPU that are in use because a 
job is running on them and CPU that are not used (no job submitted on 
that node). In this sense UR for CPU accounting does not supportit.

> In the storage case the corresponding data is the total disk space allocated to a user/VO/whatever and the amount filled by files.
>
> I believe there is a use case for charging for allocated space, not used space, so these data need collecting.

I agree with the case of reserved storage that in fact is not used but 
it is/should be not available.
Allocated vs. real usage of the storare I think is an issue that is 
difficult to deal with because because the allocation of the storage 
depends on the file system used, block size, etc. and maybe it is not 
even correct to account more space than what is really used (a small 
file in a storage with huge blocks wastes space because it allocates 
more space than it really use. But this I think is a problem of the site 
that should tailor the storage configuration to the file that are being 
stored)

> One solution would be to have two flavours of StAR: one for total, one for used, not necessarily cut at the same time or frequency. I prefer an additional field in a single StAR just to ensure that measurements are correlated and accounting systems don't have to spend time/effort in matching them.

My opinion is that total space should not be part of a Usage Record. 
Again taking the parallel with the CPU would be to put in the UR the 
number of available CPU/cores on the site.

Andrea