[glue-wg] Some thoughts on storage objects

[Maarten.Litmaath at cern.ch]

Hi Paul,
various comments inline.

> [...]
> 
> BTW, I'm implicitly assuming that StorageEnvironment.RetentionPolicy can be 
> multivalued.  If this isn't true and we have the use-case of the same 
> physical disks being part of, for example, both Custodial and Output storage, 
> then it starts to get complicated.

I think it is OK if the RetentionPolicy _can_ be multivalued, but in WLCG
it would be published single-valued, viz. along with an AccessLatency to
describe the Storage Class that is implemented by the Environment.

> [...]
> 
> StorageEnvironment:
> 
>  A StorageEnvironment is a collection of one or more StorageCapacities
>  with a set of associated (enforced) storage management policies.
>  Examples of these policies are Type (Volatile, Durable, Permanent)
>  and RetentionPolicy (Custodial, Output, Replica).

Note that we should get rid of the obsolete, confusing Type and Lifetime
attributes in favor of the ExpirationMode copied from SRM v3.

> [...]
> 
> StorageResource:
> 
>  A StorageResource is an aggregation of one or more
>  StorageEnvironments and describes the hardware that a particular
>  software instance has under its control.

See my reply to Sergio: we may rather want to allow an Environment
to be linked to multiple Resources, e.g. a disk and a tape Resource,
such that we can publish the back-end implementation name and version
for each of them.

>  A StorageResource must have at least one StorageEnvironment,
>  otherwise there wouldn't be much point publishing information
>  about it. [This isn't a strict requirement, but I think it makes sense
>  to include it.]

OK.

>  All StorageEnvironments must be part of precisely one
>  StorageResource.  SoftwareEnvironments may not be shared between
>  StorageResources.  This means that all physics hardware must
>  be published under precisely one StorageResource.

See my comment above.

> StorageShare:
> 
>  A StorageShare is a logical partitioning of one or more
>  StorageEnvironments.
> 
>  Perhaps the simplest example of a StorageShare is one
>  associated with a single StorageEnvironment with a single
>  associated StorageCapacity, and that represents all
>  the available storage of that StorageCapacity.  An example of
>  a storage that could be represented by this trivial
>  StorageShare is the classic-SE.
> 
>  StorageSpaces must have one or more associated StorageCapacities.
|         ^^^^^^
|         Shares

>  These StorageCapacities provide a complete description of the different
>  homogeneous underlying technologies that are available under the space.
> 
>  In general, the number of StorageCapacities associated with a
>  StorageShare is the sum of the number of StorageCapacities associated
>  with each of the StorageShare's associated StorageEnvironments.
> 
>  Following from this, there is an implicit association between the
>  StorageCapacity associated with a StorageShare and the corresponding
>  StorageCapacity associated with a StorageEnvironment.  Intuitively, this
>  association is from the fact that the two StorageCapacities share the
>  same underlying physical storage.  This implicit association is not
>  recorded in GLUE.
> 
>  StorageSpaces may overlap.  Specifically, given a StorageCapacity
|         ^^^^^^
|         Shares

>  (SC_E) that is associated with some StorageEnvironment and which has
>  totalSize TS_E, let the sum of the totalSize attributes for all
|                  ^^^
|                  let TS_S be .....

>  StorageCapacities that are:
> 	1. associated with a StorageSpace, and
|                                   ^^^^^
|                                   Share

> 	2. that are implicitly associated with SC_E
>  be TS_S.  If the StorageSpaces are covering then TS_S = TS_E.  If
|  ^^^^^^^                 ^^^^^^
|  XXXXXXX                 Shares

>  the StorageSpaces overlap, then TS_S > TS_E.
|             ^^^^^^
|             Shares

>   [sorry, I couldn't easily describe this with just words without it sounding
>   awful!]
> 
>  StorageSpaces may be incomplete.  Following the same definitions
|         ^^^^^^
|         Shares

>  as above, this is when TS_S < TS_E.  Intuitively, this happens if
>  the site-admin has not yet assigned all available storage.
> 
>  End-users within a UserDomain may wish to store or retrieve files.  The
>  StorageShares provides a complete, abstract description of the
>  underlying storage at their disposal.  No member of a UserDomain may
>  interact with the physical hardware except through a StorageShare.
> 
>  The partitioning is persistent through file creation and deletion.  The
>  totalSize attributes (of a StorageSpace's associated StorageCapacties)
|                                    ^^^^^
|                                    Share

>  do not change as a result of file creation or deletion.  [Does GLUE need to
>  stipulate this, or should we leave this vague?]

Why mention it at all?  You do not make statements about the behavior
of the other sizes, and I think there is no need to go there...

>  A single StorageShare may allow multiple UserDomains to access
>  storage; if so, the StorageShare is "shared" between the different
>  UserDomains.  Such a shared StorageShare is typical if a site
>  provides storage described by the trivial StorageShare (one that
>  covers a complete StorageEnvironment) whilst supporting multiple
>  UserDomains.
> 
> [...]
> 
> StorageAccessProtocol:
> 
>  A StorageAccessProtocol describes how data may be sent or received.
>  The presence of a StorageAccessProtocol indicates that data may be
>  fetched or stored using this interface.
> 
>  Access to the interface may be localised; that is, only available
>  from certain computers.  It may also be restricted to specified
>  UserDomains.  However, neither policy restrictions are published in
>  GLUE.  On observing a StorageAccessProcol, one may deduce only that
>  it is valid for at least one user of one supported UserDomain.

..... from at least one computer.

Thanks,
	Maarten