[glue-wg] Extending GLUE 2.0 for Cloud services

[Paul Millar]

Hi Salvatore,

I think your work is very interesting: GLUE-2 is trying to be extensible 
and technology agnostic.  Trying to describe cloud services as GLUE-2 
objects is an excellent test of how well we've achieved that goal.

Some further replies, in-line:

On 05/11/13 10:05, Salvatore Pinto wrote:
> we considered this option, but, for Compute, we have also some change in
> the relationships between the objects and mandatory attributes in the
> Grid elements which have no sense in the Cloud world.

If there are attributes that are mandatory which make no sense in other 
kinds of services then we should certainly consider relaxing those 
requirements.

Relationships between objects should also be expressible as extensions 
without requiring schema changes.  If this isn't the case (for whatever 
reason) then that, too, is something that should be fixed.


> Anyway, the main reason for not using the extension was that,
> considering the different kind of services the Cloud and the Grid are
> giving to the users, we wanted to keep the entities separated.

This, I think, is the point which I really disagree with.

Let me give you a concrete example.

We (dCache.org) have just started running a dCache instance that is very 
much a cloud storage service.  Client software (of which there are 
various) currently accessed it via WebDAV, but we anticipate adding 
support for CDMI.  We want to make this storage (or a similar cloud 
service) available to grid users in the future.

IMHO, we *will* have storage systems that provide cloud-like and 
grid-like services; it would be a good idea not to exclude this from the 
start.

[...]
> CDMI metadata are mostly related to file-level options, for example
> ACLs, file redundancy, file encryption, etc... (source:
> http://cdmi.sniacloud.com/cdmi_spec/16-metadata/16-metadata.htm).

Sorry, I didn't explain myself clearly; what you're describing is the 
CDMI concept of metadata (as per Chapter 16).  I meant the more general 
metadata: "data about something".

The CDMI protocol also supports a client discovering information about 
the service itself; the main motivation is to allow the client to 
discover if a service is appropriate for its needs.  This matches one of 
the core uses of GLUE-2: to select an appropriate storage services 
without querying each service individually.

For examples of this, see section 6.2 "Discovering the Capabilities of a 
Cloud Storage" and section 12.2 "Cloud Storage System-Wide Capabilities".

The "Exported Protocols" (see Section 13) is broadly similar to the 
concept of StorageEndpoints or StorageAccessProtocol in GLUE-2, so this 
would also be an interesting avenue for mapping.

There's also explicit support in CDMI for representing links to OCCI 
services.  This holds a similar role to the ToComputingService objects 
in GLUE-2.

(you see how these two worlds are not really *that* different)


> again, one point for modifying the original Storage elements to consider
> objects instead of files.

IMHO this isn't a good reason.

In most cases "objects" ("data-objects" in CDMI) are really just a 
synonym for files and a "buckets" ("container" in CDMI) is a synonym for 
a directory.

Sure, in S3, there are some limitations on the interactions, that's an 
implementation-specific detail.  If those limitations are important to 
clients then we can describe them (as per CDMI).

Whether GLUE-2 talks about 'files' or 'objects' (or Feile, Fichier or 
ファイル) really doesn't matter if the underlying concept is the same.

HTH,

Paul.