[occi-wg] confusion about status of link / headers

Sam Johnston samj at samj.net
Tue Oct 20 06:00:52 CDT 2009 

On Tue, Oct 20, 2009 at 10:49 AM, Alexis Richardson <
alexis.richardson at gmail.com> wrote:

Please don't throw around words like "bunk" willy nilly.
>

I tell you what... I'll resist the urge to declare your arguments "bunk" if
you resist the urge to declare my proposals "experimental", "new
technology", not "prior art" and whatever other weasel words are being
tossed around to imply that they are somehow risky. To quote McNealy for the
second time today “If everybody believes in your strategy you have zero
chance of [success]" - change that matters is always controversial.


> I have just had a look at
> http://www.rackspacecloud.com/cloud_hosting_products/servers/api
>
> I can see some limited use of headers for metadata in a few places.
> That's never been controversial.  But the understanding that I have of
> your proposal is to use headers wholesale for all metadata.  Rackspace
> don't appear to do that, unless I have misunderstood their document.
>

Rackspace have taken on the creation of both a format and a protocol
combined (note the absence of renderings like OVF). That's wrong for many
reasons, not the least of which include creating significant extra work for
yourself, not playing nice with other standards, raising the bar for
implementers on both sides of the interface, creating more opportunities for
interoperability problems... I could go on. However it suited them well
because their task was to expose the relatively limited functionality of a
single implementation, not to satisfy the needs of many. Also, whether they
use headers for a single attribute or all of them is irrelevant - it still
proves headers are safe for such interfaces.

We made a conscious decision early on to embrace existing standards such as
OVF and I've provided a way for us to allow them to coexist and compete on
merit by steering well clear of the HTTP entity-body. The non-Atom
alternative requires us to follow their example by developing both protocol
and format, with the additional hurdle that our format must also be
compatible and compete with others like OVF. It's a mess - I know because I
spent quite some time trying to make it work. Without changing (e.g.
complicating) the model you've got different representations for the same
resource telling different stories which is just plain wrong.

In any case, cloud storage services including both Azure and S3 (which can
certainly be considered cloud infrastructure) do use HTTP headers wholesale
for metadata. HTTP intends for headers to be used for metadata too, and
provides for arbitrary extension to suit.

I also may have misunderstood your proposal.  If nobody understands
> your proposal except you, then it will not be possible to gain
> consensus around it.
>

If indeed people don't understand the proposal then that *is* something we
need to work on. It's very intuitive for anyone who has worked with web
technologies:

   - Instances of objects we model are "resources" which are abstract &
   can't be retrieved directly (e.g. you can't pull a physical machine over an
   API, yet)
   - You can however GET "representations" of resources in one or more
   existing standard formats (e.g. OVF, VMDK, etc.)
   - Some representations will be better than others at telling the "full
   story" while others may be quite lossy (ala JPEG).
   - The non-Atom alternative is to create an extremely lossy representation
   that must be retrieved in addition to the "native" representation (e.g.
   OVF), which includes information not available in other representations, and
   which may conflict with other representation e.g. current resources
   allocated (which may be very dynamic and differ from the saved state).
   - Metadata including the organisation of the resources into categories
   and an interconnected graph, as well as owner information, related
   resources, possible state transitions, etc. needs to be captured somewhere.
   - HTTP provides a perfectly good metadata channel in the form of HTTP
   headers, whereby a single CRLF character (e.g. a blank line) delineates the
   metadata from the data itself.
   - HTTP also provides a convenient way to access the metadata alone (e.g.
   HTTP HEAD). This is useful for tasks like enumeration/discovery which don't
   require clients to have the representation itself.
   - HTTP also provides many optimisations for common tasks such as COPYing
   templates and MOVEing resources between clouds.

Basically my proposal is to, wherever possible, use HTTP exactly as it was
originally intended, deviating only where we absolutely have to.

Are you saying "OCCI metadata should be more like Rackspace metadata"?
>

No, just that Rackspace (among others) validate the use of HTTP headers in
cloud APIs.

Sam

 On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston <samj at samj.net> wrote:
> > On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson
> > <alexis.richardson at gmail.com> wrote:
> >>
> >> What about compute clouds?
> >
> > Rackspace Cloud API for a start:
> >
> >> HTTP/1.1 204 No Content
> >> Date: Mon, 12 Nov 2007 15:32:21 GMT
> >> Server: Apache
> >> X-Server-Management-Url:
> https://servers.api.rackspacecloud.com/v1.0/35428
> >> X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4
> >> X-CDN-Management-Url: https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4
> >> X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb
> >> Content-Length: 0
> >> Content-Type: text/plain; charset=UTF-8
> >
> >
> > Microsoft Azure uses headers for "attributes" as well, in the same way as
> I
> > had originally proposed:
> >
> > x-ms-meta-name: value     Returns a metadata value for the container.
> >
> > However prefer that the header names are static/opaque rather than using
> a
> > template and parsing them:
> >
> >> Attribute: name=value
> >
> > This to me is cleaner and more self-explanatory, plus it is easily
> collapsed
> > ala:
> >
> >> Attribute: name1=value1, name2=value2, ... namen=valuen
> >
> > Anyway suffice to say that claims this is somehow "experimental" are
> bunk.
> >
> > Sam
> >
> >> On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian at jclouds.org>
> wrote:
> >> > Here are options for metadata used in some of the major storage clouds
> >> > FWIW:
> >> >
> >> > S3, Rackspace, EMC Atmos, Azure - Headers
> >> > Nirvanix - query params in, xml entity out
> >> > Mezeo - entity
> >> >
> >> > Of the ones using headers, S3, Rackspace and Azure use prefix with
> >> > values stored as-us.  Atmos joins all metadata together into one
> >> > header, making parsing trivial (split /,/), but necessary.
> >> >
> >> > The most expensive option of the above is entity, where each metadata
> >> > value is a separate GET.  However, entities allow binary metadata and
> >> > zero restrictions on it, which may be useful.
> >> >
> >> > In jclouds, we time parsing of response values.  A simple XML doc with
> >> > only several elements written in SAX takes a few ms to parse.  My log
> >> > files are not precise enough to find the overhead in parsing headers:
> >> > they always start and finish within the same millisecond.
> >> >
> >> > I hope this background helps, and also helps explain why I'm vocal on
> >> > such topics such as headers vs entities :)
> >> >
> >> > Cheers,
> >> > -Adrian
> >> >
> >> >
> >> > On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston <samj at samj.net> wrote:
> >> >> On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro
> >> >> <garymazzaferro at gmail.com>
> >> >> wrote:
> >> >>
> >> >>> The http header and key/value pairs need to parsed also, there is no
> >> >>> free
> >> >>> ride here.
> >> >>
> >> >> Every HTTP library I have ever used parses HTTP headers and puts them
> >> >> in a
> >> >> nice hash for you ready to consume. If we go for "Name: Value" then
> >> >> that's
> >> >> all there is to it. If we go for "Attribute: name=value" as is
> >> >> currently
> >> >> proposed (which is arguably cleaner, follows cookies' "prior art" and
> >> >> avoids
> >> >> Amazon's prefix hack) then you just have to split on '='.
> >> >>
> >> >> To illustrate how clean this is by example:
> >> >>
> >> >>> #!/usr/bin/python
> >> >>> import urllib2
> >> >>> response = urllib2.urlopen('http://cloud.example.com/myvm')
> >> >>> representation = response.read()
> >> >>> metadata = response.info()
> >> >>> print metadata['occi-compute-cores']
> >> >>
> >> >> As soon as you start talking about payloads you have to fire up a
> >> >> parser
> >> >> (JSON/XML/Atom/etc.) or write your own (previous text rendering)
> which
> >> >> is
> >> >> significantly more work to do at both design and run times. Not to
> >> >> mention
> >> >> more work for us to do now and more scope for interoperability
> >> >> problems.
> >> >>
> >> >> Sam
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> occi-wg mailing list
> >> >> occi-wg at ogf.org
> >> >> http://www.ogf.org/mailman/listinfo/occi-wg
> >> >>
> >> >>
> >> > _______________________________________________
> >> > occi-wg mailing list
> >> > occi-wg at ogf.org
> >> > http://www.ogf.org/mailman/listinfo/occi-wg
> >> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ogf.org/pipermail/occi-wg/attachments/20091020/7846fea3/attachment.html

More information about the occi-wg mailing list