[occi-wg] confusion about status of link / headers

Adrian Cole adrian at jclouds.org
Tue Oct 20 11:14:58 CDT 2009 

I don't think we should create an interface that is innately limiting
in the name of "mainstream."  I've been told more then once that OCCI
is currently for compute, but is made for other services.  I argue
that scale is a relevant concern, unless we feel like having these
same discussions later, in haste, and in reaction to breaking points.

I agree that start/stops don't create a high burden of requests: You
could probably write a crappy cgi script and it would still perform
well enough.  However, state pings, metadata queries, etc, will
certainly happen in some frequency.  Anything we do in the future, by
extension or otherwise, will cause more traffic and higher need for
scale.

I don't think history will condemn us for either pragmatically
considering efficiency or using headers.

my 2p.
-Adrian


On Tue, Oct 20, 2009 at 7:51 AM, Gary Mazz <garymazzaferro at gmail.com> wrote:
> I've understood Sam's proposal from the beginning. I have had issue with
> from the beginning.
>
> Although I find it very interesting and valuable in terms of economy, I
> have issue with it as the OCCI's primary interface implementation for a
> few reasons which have been outlined in several other emails.
>
> It appears that Sam has created a new technology, a lower cost RPC
> mechanism, which is unproven for both small and wide scale deployment.
> For interoperability, IMO,  occi should focus on proven mainstream
> technologies as the primary interfaces. I am not suggesting, shelving or
> not supporting Sam's proposal, based on market acceptance, it may prove
> to be one of many occi interface implementations.
>
> As for cost saving in interface technologies, I do not buy into the idea
> that the occi traffic will be so overwhelming that there is a
> requirement to maximally optimize http requests and responses. IMO, in
> comparison to images and consumer data occi traffic will be
> insignificant. I think we are worrying about spilled milk while the
> house is burning.
>
> There has been some discussion on supporting technologies including OVF.
> Whether occi supports OVF has not been fully discussed, examined or any
> consensus reached. I believe adopting technologies like OVF will only
> lead to mapping issues between occi attributes and the external formats
> creating a long term management headache.
>
> cheers,
> gary
>
>
> Alexis Richardson wrote:
>> OK
>>
>> Is Sam's statement something that we can achieve consensus around?
>>
>> a
>>
>>
>> On Tue, Oct 20, 2009 at 12:47 PM, Sam Johnston <samj at samj.net> wrote:
>>
>>> Alexis,
>>>
>>> Seems our posts crossed in the mail. I hope the last one clears things up
>>> but the fact that you're using OVF and XML/JSON interchangeably suggests
>>> that you [are possibly one of many who] don't grok the proposal. Worse, it
>>> seems you've got it back to front in thinking of the "core" as metadata,
>>> which is not at all what I had intended.
>>>
>>> The focus is on the resources and their various representations. The
>>> metadata (that is, data about data) essentially provides whatever is
>>> necessary to "glue" these resources together. It is basically using HTTP as
>>> the meta-model rather than an envelope format like Atom/SOAP or, worse, a
>>> meta-model that we have to create ourselves from scratch.
>>>
>>> So it's more like:
>>>
>>> 1) OCCI is compatible with ALL existing formats, eg OVF
>>>
>>> 2) Existing formats are used are used for ALL representations
>>>
>>> 3) ALL additional metadata should be in the headers
>>>
>>> 4) ALL of the core stuff should be in the body (that is, the metadata in the
>>> headers is just there to support the data and wire it up in useful ways)
>>>
>>> I think the confusion stems from your assuming that there is still
>>> XML/JSON/Atom/etc. representations. There aren't. HTTP headers provide a
>>> single, performant, interoperable metadata channel that has already been
>>> both standardised and extensively implemented.
>>>
>>> If you've followed me this far then you should have realised that every HTTP
>>> user agent (e.g. Perl's LWP, Python's urllib2, C's libcurl, etc.) is already
>>> a basic OCCI client out-of-the-box. The result is that 5-line scripts like
>>> demo.py are functional with no dependence on client libraries (that would
>>> absolutely be necessary if you were to start trying to define your own
>>> formats).
>>>
>>> Sam
>>>
>>> PS I'm sorry my "bunk" comment came across as referring to you personally -
>>> it was directed at the argument.
>>>
>>> On Tue, Oct 20, 2009 at 12:44 PM, Alexis Richardson
>>> <alexis.richardson at gmail.com> wrote:
>>>
>>>> I think what Sam is proposing is the following:
>>>>
>>>> 1) If OCCI can handle 'any' data representation, eg OVF, XML, JSON,
>>>> then it needs some core / common model.  Otherwise: there can be no
>>>> defined behaviour / interop
>>>>
>>>> 2) This core / common model is ALL metadata
>>>>
>>>> 3) ALL this metadata should be in the headers
>>>>
>>>> 4) ALL the non core stuff eg OVF payload, XML representation, should be in
>>>> body
>>>>
>>>> Sam - is that right?
>>>>
>>>> a
>>>>
>>>>
>>>> On Tue, Oct 20, 2009 at 11:39 AM, Sam Johnston <samj at samj.net> wrote:
>>>>
>>>>> On Tue, Oct 20, 2009 at 12:28 PM, Edmonds, AndrewX
>>>>> <andrewx.edmonds at intel.com> wrote:
>>>>>
>>>>>> To that point I'd ask - If we're talking about meta-data that infers
>>>>>> there
>>>>>> is some data to accompany it. Where are the examples of this data? This
>>>>>> would help in forming the full picture.
>>>>>>
>>>>> Here, from the core spec (using OVF as an example, but could be
>>>>> anything).
>>>>> Request is green, response metadata is yellow & response data/payload is
>>>>> orange:
>>>>>
>>>>>
>>>>>> GET /us-east/webapp/vm01 HTTP/1.1
>>>>>> User-Agent: occi-client/1.0 (linux) libcurl/7.19.4 OCCI/1.0
>>>>>>
>>>>>> Host: cloud.example.com
>>>>>> Accept: */*
>>>>>>
>>>>> < HTTP/1.1 200 OK
>>>>> < Date: Sat, 10 Oct 2009 12:56:51 GMT
>>>>>
>>>>> < Content-Type: application/ovf
>>>>> < Link: </us-east/webapp/vm01;start>;
>>>>>
>>>>> <       rel="http://purl.org/occi/action/start";
>>>>> <       title="Start"
>>>>>
>>>>> < Link: </us-east/webapp/build.pdf>;
>>>>> <       rel="related";
>>>>>
>>>>> <       title="Documentation";
>>>>> <       type="application/pdf"
>>>>>
>>>>> < Category: compute;
>>>>> <       label="Compute Resource";
>>>>>
>>>>> <       scheme="http://purl.org/occi/kind/"
>>>>> < Server: occi-server/1.0 (linux) OCCI/1.0
>>>>>
>>>>> < Connection: close
>>>>> <
>>>>> < <?xml version="1.0" encoding="UTF-8"?>
>>>>>
>>>>> < <Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>> <           xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1"
>>>>>
>>>>> <           xmlns="http://schemas.dmtf.org/ovf/envelope/1"
>>>>> <           xml:lang="en-US">
>>>>>
>>>>> ...
>>>>>
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: occi-wg-bounces at ogf.org [mailto:occi-wg-bounces at ogf.org] On
>>>>>> Behalf
>>>>>> Of Alexis Richardson
>>>>>> Sent: 20 October 2009 09:49
>>>>>> To: Sam Johnston
>>>>>> Cc: Tim Bray; occi-wg at ogf.org
>>>>>> Subject: Re: [occi-wg] confusion about status of link / headers
>>>>>>
>>>>>> Sam,
>>>>>>
>>>>>> Please don't throw around words like "bunk" willy nilly.
>>>>>>
>>>>>> I have just had a look at
>>>>>> http://www.rackspacecloud.com/cloud_hosting_products/servers/api
>>>>>>
>>>>>> I can see some limited use of headers for metadata in a few places.
>>>>>> That's never been controversial.  But the understanding that I have of
>>>>>> your proposal is to use headers wholesale for all metadata.  Rackspace
>>>>>> don't appear to do that, unless I have misunderstood their document.
>>>>>> I also may have misunderstood your proposal.  If nobody understands
>>>>>> your proposal except you, then it will not be possible to gain
>>>>>> consensus around it.
>>>>>>
>>>>>> Are you saying "OCCI metadata should be more like Rackspace metadata"?
>>>>>>
>>>>>> alexis
>>>>>>
>>>>>>
>>>>>> On Tue, Oct 20, 2009 at 9:07 AM, Sam Johnston <samj at samj.net> wrote:
>>>>>>
>>>>>>> On Tue, Oct 20, 2009 at 9:55 AM, Alexis Richardson
>>>>>>> <alexis.richardson at gmail.com> wrote:
>>>>>>>
>>>>>>>> What about compute clouds?
>>>>>>>>
>>>>>>> Rackspace Cloud API for a start:
>>>>>>>
>>>>>>>
>>>>>>>> HTTP/1.1 204 No Content
>>>>>>>> Date: Mon, 12 Nov 2007 15:32:21 GMT
>>>>>>>> Server: Apache
>>>>>>>> X-Server-Management-Url:
>>>>>>>> https://servers.api.rackspacecloud.com/v1.0/35428
>>>>>>>> X-Storage-Url: https://storage.clouddrive.com/v1/CloudFS_9c83b-5ed4
>>>>>>>> X-CDN-Management-Url:
>>>>>>>> https://cdn.clouddrive.com/v1/CloudFS_9c83b-5ed4
>>>>>>>> X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb
>>>>>>>> Content-Length: 0
>>>>>>>> Content-Type: text/plain; charset=UTF-8
>>>>>>>>
>>>>>>> Microsoft Azure uses headers for "attributes" as well, in the same
>>>>>>> way
>>>>>>> as I
>>>>>>> had originally proposed:
>>>>>>>
>>>>>>> x-ms-meta-name: value     Returns a metadata value for the container.
>>>>>>>
>>>>>>> However prefer that the header names are static/opaque rather than
>>>>>>> using
>>>>>>> a
>>>>>>> template and parsing them:
>>>>>>>
>>>>>>>
>>>>>>>> Attribute: name=value
>>>>>>>>
>>>>>>> This to me is cleaner and more self-explanatory, plus it is easily
>>>>>>> collapsed
>>>>>>> ala:
>>>>>>>
>>>>>>>
>>>>>>>> Attribute: name1=value1, name2=value2, ... namen=valuen
>>>>>>>>
>>>>>>> Anyway suffice to say that claims this is somehow "experimental" are
>>>>>>> bunk.
>>>>>>>
>>>>>>> Sam
>>>>>>>
>>>>>>>
>>>>>>>> On Tue, Oct 20, 2009 at 6:19 AM, Adrian Cole <adrian at jclouds.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Here are options for metadata used in some of the major storage
>>>>>>>>> clouds
>>>>>>>>> FWIW:
>>>>>>>>>
>>>>>>>>> S3, Rackspace, EMC Atmos, Azure - Headers
>>>>>>>>> Nirvanix - query params in, xml entity out
>>>>>>>>> Mezeo - entity
>>>>>>>>>
>>>>>>>>> Of the ones using headers, S3, Rackspace and Azure use prefix with
>>>>>>>>> values stored as-us.  Atmos joins all metadata together into one
>>>>>>>>> header, making parsing trivial (split /,/), but necessary.
>>>>>>>>>
>>>>>>>>> The most expensive option of the above is entity, where each
>>>>>>>>> metadata
>>>>>>>>> value is a separate GET.  However, entities allow binary metadata
>>>>>>>>> and
>>>>>>>>> zero restrictions on it, which may be useful.
>>>>>>>>>
>>>>>>>>> In jclouds, we time parsing of response values.  A simple XML doc
>>>>>>>>> with
>>>>>>>>> only several elements written in SAX takes a few ms to parse.  My
>>>>>>>>> log
>>>>>>>>> files are not precise enough to find the overhead in parsing
>>>>>>>>> headers:
>>>>>>>>> they always start and finish within the same millisecond.
>>>>>>>>>
>>>>>>>>> I hope this background helps, and also helps explain why I'm vocal
>>>>>>>>> on
>>>>>>>>> such topics such as headers vs entities :)
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> -Adrian
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Oct 19, 2009 at 4:21 PM, Sam Johnston <samj at samj.net>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> On Tue, Oct 20, 2009 at 12:57 AM, gary mazzaferro
>>>>>>>>>> <garymazzaferro at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> The http header and key/value pairs need to parsed also, there
>>>>>>>>>>> is
>>>>>>>>>>> no
>>>>>>>>>>> free
>>>>>>>>>>> ride here.
>>>>>>>>>>>
>>>>>>>>>> Every HTTP library I have ever used parses HTTP headers and puts
>>>>>>>>>> them
>>>>>>>>>> in a
>>>>>>>>>> nice hash for you ready to consume. If we go for "Name: Value"
>>>>>>>>>> then
>>>>>>>>>> that's
>>>>>>>>>> all there is to it. If we go for "Attribute: name=value" as is
>>>>>>>>>> currently
>>>>>>>>>> proposed (which is arguably cleaner, follows cookies' "prior art"
>>>>>>>>>> and
>>>>>>>>>> avoids
>>>>>>>>>> Amazon's prefix hack) then you just have to split on '='.
>>>>>>>>>>
>>>>>>>>>> To illustrate how clean this is by example:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> #!/usr/bin/python
>>>>>>>>>>> import urllib2
>>>>>>>>>>> response = urllib2.urlopen('http://cloud.example.com/myvm')
>>>>>>>>>>> representation = response.read()
>>>>>>>>>>> metadata = response.info()
>>>>>>>>>>> print metadata['occi-compute-cores']
>>>>>>>>>>>
>>>>>>>>>> As soon as you start talking about payloads you have to fire up a
>>>>>>>>>> parser
>>>>>>>>>> (JSON/XML/Atom/etc.) or write your own (previous text rendering)
>>>>>>>>>> which
>>>>>>>>>> is
>>>>>>>>>> significantly more work to do at both design and run times. Not
>>>>>>>>>> to
>>>>>>>>>> mention
>>>>>>>>>> more work for us to do now and more scope for interoperability
>>>>>>>>>> problems.
>>>>>>>>>>
>>>>>>>>>> Sam
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> occi-wg mailing list
>>>>>>>>>> occi-wg at ogf.org
>>>>>>>>>> http://www.ogf.org/mailman/listinfo/occi-wg
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> occi-wg mailing list
>>>>>>>>> occi-wg at ogf.org
>>>>>>>>> http://www.ogf.org/mailman/listinfo/occi-wg
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> occi-wg mailing list
>>>>>> occi-wg at ogf.org
>>>>>> http://www.ogf.org/mailman/listinfo/occi-wg
>>>>>> -------------------------------------------------------------
>>>>>> Intel Ireland Limited (Branch)
>>>>>> Collinstown Industrial Park, Leixlip, County Kildare, Ireland
>>>>>> Registered Number: E902934
>>>>>>
>>>>>> This e-mail and any attachments may contain confidential material for
>>>>>> the sole use of the intended recipient(s). Any review or distribution
>>>>>> by others is strictly prohibited. If you are not the intended
>>>>>> recipient, please contact the sender and delete all copies.
>>>>>>
>>>>>
>>>
>> _______________________________________________
>> occi-wg mailing list
>> occi-wg at ogf.org
>> http://www.ogf.org/mailman/listinfo/occi-wg
>>
>>
>
> _______________________________________________
> occi-wg mailing list
> occi-wg at ogf.org
> http://www.ogf.org/mailman/listinfo/occi-wg
>

More information about the occi-wg mailing list