[occi-wg] Is OCCI the HTTP of Cloud Computing?

Tue May 5 14:58:18 CDT 2009

Ah ok now that makes things a little clearer.
I saw Benjamin’s (@benjaminblack) concerns regarding the usage of atom but if we are using it as the metamodel then it only adds more rigor to our effort. On the other hand we are dependent on the atom model. If adopting atom we need to take our nouns apply the atom metamodel to them. I’ve a simple little metamodel that I was playing about which could provide a means for our atom-based noun model (attached). It is most basic but follows some aspects such as OVF and Sun API. Perhaps that could serve as a starting point?

If we settle on the model (how nouns are related) then I don’t see any real issue in the presentation of that model via various renderings (here we could use content negotiation) like text, json and xml (although a rendering of atom-based json might look ugly – perhaps JSONSchema [1] might be of use here? As for text – no idea, yet ☺).

Andy

[1] http://www.json.com/json-schema-proposal

From: Sam Johnston [mailto:samj at samj.net]
Sent: 05 May 2009 12:23
To: Edmonds, AndrewX
Cc: occi-wg at ogf.org
Subject: Re: [occi-wg] Is OCCI the HTTP of Cloud Computing?

Andy,

Thanks for the feedback. I'm on the road so I'll be concise for a change.
On Tue, May 5, 2009 at 12:16 PM, Edmonds, AndrewX <andrewx.edmonds at intel.com<mailto:andrewx.edmonds at intel.com>> wrote:
By introducing Atom, are you proposing it to be the meta-model of OCCI? If it is the meta-model then we should define relationships at a model level as atom will not do this for us. These relationships are not detailed in the Noun/Verb/Attribute page.

That's the idea - it gives us just enough structure (but no more) and while the proposal is not without contention<http://samj.net/2009/05/is-occi-http-of-cloud-computing.html#comments>, I am unconvinced that text and/or json alone will meet the (at times esoteric) needs of the enterprise customers I represent.

Agreed, we will need to have a simple model showing compute resources having zero or more network and storage resources as well as network (and storage?) interconnects. It may be interesting/necessary to stray into the territory of these resources having parents too but we're already starting to stray onto SNIA turf there (better to have them extend OCCI too if we can).

How does it differ by choosing the approach of SUN/Vmware/Elastichosts/GoGrid?
VMware appear to be going for OVF over RESTful HTTP, possibly with some kind of XML wrapping. That's not a bad idea but it's somewhat overkill - I came up with it myself independently for OCCI but dropped it based on feedback from members of this group.

Sun do something similar<http://kenai.com/projects/suncloudapis/pages/HelloCloud> with JSON (likely becuase of the Q-Layer vintage where the API was consumed by a web interface) but that fails the "sysadmin friendly" test and as soon as you need to invoke a library it doesn't matter whether it's JSON or ASN.1 on the wire :)

GoGrid have a query API<http://wiki.gogrid.com/wiki/index.php/API:Anatomy_of_a_GoGrid_API_Call> that will talk JSON, XML and CSV and ElasticHosts ran with<http://www.elastichosts.com/products/api> space separated attribute/value pairs. They also give advice<http://www.elastichosts.com/blog/2009/01/01/designing-a-great-http-api/> which I've taken heed of in this proposal:
Choice of syntax: Different users will find different syntax most natural. At the unix shell, space-deliminated text rules. From Javascript, you’ll want JSON. From Java, you may want XML. Some tools parse x-www-form-encoded data nicely. A great HTTP APIMIME content types. (OK, we admit that we’ve only released text/plain ourselves so far, but the rest are coming very soon!).

All of which use a specific model that is rendered in various ways. Randy, Richard, Tim, Chris, any practical  implications on using atom?
I'm very interested to hear the feedback from these guys too (hence this post) and while moving to something like JSON<http://www.json.org/>/YAML<http://www.yaml.org>/YML<http://fdik.org/yml/> may make sense technically it would be at the cost of many years of maturity and a substantial base of (almost) OCCI-ready clients<http://code.google.com/apis/gdata/clientlibs.html> courtesy Google (not to mention the various feed readers/libraries etc). So long as we're going to support multiple formats anyway I figure we may as well base it on XML given the ease at which it can be downconverted to text/json/etc. and upconverted to [X]HTML, PDF, etc. - making sample XSLT's available to implementors will make this task a walk in the park (as evidenced by the reference implementation).

Anyway have a meeting on the other side of town in 8 minutes...

Sam

From: occi-wg-bounces at ogf.org<mailto:occi-wg-bounces at ogf.org> [mailto:occi-wg-bounces at ogf.org<mailto:occi-wg-bounces at ogf.org>] On Behalf Of Sam Johnston
Sent: 05 May 2009 02:34
To: occi-wg at ogf.org<mailto:occi-wg at ogf.org>
Subject: [occi-wg] Is OCCI the HTTP of Cloud Computing?

Morning all,

I'm going to break my own rules about reposting blog posts because this is very highly relevant, it's 03:30am already and I'm traveling again tomorrow. The next step for us is to work out what the protocol itself will look like on the wire, which is something I have been spending a good deal of time looking at over many months (both analysing existing efforts and thinking of "blue sky" possibilities).

I am now 100% convinced that the best results are to be had with a variant of XML over HTTP (as is the case with Amazon, Google, Sun and VMware) and that while Google's GData is by far the most successful cloud API in terms of implementations, users, disparate services, etc. Amazon's APIs are (at least for the foreseeable future) a legal minefield. I'm also very interested in the direction Sun and VMware are going and have of course been paying very close attention to existing public clouds like ElasticHosts and GoGrid (with a view to being essentially backwards compatible and sysadmin friendly).

I think the best strategy by a country mile is to standardise OCCI core protocol following Google's example (e.g. base it on Atom and/or AtomPub with additional specs for search, caching, etc.), build IaaS extensions in the spirit of Sun/VMware APIs and support alternative formats including HTML, JSON and TXT via XML Stylesheets (e.g. occi-to-html.xsl<http://code.google.com/p/occi/source/browse/trunk/xml/occi-to-html.xsl>, occi-to-json.xsl<http://code.google.com/p/occi/source/browse/trunk/xml/occi-to-json.xsl> and occi-to-text.xsl<http://code.google.com/p/occi/source/browse/trunk/xml/occi-to-text.xsl>). You can see the basics in action thanks to my Google App Engine reference implementation<http://code.google.com/p/occi/source/browse/#svn/trunk/occitest> at http://occitest.appspot.com/ (as well as HTML<http://www.w3.org/2005/08/online_xslt/xslt?xslfile=http%3A%2F%2Focci.googlecode.com%2Fsvn%2Ftrunk%2Fxml%2Focci-to-html.xsl&xmlfile=http%3A%2F%2Foccitest.appspot.com>, JSON<http://www.w3.org/2005/08/online_xslt/xslt?xslfile=http%3A%2F%2Focci.googlecode.com%2Fsvn%2Ftrunk%2Fxml%2Focci-to-json.xsl&xmlfile=http%3A%2F%2Foccitest.appspot.com> and TXT<http://www.w3.org/2005/08/online_xslt/xslt?xslfile=http%3A%2F%2Focci.googlecode.com%2Fsvn%2Ftrunk%2Fxml%2Focci-to-text.xsl&xmlfile=http%3A%2F%2Foccitest.appspot.com> versions of same), KISS junkies bearing in mind that this weighs in under 200 lines of python code! Of particular interest is the ease at which arbitrarily complex [X]HTML interfaces can be built directly on top of OCCI (optionally rendered from raw XML in the browser itself) and the use of the hCard microformat<http://microformats.org/> as a simple demonstration of what is possible.

Anyway, without further ado:

Is OCCI the HTTP of Cloud Computing?
http://samj.net/2009/05/is-occi-http-of-cloud-computing.html

The Web is built on the Hypertext Transfer Protocol (HTTP)<http://en.wikipedia.org/wiki/HTTP>, a client-server protocol that simply allows client user agents to retrieve and manipulate resources stored on a server. It follows that a single protocol could prove similarly critical for Cloud Computing<http://en.wikipedia.org/wiki/Cloud_computing>, but what would that protocol look like?

The first place to look for the answer is limitations in HTTP itself. For a start the protocol doesn't care about the payload it carries (beyond its Internet media type<http://en.wikipedia.org/wiki/Internet_media_type>, such as text/html), which doesn't bode well for realising the vision<http://www.w3.org/2001/sw/Activity.html> of the [Semantic<http://en.wikipedia.org/wiki/Semantic_Web>] Web<http://en.wikipedia.org/wiki/World_Wide_Web> as a "universal medium for the exchange of data". Surely it should be possible to add some structure to that data in the simplest way possible, without having to resort to carrying complex, opaque file formats (as is the case today)?

Ideally any such scaffolding added would be as light as possible, providing key attributes common to all objects (such as updated time) as well as basic metadata such as contributors, categories, tags and links to alternative versions. The entire web is built on hyperlinks so it follows that the ability to link between resources would be key, and these links should be flexible such that we can describe relationships in some amount of detail. The protocol would also be capable of carrying opaque payloads (as HTTP does today) and for bonus points transparent ones that the server can seamlessly understand too.

Like HTTP this protocol would not impose restrictions on the type of data it could carry but it would be seamlessly (and safely) extensible so as to support everything from contacts to contracts, biographies to books (or entire libraries!). Messages should be able to be serialised for storage and/or queuing as well as signed and/or encrypted to ensure security. Furthermore, despite significant performance improvements introduced in HTTP 1.1 it would need to be able to stream many (possibly millions) of objects as efficiently as possible in a single request too. Already we're asking a lot from something that must be extremely simple and easy to understand.

XML

It doesn't take a rocket scientist to work out that this "new" protocol is going to be XML based, building on top of HTTP in order to take advantage of the extensive existing infrastructure. Those of us who know even a little about XML will be ready to point out that the "X" in XML means "eXtensible" so we need to be specific as to the schema for this assertion to mean anything. This is where things get interesting. We could of course go down the WS-* route and try to write our own but surely someone else has crossed this bridge before - after all, organising and manipulating objects is one of the primary tasks for computers.

Who better to turn to for inspiration than a company whose mission<http://www.google.com/corporate/> it is to "organize the world's information and make it universally accessible and useful", Google. They use a single protocol for almost all of their APIs, GData<http://code.google.com/apis/gdata/>, and while people don't bother to look under the hood (no doubt thanks to the myriad client libraries<http://code.google.com/apis/gdata/clientlibs.html> made available under the permissive Apache 2.0 license), when you do you may be surprised at what you find: everything from contacts to calendar items, and pictures to videos is a feed (with some extensions for things like searching<http://code.google.com/apis/gdata/docs/2.0/basics.html#Searching> and caching<http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning>).

OCCI

Enter the OGF's Open Cloud Computing Interface (OCCI)<http://www.occi-wg.org/> whose (initial) goal it is to provide an extensible interface to Cloud Infrastructure Services (IaaS). To do so it needs to allow clients to enumerate and manipulate an arbitrary number of server side "resources" (from one to many millions) all via a single entry point. These compute, network and storage resources need to be able to be created, retrieved, updated and deleted (CRUD) and links need to be able to be formed between them (e.g. virtual machines linking to storage devices and network interfaces). It is also necessary to manage state (start, stop, restart) and retrieve performance and billing information, among other things.

The OCCI working group basically has two options now in order to deliver an implementable draft this month as promised: follow Amazon or follow Google (the whole while keeping an eye on other players including Sun and VMware). Amazon use a simple but sprawling XML based API with a PHP style flat namespace and while there is growing momentum around it, it's not without its problems. Not only do I have my doubts about its scalability outside of a public cloud environment (calls like 'DescribeImages' would certainly choke with anything more than a modest number of objects and we're talking about potentially millions) but there are a raft of intellectual property issues as well:

 *   Copyrights (specifically section 3.3 of the Amazon Software License<http://aws.amazon.com/asl/>) prevent the use of Amazon's "open source" clients with anything other than Amazon's own services.
 *   Patents pending like #20070156842<http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220070156842%22.PGNR.&OS=DN/20070156842&RS=DN/20070156842> cover the Amazon Web Services APIs and we know that Amazon have been known to use patents offensively<http://en.wikipedia.org/wiki/1-Click#Barnes_.26_Noble> against competitors.
 *   Trademarks like #3346899<http://tarr.uspto.gov/servlet/tarr?regser=serial&entry=77054011> prevent us from even referring to the Amazon APIs by name.

While I wish the guys at Eucalyptus<http://open.eucalyptus.com/> and Canonical<http://news.zdnet.com/2100-9595_22-292296.html> well and don't have a bad word to say about Amazon Web Services, this is something I would be bearing in mind while actively seeking alternatives, especially as Amazon haven't worked out<http://www.tbray.org/ongoing/When/200x/2009/01/20/Cloud-Interop> whether the interfaces are IP they should protect. Even if these issues were resolved via royalty free licensing it would be very hard as a single vendor to compete with truly open standards (RFC 4287: Atom Syndication Format<http://tools.ietf.org/html/rfc4287> and RFC 5023: Atom Publishing Protocol<http://tools.ietf.org/html/rfc5023>) which were developed at IETF by the community based on loose consensus and running code.

So what does all this have to do with an API for Cloud Infrastructure Services (IaaS)? In order to facilitate future extension my initial designs for OCCI have been as modular as possible. In fact the core protocol is completely generic, describing how to connect to a single entry point, authenticate, search, create, retrieve, update and delete resources, etc. all using existing standards including HTTP, TLS, OAuth and Atom. On top of this are extensions for compute, network and storage resources as well as state control (start, stop, restart), billing, performance, etc. in much the same way as Google have extensions for different data types (e.g. contacts vs YouTube movies).

Simply by standardising at this level OCCI may well become the HTTP of Cloud Computing.

-------------------------------------------------------------

Intel Ireland Limited (Branch)

Collinstown Industrial Park, Leixlip, County Kildare, Ireland

Registered Number: E902934

This e-mail and any attachments may contain confidential material for

the sole use of the intended recipient(s). Any review or distribution

by others is strictly prohibited. If you are not the intended

recipient, please contact the sender and delete all copies.

-------------------------------------------------------------
Intel Ireland Limited (Branch)
Collinstown Industrial Park, Leixlip, County Kildare, Ireland
Registered Number: E902934

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ogf.org/pipermail/occi-wg/attachments/20090505/9e2225c5/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mmodel.png
Type: image/png
Size: 38769 bytes
Desc: mmodel.png
Url : http://www.ogf.org/pipermail/occi-wg/attachments/20090505/9e2225c5/attachment.png 