[occi-wg] Is OCCI the HTTP of Cloud Computing?

[Sam Johnston]

Andy,

Thanks for the feedback. I'm on the road so I'll be concise for a change.

On Tue, May 5, 2009 at 12:16 PM, Edmonds, AndrewX <andrewx.edmonds at intel.com
> wrote:

> By introducing Atom, are you proposing it to be the meta-model of OCCI? If
> it is the meta-model then we should define relationships at a model level as
> atom will not do this for us. These relationships are not detailed in the
> Noun/Verb/Attribute page.
>

That's the idea - it gives us just enough structure (but no more) and while
the proposal is not without
contention<http://samj.net/2009/05/is-occi-http-of-cloud-computing.html#comments>,
I am unconvinced that text and/or json alone will meet the (at times
esoteric) needs of the enterprise customers I represent.

Agreed, we will need to have a simple model showing compute resources having
zero or more network and storage resources as well as network (and storage?)
interconnects. It may be interesting/necessary to stray into the territory
of these resources having parents too but we're already starting to stray
onto SNIA turf there (better to have them extend OCCI too if we can).

> How does it differ by choosing the approach of
> SUN/Vmware/Elastichosts/GoGrid?
>
VMware appear to be going for OVF over RESTful HTTP, possibly with some kind
of XML wrapping. That's not a bad idea but it's somewhat overkill - I came
up with it myself independently for OCCI but dropped it based on feedback
from members of this group.

Sun do something
similar<http://kenai.com/projects/suncloudapis/pages/HelloCloud>with
JSON (likely becuase of the Q-Layer vintage where the API was consumed
by a web interface) but that fails the "sysadmin friendly" test and as soon
as you need to invoke a library it doesn't matter whether it's JSON or ASN.1
on the wire :)

GoGrid have a query
API<http://wiki.gogrid.com/wiki/index.php/API:Anatomy_of_a_GoGrid_API_Call>that
will talk JSON, XML and CSV and ElasticHosts ran
with <http://www.elastichosts.com/products/api> space separated
attribute/value pairs. They also give
advice<http://www.elastichosts.com/blog/2009/01/01/designing-a-great-http-api/>which
I've taken heed of in this proposal:

Choice of syntax: Different users will find different syntax most natural.
At the unix shell, space-deliminated text rules. From Javascript, you’ll
want JSON. From Java, you may want XML. Some tools parse x-www-form-encoded
data nicely. A great HTTP APIMIME content types. (OK, we admit that we’ve
only released text/plain ourselves so far, but the rest are coming
very soon!).

All of which use a specific model that is rendered in various ways. Randy,
> Richard, Tim, Chris, any practical  implications on using atom?
>
I'm very interested to hear the feedback from these guys too (hence this
post) and while moving to something like JSON
<http://www.json.org/>/YAML<http://www.yaml.org>
/YML <http://fdik.org/yml/> may make sense technically it would be at the
cost of many years of maturity and a substantial base of (almost) OCCI-ready
clients <http://code.google.com/apis/gdata/clientlibs.html> courtesy Google
(not to mention the various feed readers/libraries etc). So long as we're
going to support multiple formats anyway I figure we may as well base it on
XML given the ease at which it can be downconverted to text/json/etc. and
upconverted to [X]HTML, PDF, etc. - making sample XSLT's available to
implementors will make this task a walk in the park (as evidenced by the
reference implementation).

Anyway have a meeting on the other side of town in 8 minutes...

Sam


>  *From:* occi-wg-bounces at ogf.org [mailto:occi-wg-bounces at ogf.org] *On
> Behalf Of *Sam Johnston
> *Sent:* 05 May 2009 02:34
> *To:* occi-wg at ogf.org
> *Subject:* [occi-wg] Is OCCI the HTTP of Cloud Computing?
>
>
>
> Morning all,
>
> I'm going to break my own rules about reposting blog posts because this is
> very highly relevant, it's 03:30am already and I'm traveling again tomorrow.
> The next step for us is to work out what the protocol itself will look like
> on the wire, which is something I have been spending a good deal of time
> looking at over many months (both analysing existing efforts and thinking of
> "blue sky" possibilities).
>
> I am now 100% convinced that the best results are to be had with a variant
> of XML over HTTP (as is the case with Amazon, Google, Sun and VMware) and
> that while Google's GData is by far the most successful cloud API in terms
> of implementations, users, disparate services, etc. Amazon's APIs are (at
> least for the foreseeable future) a legal minefield. I'm also very
> interested in the direction Sun and VMware are going and have of course been
> paying very close attention to existing public clouds like ElasticHosts and
> GoGrid (with a view to being essentially backwards compatible and sysadmin
> friendly).
>
> I think the best strategy by a country mile is to standardise OCCI core
> protocol following Google's example (e.g. base it on Atom and/or AtomPub
> with additional specs for search, caching, etc.), build IaaS extensions in
> the spirit of Sun/VMware APIs and support alternative formats including
> HTML, JSON and TXT via XML Stylesheets (e.g. occi-to-html.xsl<http://code.google.com/p/occi/source/browse/trunk/xml/occi-to-html.xsl>,
> occi-to-json.xsl<http://code.google.com/p/occi/source/browse/trunk/xml/occi-to-json.xsl>and
> occi-to-text.xsl<http://code.google.com/p/occi/source/browse/trunk/xml/occi-to-text.xsl>).
> You can see the basics in action thanks to my Google App Engine reference
> implementation<http://code.google.com/p/occi/source/browse/#svn/trunk/occitest>at
> http://occitest.appspot.com/ (as well as HTML<http://www.w3.org/2005/08/online_xslt/xslt?xslfile=http%3A%2F%2Focci.googlecode.com%2Fsvn%2Ftrunk%2Fxml%2Focci-to-html.xsl&xmlfile=http%3A%2F%2Foccitest.appspot.com>,
> JSON<http://www.w3.org/2005/08/online_xslt/xslt?xslfile=http%3A%2F%2Focci.googlecode.com%2Fsvn%2Ftrunk%2Fxml%2Focci-to-json.xsl&xmlfile=http%3A%2F%2Foccitest.appspot.com>and
> TXT<http://www.w3.org/2005/08/online_xslt/xslt?xslfile=http%3A%2F%2Focci.googlecode.com%2Fsvn%2Ftrunk%2Fxml%2Focci-to-text.xsl&xmlfile=http%3A%2F%2Foccitest.appspot.com>versions of same), KISS junkies bearing in mind that this weighs in under
> 200 lines of python code! Of particular interest is the ease at which
> arbitrarily complex [X]HTML interfaces can be built directly on top of OCCI
> (optionally rendered from raw XML in the browser itself) and the use of the
> hCard microformat <http://microformats.org/> as a simple demonstration of
> what is possible.
>
> Anyway, without further ado:
>
> Is OCCI the HTTP of Cloud Computing?
> http://samj.net/2009/05/is-occi-http-of-cloud-computing.html
>
> The Web is built on the Hypertext Transfer Protocol (HTTP)<http://en.wikipedia.org/wiki/HTTP>,
> a client-server protocol that simply allows client user agents to retrieve
> and manipulate resources stored on a server. It follows that a single
> protocol could prove similarly critical for Cloud Computing<http://en.wikipedia.org/wiki/Cloud_computing>,
> but what would that protocol look like?
>
> The first place to look for the answer is limitations in HTTP itself. For a
> start the protocol doesn't care about the payload it carries (beyond its Internet
> media type <http://en.wikipedia.org/wiki/Internet_media_type>, such as
> text/html), which doesn't bode well for realising the vision<http://www.w3.org/2001/sw/Activity.html>of the [
> Semantic <http://en.wikipedia.org/wiki/Semantic_Web>] Web<http://en.wikipedia.org/wiki/World_Wide_Web>as a "universal medium for the exchange of data". Surely it should be
> possible to add some structure to that data in the simplest way possible,
> without having to resort to carrying complex, opaque file formats (as is the
> case today)?
>
> Ideally any such scaffolding added would be as light as possible, providing
> key attributes common to all objects (such as updated time) as well as basic
> metadata such as contributors, categories, tags and links to alternative
> versions. The entire web is built on hyperlinks so it follows that the
> ability to link between resources would be key, and these links should be
> flexible such that we can describe relationships in some amount of detail.
> The protocol would also be capable of carrying opaque payloads (as HTTP does
> today) and for bonus points transparent ones that the server can seamlessly
> understand too.
>
> Like HTTP this protocol would not impose restrictions on the type of data
> it could carry but it would be seamlessly (and safely) extensible so as to
> support everything from contacts to contracts, biographies to books (or
> entire libraries!). Messages should be able to be serialised for storage
> and/or queuing as well as signed and/or encrypted to ensure security.
> Furthermore, despite significant performance improvements introduced in HTTP
> 1.1 it would need to be able to stream many (possibly millions) of objects
> as efficiently as possible in a single request too. Already we're asking a
> lot from something that must be extremely simple and easy to understand.
>
> XML
>
> It doesn't take a rocket scientist to work out that this "new" protocol is
> going to be XML based, building on top of HTTP in order to take advantage of
> the extensive existing infrastructure. Those of us who know even a little
> about XML will be ready to point out that the "X" in XML means "eXtensible"
> so we need to be specific as to the schema for this assertion to mean
> anything. This is where things get interesting. We could of course go down
> the WS-* route and try to write our own but surely someone else has crossed
> this bridge before - after all, organising and manipulating objects is one
> of the primary tasks for computers.
>
> Who better to turn to for inspiration than a company whose mission<http://www.google.com/corporate/>it is to "organize the world's information and make it universally
> accessible and useful", Google. They use a single protocol for almost all of
> their APIs, GData <http://code.google.com/apis/gdata/>, and while people
> don't bother to look under the hood (no doubt thanks to the myriad client
> libraries <http://code.google.com/apis/gdata/clientlibs.html> made
> available under the permissive Apache 2.0 license), when you do you may be
> surprised at what you find: everything from contacts to calendar items, and
> pictures to videos is a feed (with some extensions for things like
> searching<http://code.google.com/apis/gdata/docs/2.0/basics.html#Searching>and
> caching<http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning>
> ).
>
> OCCI
>
> Enter the OGF's Open Cloud Computing Interface (OCCI)<http://www.occi-wg.org/>whose (initial) goal it is to provide an extensible interface to Cloud
> Infrastructure Services (IaaS). To do so it needs to allow clients to
> enumerate and manipulate an arbitrary number of server side "resources"
> (from one to many millions) all via a single entry point. These compute,
> network and storage resources need to be able to be created, retrieved,
> updated and deleted (CRUD) and links need to be able to be formed between
> them (e.g. virtual machines linking to storage devices and network
> interfaces). It is also necessary to manage state (start, stop, restart) and
> retrieve performance and billing information, among other things.
>
> The OCCI working group basically has two options now in order to deliver an
> implementable draft this month as promised: follow Amazon or follow Google
> (the whole while keeping an eye on other players including Sun and VMware).
> Amazon use a simple but sprawling XML based API with a PHP style flat
> namespace and while there is growing momentum around it, it's not without
> its problems. Not only do I have my doubts about its scalability outside of
> a public cloud environment (calls like 'DescribeImages' would certainly
> choke with anything more than a modest number of objects and we're talking
> about potentially millions) but there are a raft of intellectual property
> issues as well:
>
>    - *Copyrights* (specifically section 3.3 of the Amazon Software License<http://aws.amazon.com/asl/>)
>    prevent the use of Amazon's "open source" clients with anything other than
>    Amazon's own services.
>    - *Patents* pending like #20070156842<http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220070156842%22.PGNR.&OS=DN/20070156842&RS=DN/20070156842>cover the Amazon Web Services APIs and we know that Amazon have been known
>    to use patents offensively<http://en.wikipedia.org/wiki/1-Click#Barnes_.26_Noble>against competitors.
>    - *Trademarks* like #3346899<http://tarr.uspto.gov/servlet/tarr?regser=serial&entry=77054011>prevent us from even referring to the Amazon APIs by name.
>
> While I wish the guys at Eucalyptus <http://open.eucalyptus.com/> and
> Canonical <http://news.zdnet.com/2100-9595_22-292296.html> well and don't
> have a bad word to say about Amazon Web Services, this is something I would
> be bearing in mind while actively seeking alternatives, especially as Amazon
> haven't worked out<http://www.tbray.org/ongoing/When/200x/2009/01/20/Cloud-Interop>whether the interfaces are IP they should protect. Even if these issues were
> resolved via royalty free licensing it would be very hard as a single vendor
> to compete with truly open standards (RFC 4287: Atom Syndication Format<http://tools.ietf.org/html/rfc4287>and RFC
> 5023: Atom Publishing Protocol <http://tools.ietf.org/html/rfc5023>) which
> were developed at IETF by the community based on loose consensus and running
> code.
>
> So what does all this have to do with an API for Cloud Infrastructure
> Services (IaaS)? In order to facilitate future extension my initial designs
> for OCCI have been as modular as possible. In fact the core protocol is
> completely generic, describing how to connect to a single entry point,
> authenticate, search, create, retrieve, update and delete resources, etc.
> all using existing standards including HTTP, TLS, OAuth and Atom. On top of
> this are extensions for compute, network and storage resources as well as
> state control (start, stop, restart), billing, performance, etc. in much the
> same way as Google have extensions for different data types (e.g. contacts
> vs YouTube movies).
>
> Simply by standardising at this level OCCI may well become the HTTP of
> Cloud Computing.
>
> -------------------------------------------------------------
> Intel Ireland Limited (Branch)
> Collinstown Industrial Park, Leixlip, County Kildare, Ireland
> Registered Number: E902934
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ogf.org/pipermail/occi-wg/attachments/20090505/303e96e8/attachment.html