[occi-wg] Is OCCI the HTTP of Cloud Computing?

[Sam Johnston]

On Tue, May 5, 2009 at 9:58 PM, Edmonds, AndrewX
<andrewx.edmonds at intel.com>wrote:

>  Ah ok now that makes things a little clearer.
>
> I saw Benjamin’s (@benjaminblack) concerns regarding the usage of atom but
> if we are using it as the metamodel then it only adds more rigor to our
> effort. On the other hand we are dependent on the atom model. If adopting
> atom we need to take our nouns apply the atom metamodel to them. I’ve a
> simple little metamodel that I was playing about which could provide a means
> for our atom-based noun model (attached). It is most basic but follows some
> aspects such as OVF and Sun API. Perhaps that could serve as a starting
> point?
>
Thanks Andy, this looks like a great start. Atom is exactly that - adding
just enough structure in the form of links and metadata and the vast
majority of it is stuff we were already trying to reinvent from scratch
before.

> If we settle on the model (how nouns are related) then I don’t see any real
> issue in the presentation of that model via various renderings (here we
> could use content negotiation) like text, json and xml (although a rendering
> of atom-based json might look ugly – perhaps JSONSchema [1] might be of use
> here? As for text – no idea, yet J).
>
Content negotiation is part of the spec as well but something I did consider
was brain dead/secure devices that run e.g. desktop images might want to
have a *very* simple implementation consisting of just static "OCCI" files
sitting on an embedded web server. To allow for that, in addition to
Internet media (mime) types we may want to specify file extensions too (e.g.
.xml, .txt, .html, .json).

This is a nice to have for sysadmins and anyone interacting with the API
manually (e.g. point your browser at
http://example.com/d1a12188-59ed-44a2-bfdd-cdcd44ff5c3d.json) as otherwise
it's virtually impossible to access the different renderings. I'm pretty
sure some of the existing APIs do this already.

Sam

> Andy
>
>
>
> [1] http://www.json.com/json-schema-proposal
>
>
>
>
>
> *From:* Sam Johnston [mailto:samj at samj.net]
> *Sent:* 05 May 2009 12:23
> *To:* Edmonds, AndrewX
> *Cc:* occi-wg at ogf.org
> *Subject:* Re: [occi-wg] Is OCCI the HTTP of Cloud Computing?
>
>
>
> Andy,
>
> Thanks for the feedback. I'm on the road so I'll be concise for a change.
>
> On Tue, May 5, 2009 at 12:16 PM, Edmonds, AndrewX <
> andrewx.edmonds at intel.com> wrote:
>
> By introducing Atom, are you proposing it to be the meta-model of OCCI? If
> it is the meta-model then we should define relationships at a model level as
> atom will not do this for us. These relationships are not detailed in the
> Noun/Verb/Attribute page.
>
>
> That's the idea - it gives us just enough structure (but no more) and while
> the proposal is not without contention<http://samj.net/2009/05/is-occi-http-of-cloud-computing.html#comments>,
> I am unconvinced that text and/or json alone will meet the (at times
> esoteric) needs of the enterprise customers I represent.
>
> Agreed, we will need to have a simple model showing compute resources
> having zero or more network and storage resources as well as network (and
> storage?) interconnects. It may be interesting/necessary to stray into the
> territory of these resources having parents too but we're already starting
> to stray onto SNIA turf there (better to have them extend OCCI too if we
> can).
>
>  How does it differ by choosing the approach of
> SUN/Vmware/Elastichosts/GoGrid?
>
>  VMware appear to be going for OVF over RESTful HTTP, possibly with some
> kind of XML wrapping. That's not a bad idea but it's somewhat overkill - I
> came up with it myself independently for OCCI but dropped it based on
> feedback from members of this group.
>
> Sun do something similar<http://kenai.com/projects/suncloudapis/pages/HelloCloud>with JSON (likely becuase of the Q-Layer vintage where the API was consumed
> by a web interface) but that fails the "sysadmin friendly" test and as soon
> as you need to invoke a library it doesn't matter whether it's JSON or ASN.1
> on the wire :)
>
> GoGrid have a query API<http://wiki.gogrid.com/wiki/index.php/API:Anatomy_of_a_GoGrid_API_Call>that will talk JSON, XML and CSV and ElasticHosts ran
> with <http://www.elastichosts.com/products/api> space separated
> attribute/value pairs. They also give advice<http://www.elastichosts.com/blog/2009/01/01/designing-a-great-http-api/>which I've taken heed of in this proposal:
>
> Choice of syntax: Different users will find different syntax most natural.
> At the unix shell, space-deliminated text rules. From Javascript, you’ll
> want JSON. From Java, you may want XML. Some tools parse
> x-www-form-encoded data nicely. A great HTTP APIMIME content types. (OK,
> we admit that we’ve only released text/plain ourselves so far, but the rest
> are coming very soon!).
>
>  All of which use a specific model that is rendered in various ways.
> Randy, Richard, Tim, Chris, any practical  implications on using atom?
>
>  I'm very interested to hear the feedback from these guys too (hence this
> post) and while moving to something like JSON <http://www.json.org/>/YAML<http://www.yaml.org>
> /YML <http://fdik.org/yml/> may make sense technically it would be at the
> cost of many years of maturity and a substantial base of (almost) OCCI-ready
> clients <http://code.google.com/apis/gdata/clientlibs.html> courtesy
> Google (not to mention the various feed readers/libraries etc). So long as
> we're going to support multiple formats anyway I figure we may as well base
> it on XML given the ease at which it can be downconverted to text/json/etc.
> and upconverted to [X]HTML, PDF, etc. - making sample XSLT's available to
> implementors will make this task a walk in the park (as evidenced by the
> reference implementation).
>
> Anyway have a meeting on the other side of town in 8 minutes...
>
> Sam
>
>
>   *From:* occi-wg-bounces at ogf.org [mailto:occi-wg-bounces at ogf.org] *On
> Behalf Of *Sam Johnston
> *Sent:* 05 May 2009 02:34
> *To:* occi-wg at ogf.org
> *Subject:* [occi-wg] Is OCCI the HTTP of Cloud Computing?
>
>
>
> Morning all,
>
> I'm going to break my own rules about reposting blog posts because this is
> very highly relevant, it's 03:30am already and I'm traveling again tomorrow.
> The next step for us is to work out what the protocol itself will look like
> on the wire, which is something I have been spending a good deal of time
> looking at over many months (both analysing existing efforts and thinking of
> "blue sky" possibilities).
>
> I am now 100% convinced that the best results are to be had with a variant
> of XML over HTTP (as is the case with Amazon, Google, Sun and VMware) and
> that while Google's GData is by far the most successful cloud API in terms
> of implementations, users, disparate services, etc. Amazon's APIs are (at
> least for the foreseeable future) a legal minefield. I'm also very
> interested in the direction Sun and VMware are going and have of course been
> paying very close attention to existing public clouds like ElasticHosts and
> GoGrid (with a view to being essentially backwards compatible and sysadmin
> friendly).
>
> I think the best strategy by a country mile is to standardise OCCI core
> protocol following Google's example (e.g. base it on Atom and/or AtomPub
> with additional specs for search, caching, etc.), build IaaS extensions in
> the spirit of Sun/VMware APIs and support alternative formats including
> HTML, JSON and TXT via XML Stylesheets (e.g. occi-to-html.xsl<http://code.google.com/p/occi/source/browse/trunk/xml/occi-to-html.xsl>,
> occi-to-json.xsl<http://code.google.com/p/occi/source/browse/trunk/xml/occi-to-json.xsl>and
> occi-to-text.xsl<http://code.google.com/p/occi/source/browse/trunk/xml/occi-to-text.xsl>).
> You can see the basics in action thanks to my Google App Engine reference
> implementation<http://code.google.com/p/occi/source/browse/#svn/trunk/occitest>at
> http://occitest.appspot.com/ (as well as HTML<http://www.w3.org/2005/08/online_xslt/xslt?xslfile=http%3A%2F%2Focci.googlecode.com%2Fsvn%2Ftrunk%2Fxml%2Focci-to-html.xsl&xmlfile=http%3A%2F%2Foccitest.appspot.com>,
> JSON<http://www.w3.org/2005/08/online_xslt/xslt?xslfile=http%3A%2F%2Focci.googlecode.com%2Fsvn%2Ftrunk%2Fxml%2Focci-to-json.xsl&xmlfile=http%3A%2F%2Foccitest.appspot.com>and
> TXT<http://www.w3.org/2005/08/online_xslt/xslt?xslfile=http%3A%2F%2Focci.googlecode.com%2Fsvn%2Ftrunk%2Fxml%2Focci-to-text.xsl&xmlfile=http%3A%2F%2Foccitest.appspot.com>versions of same), KISS junkies bearing in mind that this weighs in under
> 200 lines of python code! Of particular interest is the ease at which
> arbitrarily complex [X]HTML interfaces can be built directly on top of OCCI
> (optionally rendered from raw XML in the browser itself) and the use of the
> hCard microformat <http://microformats.org/> as a simple demonstration of
> what is possible.
>
> Anyway, without further ado:
>
> Is OCCI the HTTP of Cloud Computing?
> http://samj.net/2009/05/is-occi-http-of-cloud-computing.html
>
> The Web is built on the Hypertext Transfer Protocol (HTTP)<http://en.wikipedia.org/wiki/HTTP>,
> a client-server protocol that simply allows client user agents to retrieve
> and manipulate resources stored on a server. It follows that a single
> protocol could prove similarly critical for Cloud Computing<http://en.wikipedia.org/wiki/Cloud_computing>,
> but what would that protocol look like?
>
> The first place to look for the answer is limitations in HTTP itself. For a
> start the protocol doesn't care about the payload it carries (beyond its Internet
> media type <http://en.wikipedia.org/wiki/Internet_media_type>, such as
> text/html), which doesn't bode well for realising the vision<http://www.w3.org/2001/sw/Activity.html>of the [
> Semantic <http://en.wikipedia.org/wiki/Semantic_Web>] Web<http://en.wikipedia.org/wiki/World_Wide_Web>as a "universal medium for the exchange of data". Surely it should be
> possible to add some structure to that data in the simplest way possible,
> without having to resort to carrying complex, opaque file formats (as is the
> case today)?
>
> Ideally any such scaffolding added would be as light as possible, providing
> key attributes common to all objects (such as updated time) as well as basic
> metadata such as contributors, categories, tags and links to alternative
> versions. The entire web is built on hyperlinks so it follows that the
> ability to link between resources would be key, and these links should be
> flexible such that we can describe relationships in some amount of detail.
> The protocol would also be capable of carrying opaque payloads (as HTTP does
> today) and for bonus points transparent ones that the server can seamlessly
> understand too.
>
> Like HTTP this protocol would not impose restrictions on the type of data
> it could carry but it would be seamlessly (and safely) extensible so as to
> support everything from contacts to contracts, biographies to books (or
> entire libraries!). Messages should be able to be serialised for storage
> and/or queuing as well as signed and/or encrypted to ensure security.
> Furthermore, despite significant performance improvements introduced in HTTP
> 1.1 it would need to be able to stream many (possibly millions) of objects
> as efficiently as possible in a single request too. Already we're asking a
> lot from something that must be extremely simple and easy to understand.
>
> XML
>
> It doesn't take a rocket scientist to work out that this "new" protocol is
> going to be XML based, building on top of HTTP in order to take advantage of
> the extensive existing infrastructure. Those of us who know even a little
> about XML will be ready to point out that the "X" in XML means "eXtensible"
> so we need to be specific as to the schema for this assertion to mean
> anything. This is where things get interesting. We could of course go down
> the WS-* route and try to write our own but surely someone else has crossed
> this bridge before - after all, organising and manipulating objects is one
> of the primary tasks for computers.
>
> Who better to turn to for inspiration than a company whose mission<http://www.google.com/corporate/>it is to "organize the world's information and make it universally
> accessible and useful", Google. They use a single protocol for almost all of
> their APIs, GData <http://code.google.com/apis/gdata/>, and while people
> don't bother to look under the hood (no doubt thanks to the myriad client
> libraries <http://code.google.com/apis/gdata/clientlibs.html> made
> available under the permissive Apache 2.0 license), when you do you may be
> surprised at what you find: everything from contacts to calendar items, and
> pictures to videos is a feed (with some extensions for things like
> searching<http://code.google.com/apis/gdata/docs/2.0/basics.html#Searching>and
> caching<http://code.google.com/apis/gdata/docs/2.0/reference.html#ResourceVersioning>
> ).
>
> OCCI
>
> Enter the OGF's Open Cloud Computing Interface (OCCI)<http://www.occi-wg.org/>whose (initial) goal it is to provide an extensible interface to Cloud
> Infrastructure Services (IaaS). To do so it needs to allow clients to
> enumerate and manipulate an arbitrary number of server side "resources"
> (from one to many millions) all via a single entry point. These compute,
> network and storage resources need to be able to be created, retrieved,
> updated and deleted (CRUD) and links need to be able to be formed between
> them (e.g. virtual machines linking to storage devices and network
> interfaces). It is also necessary to manage state (start, stop, restart) and
> retrieve performance and billing information, among other things.
>
> The OCCI working group basically has two options now in order to deliver an
> implementable draft this month as promised: follow Amazon or follow Google
> (the whole while keeping an eye on other players including Sun and VMware).
> Amazon use a simple but sprawling XML based API with a PHP style flat
> namespace and while there is growing momentum around it, it's not without
> its problems. Not only do I have my doubts about its scalability outside of
> a public cloud environment (calls like 'DescribeImages' would certainly
> choke with anything more than a modest number of objects and we're talking
> about potentially millions) but there are a raft of intellectual property
> issues as well:
>
>    - *Copyrights* (specifically section 3.3 of the Amazon Software License<http://aws.amazon.com/asl/>)
>    prevent the use of Amazon's "open source" clients with anything other than
>    Amazon's own services.
>    - *Patents* pending like #20070156842<http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220070156842%22.PGNR.&OS=DN/20070156842&RS=DN/20070156842>cover the Amazon Web Services APIs and we know that Amazon have been known
>    to use patents offensively<http://en.wikipedia.org/wiki/1-Click#Barnes_.26_Noble>against competitors.
>    - *Trademarks* like #3346899<http://tarr.uspto.gov/servlet/tarr?regser=serial&entry=77054011>prevent us from even referring to the Amazon APIs by name.
>
> While I wish the guys at Eucalyptus <http://open.eucalyptus.com/> and
> Canonical <http://news.zdnet.com/2100-9595_22-292296.html> well and don't
> have a bad word to say about Amazon Web Services, this is something I would
> be bearing in mind while actively seeking alternatives, especially as Amazon
> haven't worked out<http://www.tbray.org/ongoing/When/200x/2009/01/20/Cloud-Interop>whether the interfaces are IP they should protect. Even if these issues were
> resolved via royalty free licensing it would be very hard as a single vendor
> to compete with truly open standards (RFC 4287: Atom Syndication Format<http://tools.ietf.org/html/rfc4287>and RFC
> 5023: Atom Publishing Protocol <http://tools.ietf.org/html/rfc5023>) which
> were developed at IETF by the community based on loose consensus and running
> code.
>
> So what does all this have to do with an API for Cloud Infrastructure
> Services (IaaS)? In order to facilitate future extension my initial designs
> for OCCI have been as modular as possible. In fact the core protocol is
> completely generic, describing how to connect to a single entry point,
> authenticate, search, create, retrieve, update and delete resources, etc.
> all using existing standards including HTTP, TLS, OAuth and Atom. On top of
> this are extensions for compute, network and storage resources as well as
> state control (start, stop, restart), billing, performance, etc. in much the
> same way as Google have extensions for different data types (e.g. contacts
> vs YouTube movies).
>
> Simply by standardising at this level OCCI may well become the HTTP of
> Cloud Computing.
>
> -------------------------------------------------------------
>
> Intel Ireland Limited (Branch)
>
> Collinstown Industrial Park, Leixlip, County Kildare, Ireland
>
> Registered Number: E902934
>
>
>
> This e-mail and any attachments may contain confidential material for
>
> the sole use of the intended recipient(s). Any review or distribution
>
> by others is strictly prohibited. If you are not the intended
>
> recipient, please contact the sender and delete all copies.
>
>
>
> -------------------------------------------------------------
> Intel Ireland Limited (Branch)
> Collinstown Industrial Park, Leixlip, County Kildare, Ireland
> Registered Number: E902934
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ogf.org/pipermail/occi-wg/attachments/20090506/74738389/attachment.html