[occi-wg] Syntax of OCCI API

Thu Apr 16 04:32:58 CDT 2009

Moving from semantics to syntax, I've been reviewing the draft API design
which Sam has posted on the wiki at

  http://forge.ogf.org/sf/wiki/do/viewPage/projects.occi-wg/wiki/APIDesign

Sam - congratulations on getting the ball running and something up for us to
comment on! What follows may sound like criticism, but I agree with the
majority of your design goals.

The beauty of the best existing APIs (e.g. ElasticHosts, GoGrid) is that
they are very simple to use and to implement. They have a small number of
nouns and verbs (see my previous mail) and also very straightforward syntax:
typical commands take only ~100 bytes and could be typed by hand.

I'm concerned that the draft API is heading towards syntax with ~1000s of
bytes of a relatively complex XML schema - this is a consequence of "Design
Pattern A" following the Google GData approach.

  http://wiki.cloudcommunity.org/wiki/Open_Cloud_Computing_Interface
  [I can't find this on the OGF wiki - where is it?]
  http://www.ogf.org/pipermail/capi-bof/2009-March/000138.html

I'd like to revisit Sam's original email to the CAPI list on design
goals---I did not see any response from others at the time---and propose
"Design Pattern B" following the ElasticHosts and GoGrid approach.

I'm happy to expand on this on the wiki. I propose splitting the existing
API Design page into several parts:

  - Design goals (as Sam's original mail and my comments below)
  - Nouns and verbs (i.e. semantics of objects; independent of syntax)
  - Design Pattern A (Sam's syntax in style of Google GData)
  - Design Pattern B (syntax in style of ElasticHosts/GoGrid)

Do people agree?

Revisiting design goals
=======================

> Design goals
> 
> * Clarity making it easy to implement and consume (15-20 calls max)
> * Based on Hypertext Transfer Protocol (HTTP)
> * Using Representational State Transfer (REST) (with HATEOAS?)
> * Requires Transport Layer Security (TLS) for integrity and
>   confidentiality
> * Relies on HTTP for security (rather than signing requests which caused
>   problems for Amazon):
>   o HTTP Basic access authentication
>   o TLS Client certificate authentication
>   o OAuth

We agree with all of these. I think the authentication should be via the
Authorization: header with a set of supported methods announced by
WWW-Authenticate: in the standard HTTP way. I think it's outside the scope
of this standard to specify which methods must be supported, although we
would probably implement plain and cram-md5 as a minimum.

> * Single URL entry point for trivial client configuration and support of
>   multiple different APIs (e.g. http://api.example.com/occi)

I agree that all API calls should be under a single stem. I would separate
the ~15 verbs to separate leaf URIs under that stem for simplicity of each
individual call, as you already do with machine operations, so that the CRUD
operations also each have their own leaf URIs as opposed to operating on the
object URI directly. (This is the style of ElasticHosts verbs create, info, set,
destroy on drives, guests and resources, e.g.

  POST /guests/create
  POST /drives/create
  POST /guests/GUEST/destroy

etc.)

> * Single structured data format (e.g. XML vs JSON vs ???)

I disagree here. In the spirit of making the API simple for users with
different platforms and languages, I recommend multiple skins distinguished
by HTTP Content-Type. (ElasticHosts today support text and JSON with plans
for XML; GoGrid support JSON, XML and CSV; Sun support JSON.)

> * XML responses (with XML namespaces for extensibility)
>   o Atom and/or AtomPub format (for extensive client library support?) ala
>     Google GData APIs

The key mistake is using complicated data structures: either complex XML
schemas (c.f. existing draft, which already nests 7 levels of indentation!)
or simply lists of properties of lists, etc. (c.f. Sun's cloud API).

In both the ElasticHosts and GoGrid APIs, we've been very disciplined in
making each of our verbs take and return a very simple data structure:
either a simple list of uuids or key/value pairs of properties for a single
object.

This makes parsing trivial in any language, and makes it easy for us to
support receiving and generating the same data with different content types
(XML, JSON, text, etc. as above).

> * Sensible, transparent versioning - only one API URL for all clients
>   (perhaps using HTTP content negotiation)

Content negotiation is for agreeing the content-type (i.e. syntax) not
anything higher level. The content type of (say) JSON is application/json or
text/json, not something OCCI-specific, so this looks like an abuse of
mechanism to me. I'm open to suggestions about how this can be done cleanly
in a way that isn't cumbersome for end-users, though.

> * Random UUIDs (version 4) for internal object identification (opaque is
>   good for security) e.g. {f47ac10b-58cc-4372-a567-0e02b2c3d479}

I support UUIDs. They can be allocated in a distributed manner without
locking or worries about collisions, and we use them throughout our API.

However, some providers may want to issue numeric ids, etc., so instead of
constraining identifiers to UUID syntax, perhaps any standard need only
specify that object IDs are a unique string from a given alphabet
([a-z0-9-]+ say: best to avoid whitespace and syntactically-significant
characters), which are issued by the provider when the object is created.

> * WebDAV interface for native OS GUI manipulation? (e.g. drag VM from
>   library folder to cloud to start, delete to stop, etc.)
>   o WebDAV adds collections et al in a "standard" fashion
>   o Interesting implications for getting files into and out of the cloud

I think this is something outside the scope of an HTTP API standard, even if
it's a cute feature for a cloud provider!

> Features
>
> * Object persistence optional (e.g. Servers in "stopped" state rather than
>   ephemeral)
>   o Allows for "lite" implementations such as those burnt into BIOSes.

This is an issue of semantics rather than syntax, and I tackled it in my
"nouns and verbs" post. Today APIs vary (ElasticHosts, Amazon have ephemeral
servers; GoGrid have persistent servers). I lean towards an API supporting
persistent servers for greater generality than ElasticHosts currently offer.

> Objects
>
> * Workload: initially virtual machines (e.g. OVF), potentially later
>   applications (e.g. Ruby gem, Python egg, Java WAR/EAR)
>   o Can be identified by Internet media type (e.g. application/ovf+xml)
> * Container: initially hypervisor (e.g. Hyper-V/KVM/VMware/Xen),
>   potentially later depending on workload (e.g. Java app server for Java
>   WAR/EAR)

My concern here is that we do not descend into complicated data structures
(see above). For example OVF has a significantly complex XML schema in
itself. If all we need to do is specify simple parameters like megabytes of
memory or gigabytes of disk space then we do not need OVF to do this.

Design pattern B
================

In light of these thoughts, I'd propose an alternative "Design Pattern B" in
the style of the ElasticHosts <url:http://www.elastichosts.com/products/api>
and GoGrid <url:http://wiki.gogrid.com/wiki/index.php/API> APIs, with one
URL per verb and much simpler syntax.

Essentially, we would agree a small set of ~3 nouns and ~15 verbs as per my
other mail. Each verb should take or return a very simple data structure:
either a simple list of uuids or key/value pairs of properties for a single
object. This data structure could be expressed in a choice of formats (XML,
JSON, text, etc.) and would take ~100 bytes or less in any of these.

Shall I go ahead, split out the API design wiki page as I described and
expand this description of design pattern B?

Best wishes,

Chris.