[occi-wg] OCCI Model UML, XSD, XML, Code...

Tue May 26 04:55:34 CDT 2009

Sam,

That's great - please can you create a wiki page for these
requirements, and provide some colour on each one?  Then we can refer
to them in one place, discuss them, get them into priority order.
Call the wiki page something memorable ;-)

alexis

On Tue, May 26, 2009 at 10:45 AM, Sam Johnston <samj at samj.net> wrote:
> On Tue, May 26, 2009 at 11:18 AM, Alexis Richardson
> <alexis.richardson at gmail.com> wrote:
>>
>> Sam, Andy,
>>
>> Good dialogue.  It would be great if you could break out requirements
>> as they come up.
>>
>> For example: requirement to cache. Use case: ... Possible solution(s):...
>
> Performance-related concerns like caching are standard features of any good
> protocol. If you ignore the issue you'll certainly end up revisiting it when
> people start implementing or worse, deploying. HTTP pipelining for example
> didn't appear until HTTP 1.1, yet without it the web runs like molasses. I
> guess the use case here is any large [enterprise] deployment.
>
> Here's a handful of other use cases I came up with for my presentations
> (most of them should be obvious):
>
> Automated/autonomic management (e.g. RightScale)
> Web management interfaces (e.g. Q-Layer)
> Desktop management interfaces (e.g. MMC Snap-Ins)
> Scheduled tasks/’cron’ jobs (e.g. bash scripts, curl)
> Backup of cloud infrastructure (e.g. ‘Save As’ for the cloud)
> Migration of resources
>
> Physical to Cloud (P2C)
> Virtual to Cloud (V2C)
> Cloud to Cloud (C2C)
>
> System administration tasks (e.g. telnet, scripting)
> Management reporting (e.g. CSV, PDF outputs)
> Failover/Disaster Recovery/Business Continuity (e.g. rsync)
> Internetworking the “Intercloud”
> <your use case here>
>
> Sam
>
>>
>> On Tue, May 26, 2009 at 9:39 AM, Sam Johnston <samj at samj.net> wrote:
>> > On Mon, May 25, 2009 at 7:53 PM, Edmonds, AndrewX
>> > <andrewx.edmonds at intel.com> wrote:
>> >>
>> >> Thanks for your efforts - I find it easiest to look at this rendering
>> >> to
>> >> work out what it boils down to and I can't help but to notice there are
>> >> already some strong similarities with Atom (as well as some serious
>> >> limitations courtesy the "simplification"):
>> >>
>> >> [AE: ] Compare atom to many other models and I’m sure there’s a
>> >> reasonably
>> >> easy mapping. The rendering may not be perfect, I’ll be the first to
>> >> admit
>> >> it J If there are other limitations do please list them. I’d be keen to
>> >> hear.
>> >
>> > Creating our own protocol, from scratch, with our own ideas as to how
>> > things
>> > should be represented, categoried, associated etc. significantly raises
>> > the
>> > risk of failure as well as the barrier to entry - that's reason enough
>> > in
>> > itself not to do it, particularly when perfectly good alternatives
>> > already
>> > exist. My point is that your current proposal is very nearly Atom anyway
>> > so
>> > why not just use it? Your "simplification" complicates.
>> >>
>> >> How do I tell the difference between resources without a unique
>> >> identifier
>> >> (Atom "id")
>> >>
>> >> [AE: ] The current OCCI nouns/verbs/attributes do not specify this,
>> >> hence
>> >> a requirement. I’ve only taken what’s on the wiki and worked with that
>> >> along
>> >
>> > Surely the requirement for a unique identifier for each resource is so
>> > bleeding obvious that it's implicit... you can already add an "id"
>> > field.
>> >>
>> >> How do I tell when the resource was updated (Atom "updated") and/or
>> >> what
>> >> version it is (ETag)
>> >>
>> >> [AE: ] Same as above
>> >
>> > This is an implementation issue - one of many that implementors will
>> > trip up
>> > on as they try to turn our spec into reality. Actually it's worse than
>> > that
>> > in that it may not appear until users try to deploy the resulting
>> > systems
>> > and find that they do not perform adequately. Caching information also
>> > falls
>> > squarely into the "obvious" category for me and this stuff is very hard
>> > to
>> > get right.
>> >
>> > Using HTTP as the meta-model for individual resources and Atom for
>> > collections gives us immediate access to an entire Internet worth of
>> > implementations and optimisations - am I the only one here who sees
>> > significant benefit in being able to put reverse proxies in front of
>> > your
>> > API servers for example? I doubt I'm the only person here implementing
>> > systems at scale... this approach of moving away from a hard-wired RPC
>> > style
>> > API to a loosely coupled document based system paves the way for
>> > extremely
>> > scalable architectures - for example where nodes periodically PUT their
>> > state for clients to GET resulting in the command & control being little
>> > more than a repository.
>> >>
>> >> How do I find one or more resources by search/category/tag/etc.?
>> >>
>> >> [AE: ] See occi:Tag
>> >
>> > One global namespace for rudimentary tagging is not at all what I had in
>> > mind - again this is a problem that is long since solved and one that
>> > gets
>> > exponentially complicated with the number of items.
>> >>
>> >> How do I represent a single object?
>> >>
>> >> [AE: ] See occi:Compute or indeed occi:Resource
>> >
>> > So these things can exist independently of groups? I'd have known that
>> > already if it were a common format but here I have to resort to reading
>> > the
>> > XSD.
>> >>
>> >> How do I differentiate between tag vocabularies - "small" and
>> >> "virtualmachine" are talking about two completely different concepts
>> >> (Atom
>> >> "category")
>> >>
>> >> [AE: ] J tag == folksonomy. I guess you are referring to ontology?
>> >
>> > Taxonomy - a subtle but important difference.
>> >>
>> >> How do I create generic links to supporting resources e.g. screenshots,
>> >> consoles, etc. (Atom "link")
>> >>
>> >> [AE: ] See my first answer. Also occi:Link – oops my bad this is there
>> >> but
>> >> should be xs:complexType not xs:attributeGroup. I’ll update this.
>> >
>> > This is yet another reimplementation of an existing, well established
>> > concept that developers will have to learn.
>> >>
>> >> How do I create associations with arbitrary resources e.g. users (Atom
>> >> "link")
>> >>
>> >> [AE: ] See first answer. Extend atom:linkType or simply import it.
>> >> Arbitrary? I thought OCCI was focused on the specific domain of
>> >> Infrastructure and we spent time on extracting core nouns.
>> >
>> > Just because you don't care who owns or has access to a resource doesn't
>> > mean that I don't - rich ACLs are something that will need to be
>> > supportable
>> > (if not directly supported) and I'm sure there will be many other
>> > examples.
>> > Generic links are going to work better here, and we already have them
>> > for
>> > free.
>> >>
>> >> How do I carry critical information that we nonetheless consider out of
>> >> scope (e.g. OVF)
>> >>
>> >> [AE: ] can introduce xsd:anyType into the model easily
>> >
>> > Ok so I've got an anyType and I want to know what it's for (relation)
>> > and
>> > how to read it (content type) - how will that be expressed? Yet again,
>> > all
>> > roads lead to Atom.
>> >>
>> >> How do we extend the spec? (e.g. rinse & repeat)
>> >>
>> >> [AE: ] Extend the XSD.
>> >
>> > As I said. Anyway I've thus far managed to avoid having to touch XSD so
>> > I
>> > guess we'll have to schedule an XSD vs RELAX NG schema war at some point
>> > too... in any case better schema than no schema.
>> >>
>> >> How do vendors/users extend the spec? (e.g. namespaces?)
>> >>
>> >> [AE: ] As above plus some notions of governance from OCCI, yet to be
>> >> discussed.
>> >
>> > The point is the problem is already solved [by Atom] and we're going to
>> > have
>> > to cover the same ground.
>> >>
>> >> How do I import/export resource(s) (e.g. OVA)
>> >>
>> >> [AE: ] Now we’re into application specific functionality/features…
>> >> something of an orthogonal for modelling activities. Perhaps closest
>> >> here is
>> >> model transforms.
>> >
>> > I'd say that this requirement is extremely pertinent for model
>> > discussions -
>> > it obviously needs to support this functionality somehow or it will be
>> > impossible to get anything in/out of an implementation (and therefore to
>> > achieve portability between them). Transforms are something else
>> > entirely.
>> >>
>> >> etc.
>> >>
>> >> [AE: ] The more requirements the better. In fact OCCI does not define
>> >> the
>> >> base units of attribute values that are part of the model. I don’t
>> >> think
>> >> these can be found within Atom. Perhaps there’s an XSD out there that
>> >> we
>> >> could import and reuse those based units (e.g. MB, MHz etc). Also it
>> >> would
>> >> be nice to be able to add arbitrary infrastructure related parameters
>> >> to any
>> >> entity in the model.
>> >
>> > Your "nice" is my "necessary". Units are optional - at the very least we
>> > need to specify the smallest units (e.g. Mb for memory) but after that
>> > making the protocol pretty [to humans] makes it harder to interpret [for
>> > computers] - and therefore less interoperable. Jury's still out on this
>> > one
>> > for me.
>> >>
>> >> Following a (long) conversation over (numerous) beers with Alexis on
>> >> Friday I have come to the conclusion that the meta-model is quite
>> >> separate
>> >> from the representation, which is to say that collections,
>> >> organisation,
>> >> association, etc. can be done generically while providing a simple
>> >> "OCCI"
>> >> representation of the various resources is a domain-specific task.
>> >>
>> >> [AE: ] Meta-model constrains the model. The model defines how a model
>> >> instance can be created. Representation is the model instance and can
>> >> be
>> >> rendered. So in my view they are related, albeit through 2 levels of
>> >> indirection.
>> >
>> > How does the proposed meta-model constrain the model?
>> >>
>> >> For the former (meta-model) HTTP and/or Atom clearly make the most
>> >> sense
>> >> but for the latter we need to be careful not to end up reinventing CIM.
>> >> Gary's list of network attributes gave me pause over the weekend and I
>> >> think
>> >> the best way to filter this is to consider what functionality is
>> >> supplied
>> >> rather than what functionality is demanded. That is to say, today
>> >> public
>> >> cloud providers do IPv4 so we provide attributes for IPv4. If multiple
>> >> vendors do IPv6 we add IPv6. If nobody does IPX/SPX then too bad for
>> >> those
>> >> who need it. This stuff still needs to go somewhere though, which is
>> >> why
>> >> multiple representations are essential.
>> >>
>> >> [AE: ] I agree on this (being careful not extending CIM). In fact this
>> >> is
>> >> one of OVF’s strengths. It’s quite easy to include elements and
>> >> attributes
>> >> defined in the RASD schema in OVF, allowing people extending OVF reach
>> >> into
>> >> a huge library of entity definitions. Regarding IP issues and hence
>> >> specific
>> >> technology dependencies, I remember you describing a solution where we
>> >> not
>> >> concentrate on such details but rather assign compute resources (VM) to
>> >> networks by tag. Whatever came of that?
>> >
>> > Something better appeared, along with the realisation that if we don't
>> > give
>> > folks like SNIA a way to play then they'll blaze their own trail. In
>> > reality
>> > it's not much more complicated - networks just became first class
>> > citizens
>> > which allows for extensibility. Ditto for storage.
>> >>
>> >> If it wasn't already clear, I'm very anti the approach of modeling the
>> >> problem and blatting out one or more representations, irrespective of
>> >> the
>> >> delimiters used. A lot of good work has already been done on HTTP and
>> >> Atom
>> >> meta-models and giving everyone else the forks by creating yet another
>> >> domain-specific language is not something I plan to be in any way
>> >> associated
>> >> with... the result will almost certainly be an interop train wreck not
>> >> unlike WS-*
>> >>
>> >> [AE: ] The modelling approach through to XSD is just one that was
>> >> followed
>> >> by the Atom folks in one way or another to arrive at a domain-specific
>> >> model
>> >> for the publishing and manipulation of content. If you don’t model the
>> >> problem – even on paper, napkin or beer mat how do you expect to fully
>> >> understand it? /joke If you wish not to be domain-specific then might I
>> >> introduce you to my little friend void* :-D
>> >
>> > Atom is quite clearly not domain-specific (as evidenced by its rapid,
>> > widespread adoption in the cloud space for a myriad applications)... the
>> > concepts apply to any information organisation activity.
>> >>
>> >> With atom imho if you want to use it other than publish blog articles
>> >> then
>> >> you will have to create something more domain specific – I wouldn’t be
>> >> attempting to cout my void* to atom:entry and hope someone understand
>> >> the
>> >> content within atom:entry.
>> >
>> > We do need to have a simple "OCCI" format to represent functionality
>> > that
>> > has multiple implementations, agreed. That said, if we palm the metadata
>> > (id, updated, etc.) and metamodel (links) off to HTTP/Atom as suggested
>> > by
>> > Alexis last week then the resulting format ends up being a lot more like
>> > what Chris and Richard from ElasticHosts were pushing for. In fact it
>> > gets
>> > so simple that key-value pairs in $PREFERRED_FORMAT makes the most
>> > sense,
>> > and having multiple formats available for interop becomes feasible. This
>> > fits with the HTML forms model too, which is also highly advantageous -
>> > creating and updating resources is as simple as a HTML form submission
>> > with
>> > no other formatting required.
>> >>
>> >> In summary I'd like to see us use Atom where absolutely necessary (e.g.
>> >> collections) and raw HTTP with multiple representations everywhere
>> >> else. One
>> >> of those representations can be a restricted syntax based on features
>> >> with
>> >> multiple deployments (but users would also be free to send/receive OVF
>> >> or
>> >> even their own home grown guff if they want).
>> >>
>> >> [AE: ] That’s perfectly fine but what will those representations look
>> >> like? Who will define them? How will you verify them? If we’re not
>> >> using
>> >> even Atom, what will it be?
>> >
>> > The only representation we care about is the simplest representation
>> > necessary to create resources - cores, speed, memory etc. for compute,
>> > size
>> > for storage and ip, netmask and gateway for network, etc. The others
>> > will
>> > use existing standards:
>> >
>> > OVF for portable workloads
>> > VMX et al for proprietary workloads
>> > PNG/JPG/etc. for screenshots
>> > VNC/RDP/SSH/etc. for consoles
>> > PDF/ODF for documentation
>> > etc.
>> >
>> > Given this representation will then end up being key-value pairs we can
>> > easily specify (and implement) text, json and xml renderings for
>> > interop,
>> > while others who care about performance can use things like protocol
>> > buffers
>> > for talking to the nodes (XML parsing overhead can be non-negligible for
>> > automated workloads at scale even if not a problem for most tasks).
>> >
>> > Sam
>> >>
>> >> On Mon, May 25, 2009 at 5:57 PM, Edmonds, AndrewX
>> >> <andrewx.edmonds at intel.com> wrote:
>> >>
>> >> I've been spent some time on the proposed meta-model [1] and model [1]
>> >> UML
>> >> that is up on the wiki along with the inputs of others. Following some
>> >> discussions with Roger and Andre, I decided to take the model in its
>> >> UML
>> >> form and create the corresponding XSD. In fact Roger might have the a
>> >> RDF
>> >> version of the XSD sometime. This XSD can be found here [2]. Now I
>> >> don't
>> >> class myself as a XSD expert so any comments, updates etc are
>> >> appreciated.
>> >> From this XSD I was able to generate an API to create and manipulate an
>> >> in-memory representation of a model instance. This example code can be
>> >> found
>> >> here [3]. It's Java and uses XMLBeans[4], but there are many other
>> >> libraries
>> >> out there that will allow the generation of an API in languages such as
>> >> Python[5], Ruby[6], CSharp[7] (typically any language that has SOAP
>> >> libraries will have an XSD class generator). After rendering the
>> >> in-memory
>> >> model the XML rendering is specific but adheres to the OCCI model
>> >> schema and
>> >> can be viewed here[8]. Again how things appear in the XML can be
>> >> tweaked via
>> >> iteration of the schema. For those JSON peeps out there there's an
>> >> already
>> >> XSLT transform sheets out there that will convert this XML to JSON
>> >> using
>> >> different conventions [9][10].
>> >>
>> >> This model is abstract in a sense it does not import any dependencies,
>> >> such as Atom. @Sam perhaps you could take what I've done and either by
>> >> way
>> >> of model inheritance or composition show how Atom would integrate with
>> >> this
>> >> model?
>> >>
>> >> My view on Atom is that it is useful as a wrapper to renderings of OCCI
>> >> model instances. If I wanted to publish an OCCI model via Atom then I'd
>> >> insert a model instance rendering into an atom:entry element (note we
>> >> could
>> >> include JSON via CDATA but this is where JSON perhaps gets a little
>> >> ugly).
>> >> This would be OCCI using Atom by way of composition and would allow
>> >> both
>> >> Atom and OCCI live harmoniously together or separately and not require
>> >> Atom
>> >> dependencies should that be the choice of a provider. What's important
>> >> is
>> >> that the OCCI model be it via Atom or something else could be
>> >> understood by
>> >> a receiving system. Using Atom with OCCI by way of inheritance makes
>> >> the
>> >> previous "with or without" approach a little more difficult as there's
>> >> a
>> >> hard-dependency on Atom.
>> >>
>> >> Atom reminds me somewhat of WS-Agreement, SOAP etc. It doesn't solve
>> >> your
>> >> domain-specific model problems (i.e. what you are trying to represent -
>> >> OCCI==Infrastructure), only provide a means to send this content about
>> >> and
>> >> mechanisms supporting this but perhaps not directly to the model
>> >> contained
>> >> as content. We still have to understand and represent our
>> >> noun/verb/attribute triplets. Atom doesn't really help out bar the
>> >> features
>> >> it offers by way of sitting on top of HTTP. Now of course you can
>> >> extend
>> >> Atom base types (in fact AFAIK they're just xs:complexTypes W3C XSD
>> >> types)
>> >> but the point is you have to extend them to _something_. That something
>> >> is
>> >> your domain specific model. There is value however in Atom, not from
>> >> the
>> >> model point of view - it's a very simple model. The value in Atom is in
>> >> the
>> >> directions the specifications gives pertaining to Atom service
>> >> behaviour.
>> >> Another way to view Atom is the interoperability glue in between the
>> >> now
>> >> almost commonly accepted cloud stack of Infrastructure -> Platform ->
>> >> Service. This is where OCCI could learn quite an amount but when you
>> >> consider that the basis of these behaviours are steeply entrenched in
>> >> REST
>> >> design principles it's easy to see 1) why Atom is attractive 2) that
>> >> you
>> >> don't necessarily need Atom to have correct and/or standard service
>> >> behaviour - just adopt good, sound REST design principles!
>> >>
>> >> On more specifics and related to the attributes of each noun, we are
>> >> still
>> >> rather under-specified especially in the Storage and Network nouns.
>> >> @Richard
>> >> and @Gary (in separate threads) started a useful discussion on
>> >> attributes
>> >> for these nouns. Perhaps you two guys could add your findings and
>> >> suggestions to the wiki?
>> >>
>> >> Andy
>> >>
>> >> PS: for those of you wanting to run the code I've exported the eclipse
>> >> project and can be found here [11].
>> >>
>> >> [1]
>> >>
>> >> http://forge.ogf.org/sf/wiki/do/viewPage/projects.occi-wg/wiki/NounsVerbsAndAttributes
>> >> [2]
>> >>
>> >> http://forge.ogf.org/sf/wiki/do/viewAttachment/projects.occi-wg/wiki/NounsVerbsAndAttributes/occi.xsd
>> >> [3]
>> >>
>> >> http://forge.ogf.org/sf/wiki/do/viewAttachment/projects.occi-wg/wiki/NounsVerbsAndAttributes/OCCITest.java
>> >> [4] http://xmlbeans.apache.org
>> >> [5] http://www.rexx.com/~dkuhlman/generateDS.html
>> >> [6] http://dev.ctor.org/soap4r/browser/trunk/bin/xsd2ruby.rb
>> >> [7] http://xsd2code.codeplex.com/Wiki/View.aspx
>> >> [8]
>> >>
>> >> http://forge.ogf.org/sf/wiki/do/viewAttachment/projects.occi-wg/wiki/NounsVerbsAndAttributes/occi-instance.xml
>> >> [9] http://code.google.com/p/xml2json-xslt/
>> >> [10] http://www.bramstein.com/projects/xsltjson/
>> >> [11]
>> >>
>> >> http://forge.ogf.org/sf/wiki/do/viewAttachment/projects.occi-wg/wiki/NounsVerbsAndAttributes/OCCIModel-eclipse-project.zip
>> >>
>> >> Andy Edmonds
>> >> skype: andy.edmonds
>> >> tweets: @dizz
>> >> tel: +353 (0)1 6069232
>> >>
>> >> IT Research - IT Innovation Centre - Intel Ireland Ltd.
>> >> Intel Ireland Limited (Branch)
>> >> Collinstown Industrial Park, Leixlip, County Kildare, Ireland
>> >> Registered Number: E902934
>> >>
>> >> This e-mail and any attachments may contain confidential material for
>> >> the sole use of the intended recipient(s). Any review or distribution
>> >> by others is strictly prohibited. If you are not the intended
>> >> recipient, please contact the sender and delete all copies.
>> >>
>> >>
>> >> -------------------------------------------------------------
>> >> Intel Ireland Limited (Branch)
>> >> Collinstown Industrial Park, Leixlip, County Kildare, Ireland
>> >> Registered Number: E902934
>> >>
>> >> This e-mail and any attachments may contain confidential material for
>> >> the sole use of the intended recipient(s). Any review or distribution
>> >> by others is strictly prohibited. If you are not the intended
>> >> recipient, please contact the sender and delete all copies.
>> >>
>> >>
>> >>
>> >> -------------------------------------------------------------
>> >> Intel Ireland Limited (Branch)
>> >> Collinstown Industrial Park, Leixlip, County Kildare, Ireland
>> >> Registered Number: E902934
>> >>
>> >> This e-mail and any attachments may contain confidential material for
>> >> the sole use of the intended recipient(s). Any review or distribution
>> >> by others is strictly prohibited. If you are not the intended
>> >> recipient, please contact the sender and delete all copies.
>> >
>> >
>> > _______________________________________________
>> > occi-wg mailing list
>> > occi-wg at ogf.org
>> > http://www.ogf.org/mailman/listinfo/occi-wg
>> >
>> >
>
>