[occi-wg] Test suite for OCCI

[Gary Mazz]

That's good to hear. Maybe there could be an mutual beneficial exchange 
of experiences. I know that had some issues with the affects of network 
latencies and found a solution for them.

I haven't used PyUnit. I'm suspect this is a "can 'o worms". Maybe we 
can find a group experienced in this area to contribute.

gary

Sam Johnston wrote:
> Gary,
>
> It's a small world - Anna's from my old university department (where I 
> studied computer science and worked as a casual academic and NT/Unix 
> sysadmin for a good few years) and my OS lecturer (among others) is 
> doing some interesting things 
> <http://www.ok-labs.com/blog/entry/mobile-virtualization-and-the-cloud-a-pleasing-symmetry/> 
> in cloud too. It's good to see them putting UNSW on the [cloud 
> computing] map.
>
> Anyway there was a followup article 
> <http://www.itnews.com.au/News/153819,more-data-released-on-cloud-stress-tests.aspx> 
> (after the paper was presented at the conference) in which she nails 
> one of the main problems with performance metrics in the cloud:
>
> "It is not like the (database benchmarking organisation) TPCC 
> <http://www.tpc.org/tpcc/>, which measures the performance of 
> relational databases," she said. "There is a well-defined set of 
> capabilities and usage scenarios for a relational database, not so for 
> a cloud service."
>
> I have a feeling PyUnit captures the time it takes to run each test, 
> but if not I'd say it will be reasonably easy to gather this 
> information if it's deemed useful. I'd wager the more interesting 
> performance information relates to the workloads themselves rather 
> than the performance of [what should be a relatively efficient] API.
>
> Sam
>
> On Mon, Oct 26, 2009 at 4:36 AM, Gary Mazz <garymazzaferro at gmail.com 
> <mailto:garymazzaferro at gmail.com>> wrote:
>
>     Good idea to get his process started now.
>
>     There has been work done in this area by Anna Liu, Associate
>     Professor at University of New South Wales School of Computer Science.
>
>     Here is a link to th itnews article covering the work:
>     http://www.itnews.com.au/News/153451,stress-tests-rain-on-amazons-cloud.aspx#
>
>     gary
>
>
>
>     Sam Johnston wrote:
>
>         Morning all,
>
>         Last for tonight I promise... I (well, and a bunch of people
>         much smarter than myself) figure the best way to make sure
>         that implementations are interoperable is to develop a
>         comprehensive test suite from the specification. Accordingly
>         I've been looking at the various options and having written
>         the earlier reference implementation in Python (for Google App
>         Engine) I figure PyUnit is as good a candidate as ever. It
>         helps that virtually all systems have python either
>         preinstalled or at least available for download nowadays and
>         that there's a Google App Engine harness for it (gaeunit.py
>         <http://code.google.com/p/gaeunit/>). This should allow anyone
>         (including end users) to point it at their implementations and
>         test compliance without having to download anything. I say
>         "should" because I'm not sure if GAE supports custom HTTP
>         methods yet (e.g. PUT, OPTIONS and more obscure ones like COPY
>         and MOVE), but I'll soon find out.
>
>
>         Before I spend too much more time looking into this I wanted
>         to give you all an opportunity to comment, particularly if you
>         have better ideas? Do any of you think this (and test suites
>         in general) are a particularly good or bad idea?
>
>         Sam
>
>         ------------------------------------------------------------------------
>
>
>
>         _______________________________________________
>         occi-wg mailing list
>         occi-wg at ogf.org <mailto:occi-wg at ogf.org>
>         http://www.ogf.org/mailman/listinfo/occi-wg
>          
>
>
>