[cddlm] deployment api test plan

[Steve Loughran]

As promised in the phone call, my expectations for interop testing.

Deploy API testing

Goal:

To achieve interoperability between api clients and api
servers written by different people.


Means of achievement: Testing

Means of testing:

Cross-testing client code against server code, SOAPBuilder style.

Process:

Everyone implements a test suite that uses their own client API to make
calls of the server. This test suite must accept a URL to point to the
portal endpoint. (NB: it should be a simple URL, please don't use full
WS-Addresses here as life would be too complex)

This client test suite must implement the tests as defined by the
deployment test plan, and should work against the local implementation.

To test against the other implementations, we need accessible endpoints.
This can be done in three ways

1. public endpoint. Lets implementors update nightly, good for iterative
development, and for letting implementors trace/debug their side of the
app. Bad for security: we need to lock down our implementations
to only deploy a finite set of components from CDL. There may also be
WS-N problems here, as we wont be able to send messages back through a
firewall.

2. Redistributable installer. Implementors share releases of their
application, testers install it themselves on suitable hardware.

3. VMWare image. Implementors create a vmware image and share it around.
Good for complex installs, good for security. Takes up lots of disk
space and bandwidth, unless ultrasmall linux distros get used.

I don't know which of these three I'll use yet, I may do #2 before #1,
because for smartfrog, we have the build targets to create distributions
already.

Next Steps

-Implement your deployment API, if not already done so
-Implement the test clients
-make sure that your WS-A handling is fairly broad, and handles the
reference properties/params, plus addresses from a broad set of WS-A 
namespaces.
-get your tests running against your local implementation
-get your tests running against a remote implementation
-decide which of the three ways to accessible endpoints you want to
follow, and go about it.

I've said I'm targeting the end of march for an accessible endpoint, and
that is still on my roadmap. There is nothing to stop anyone bringing up
their endpoint earlier and sharing the URL.