[ogsa-rss-wg] Re: What infromation does CGS expect to have?

[Donal K. Fellows]

Hiro Kishimoto wrote:
> OGSA-WG will have another joint call with CDDLM/ACS/GRAAP on this
> Wednesday (Nov. 9, 10pm GMT). This is continuation of the joint meeting
> in Boston. As you remember, there is a link between ACS and CGS in the
> proposed EMS architecture picture.
> 
> CDDLM-WG and ACS-WG is asking us which information CGS expects to get.
> I guess RSS-WG already has a list of such information. It does not
> need to be final one. Early draft will help us a lot. Is it possible to
> share draft memo with OGSA/CDDLM/ACS/GRAAP on this Wednesday?

I don't have anything in specific at this stage, but if we look at the
basic cases of what we are trying to achieve in the Candidate Set
Generator, we can see that we are really after:

  * A method of enumerating the job containers that are available. Note
    that this is different to the collection of containers that exist;
    knowing about a container factory service would be sufficient too.
    This has to be some kind of directory or registry service.

  * A method of discovering what physical resources are available to jobs
    running within a particular container. This *ought* to be information
    published by the container itself, probably aggregated through some
    kind of information service. (It's possible that combining this with
    the previous major requirement is a good idea).

  * A method of discovering what applications are available within a
    particular container. The meaning of "available" here will also vary
    by site; on our production systems, an app is only available when it
    is also supported by us, and that's quite different from "I can find
    the binary to run"!

Given these are the main requirements as I see them, the question
remains as to how these interact with the ACS and CDDLM space.

Firstly, I view ACS as primarily an information service that answers
questions like "What applications do you know about?" and "What sort of
systems can install from your app archives?" As far as I can tell, the
thing that's needed in this space is some kind of unified scheme for
naming applications; I liked Andrew Grimshaw's strawman suggestion on
this (hierarchical naming like "/appl/chemistry/molecular/gaussian" or
"/appl/biology/cellular/genomics/BLAST") as I'm sure that it is
something that will scale well and which it will be easy to sell to the
end-user community (and have them populate the space, saving us effort!)
Aside from that, the thing that needs to be considered is what extra
information to place within the ACS's metadata. I've no answer to that yet.

CDDLM is more tricky, because that is really information about container
factories. I do not fully understand yet whether CDDLM is the factory
itself, or whether it is a service that configures a job container
created by some other factory. Or a mix of the two; I suspect that this
is the case, and CDDLM isn't couched in terms of the deployment of
services as such, but rather of software that might happen to implement
a service. I do not know how this difference matters.

Given the above, all I think the CSG *needs* at this stage is a way to
get from "this app is (abstractly) available" and "this container is
available" to "this app is (concretely) available in this container"; we
cannot assume that every app that is available anywhere is available in
a particular container, as that precludes heterogenous grids.

However, I do not think that we can go beyond that very much (into the
space of "user wants to run the app, but I have to deploy it somewhere,
so rewrite the job to make that happen") due to the lack of some kind of
easy way of expressing workflow of sufficient complexity. The only other
alternative would be to put some kind of magic tag inside the rewritten
job that the job manager would understand as indicating that deployment
needs to happen before the job can be started. But that's really just
workflow by the back door, and would suck.

Note that neither the CSG (nor the EPS for that matter) can tell CDDLM
to commence deployment since that is a point of committal (since that
deployment might well cause a charge to the user; resources start being
consumed, even if not at anything like as great a rate as when the job
itself is running). This point (that getting a job plan back from the
resource selection services does not imply committal[*]) was agreed upon
during the RSS-WG meeting in Boston.

Or at least this is all my 2 cents.

Alas, I've no time to expand on these few short notes now. Someone
else's turn!

Donal.
[* That means using either "plain" jobs[**] for returns, or WS-Ag
    templates, or WS-Ag pending agreement factory services where the user
    is in the position to say when the commit should happen. I tend to
    favour the first at the moment on the grounds of simplicity. ]
[** The even simpler model, that of handing back just EPRs to the job
     container to run the job in, is out because it may be possible to
     perform other rewrites upon the job in order for it to be possible
     to run the job on a particular container. This is more complex I
     know, but I believe it is unavoidably so. ]