[ogsa-wg] Re:[RSS Architecture Discussion]

[Donal K. Fellows]

Soonwook Hwang wrote:
> I was wondering whether you meant "divided single apps" by "parameter 
> sweep types of apps." If that is the case, I don't think that it's less 
> common than "multiple coupled apps," and that it's a rare application.
> I think that it's in fact the kind of applications that will perhaps benefit
> most from exploiting the Grid with the current limitations in
> network bandwidth and latercy between WAN interconnections.
> As we see in case of Condor, Nimrod and other systems. 
> there are many existing systems aiming at tackling "parameter sweep apps."
> In addition, we might be able to consider a collection of single job
> submitted from different job managers to EPS/CSG to be a kind (?) of
> parameter sweep apps that EPS/CSG needs to make mapping plans. In brief,
> the point I am trying to make here is that when it comes to the design of 
> EPS/CSG interface and protocol, having the matching/scheduling of parameter 
> sweep type of apps (i.e., a set of independent tasks of job type) in mind 
> is important as well. 

As far as I can tell, there are two major classes of non-trivial grid
usage here. One is the "we have this complex plan of work to carry out"
(i.e. a workflow) and the other is "we want to do this long list of
fairly simple things" (i.e. a parameter sweep). These two major classes
of system have very different properties; in the workflow, the main
desire is to coordinate what may be very large atomic jobs to carry out
some larger task and the resource-selection plan is focussed on
maximizing the likelihood of success while minimizing the cost, and in
the parameter sweep the focus is no longer on the individual executions
but on the maximizing of "job throughput" and resource-selection can be
done more on the basis of "who is free for another work packet". Indeed
I suspect that the best approach might involve the use of a CSG to
select a set of worker containers, with very little planning at all. Oh
well, that would at least demonstrate the need for having the CSG as an
entity separate to the EPS...

Donal.