[jsdl-wg] [Fwd: Proposed extension]

[Michel Drescher]

Hi Donal, all,

Donal K. Fellows wrote:
> Michel Drescher wrote:
>> Please find attached two proposed JSDL extensions. The attachment
>> contains the schemas, and a list of examples.
>>
>> NOTE: The multi job extension requires two backwards compatible changes
>> to the JSDL v1.0 schema itself:
>> a) jsdl:JobDescription gets an "id" attribute of type xsd:ID
>> b) jsdl:JobDefinition allows more than one jsdl:JobDescription element.
>>
>> I will update/create the necessary specification documents after the
>> next round of rotten eggs. ;-)
> 
> [...]
> 
> Interesting. I used to think in terms of doing job dependencies like you
> illustrate in example2, but recently I've been considering treating a
> job (or rather the outcome of that job) as a resource in itself. This
> would mean that, once a job has been labelled, all the other things need
> to do to depend on it (i.e. follow in the job-graph) is to put a
> reference to that job in their Resources element. I don't know whether
> this would permit the definition loops, but maybe loops are only ever
> useful for things like parameter sweeps anyway. (Need more real
> use-cases to decide that!)

I know exactly what you mean. I had pretty similar thoughts back then 
when we carved out the *real* UNICORE (if I may be so bold ;-) - back in 
1997. I had had the idea to model a Task so that it has a list of "input 
slots" and a list of "output slots". Each slot was supposed to be a 
typed slot, so that you could couple two tasks by associating two 
compatible slots, one being an output slot, and one being an input slot.

Eventually we decided that this type of model is very much compatible to 
human brains, but (unfortunately) not to relatively dumb computer brains.

> It doesn't solve the other thing that ought to be expressed though. It
> is sometimes *really* useful to be able to specify that two jobs must
> execute at the same time (otherwise the job engine might just run the
> two processes through the same best-effort batch queue, resulting in
> effective serialization of the execution and causing both sub-jobs to
> fail[*]). although this is a dependency of sorts, it is an oddball case
> because each is dependent on the other and yet neither is a resource
> that exists before the other starts (this is why it is the output of the
> job that is the resource BTW). As such, it needs a different method of
> expression.

Touché. :-)

My riposte is as follows. Instead of limiting ourselves to add only 
dependencies - which are unidirectional constraints - we should broaden 
the scope, e.g. by allowing non-dicectional (and/or bidirectional) 
associations. The following may solve your problem:
a) Rename jobgroup:Dependencies to jobgroup:Constraints
b) Have an abstract element named jobgroup:COnstraint
c) Use XML Schema substitution groups for types of constraints:
c.1) jobgroup:Dependency for unidirectional dependencies
c.2) jobgroup:Coupling for jobs that need to co-execute.

Any more suggestions?

Cheers,
Michel

-- 
Michel <dot> Drescher <at> uk <dot> fujitsu <dot> com
Fujitsu Laboratories of Europe
+44 20 8606 4834