[ogsa-rss-wg] Candidate Ordering Language

[Arvid Norberg]

On Aug 8, 2006, at 13:01, Donal K. Fellows wrote:

> Arvid Norberg wrote:
>> I still don't see where the actual information about the resource   
>> itself becomes available to the candidate ordering language. The  
>> jsdl  contains the requirements of the resource it will be  
>> submitted to,  not a specific resource's properties. The CSG uses  
>> those requirements  to match against each of the resource  
>> available to it, to filter out  those that cannot meet those  
>> requirements. Is the idea that those  requirements are replaced by  
>> the resource's actual properties by the  CSG? (that would sound  
>> slightly counter-intuitive to me, since it  would redefine the  
>> semantics of the jsdl).
>
> It's not at all clear to me why you would care whether a machine is  
> just
> a little bit bigger than the (satisfied) request or much much larger.

It's not completely clear to me either, that was just an assumption.

> If
> my job needs just two processors, how could it possibly matter whether
> I'm on a dual CPU machine or a 512 CPU monster?

Some (admittedly quite weak) use case could be having a low priority  
job that would try to avoid the big clusters and leaving them for the  
big jobs.

> The correct basis to
> distinguish between those two situations has got to be that one will
> cost more than the other, or will start sooner, or something like  
> that.
> Those in turn are really aspects of the quality of service available.
>
> Given that, I wonder whether the problem is really that you're asking
> not quite the right question. If a job really requires some  
> resource, it
> should state that in the input document, as it is a constraint and not
> an ordering input.
>
> On the other hand, it might be an idea to have the machine load be an
> optional part of the QoS terms (optional, because I know it is
> meaningless on some systems such as those that use batch queues).

For those using batch queues, the queue size and queue size limits  
could be interesting optional parts as well.

>>  From the examples you gave of the candidate ordering language:
>>   <!-- Assume that somewhere there's some JSDL -->
>>   <value> //JobDescription/Resources/IndividualCPUCount </value>
>> This looks like the Resources part of the jsdl contains properties  
>> of  the container, and not the requirements of the container. Is  
>> that the  idea?
>
> No. The idea in that example was that the candidates coming back would
> be for different numbers of CPUs (which was an aspect unconstrained or
> loosely constrained in the original input) where that information had
> been written into the job document to be submitted (how else would you
> convey it to the BES?) In that case, the optimization strategy was to
> choose to maximize the number of CPUs used, under the assumption that
> this would minimize runtime (or responsiveness or somesuch). This  
> is not
> to say that this is an optimal strategy though; it's just an  
> example. :-)

Ok, I see.

> Picking up on something said earlier, I'd like to note that I'd expect
> the resource constraints in the Candidate to be at least as tight  
> as the
> resource constraints in the request, though possibly with extra  
> resource
> constraints too. This is because the standard JSDL interpretation is
> that if a resource is not asked for, it is up to the JSDL consumer  
> (i.e.
> the BES) to decide what to do about it. For some resources, this is
> obvious (if I don't ask for lots of processors, I won't get them) but
> for others (e.g. memory) it's less straight-forward, since they may be
> coupled to other resources (e.g. a particular processor will probably
> have a fixed amount of local memory).
>
>> I was thinking along the lines that the jsdl for the job was used  
>> for  that, and that the candidate ordering language didn't  
>> necessarily  need access to that info in order to rank a resource  
>> (since  everything in the jsdl are constants anyway, it is already  
>> known by  the one writing the ranking expression, unless the  
>> Resources part of  the jsdl is rewritten).
>
> As far as I'm concerned, all parts of the JSDL may be rewritten. For
> example, there may be two versions of an application available, one of
> which is for a single-processor system and the other of which is MPI
> enabled. Smoothly switching from the version for a single-processor
> offer to the version for a multi-processor cluster requires  
> substantial
> rewriting of the JSDL (different executable, different arguments,  
> etc.)
> and yet in some sense they refer to the "same" app. Obviously, the
> performance characteristics would differ though. :-)
>
>> Hopefully I've been clear enough of how I'm thinking. You'll have  
>> to  excuse me if I'm misunderstanding something fundamental, I  
>> haven't  been part of any of this and I basically only have access  
>> to the  released drafts on the gridforge site.
>
> I've been so busy (and then away) that I'm a long way behind on  
> writing
> up all these things for GridForge. For that, I must apologize. But
> please keep on asking questions; they'll help a lot!

Thank you!

--
Arvid Norberg