[saga-rg] JobService.submitJob() query

[Christopher Smith]

I'll finally provide some context first ....

As Andre mentioned in a previous email, we had left the choice of resource
manager up to the implementation of the library, and didn't expose it up
into the API layer. Feedback indicates that this was a mistake. :-)

I can go for either 1 or 2, with a preference for 2. The reason is that to
me the JobService represents the service endpoint to a job scheduler or
resource manager. That said, I'm not religious about it.

-- Chris


On 22/2/06 01:27, "Graeme Pound" <G.E.POUND at soton.ac.uk> wrote:

> Andre,
> 
> I may have confused the issue by mentioning the JobDefinition attribute
> "SAGA_HostList". This appears to have a valid role mapping to the
> "CandidateHosts" element of a JSDL document (and also in LSF?). The
> "SAGA_HostList" attribute should not be used to specify the resource
> manager.
> 
> The problem rather is how is the resource manager (in the form of a URL
> or machine name) specified when the user invokes submitJob(). There are
> three possible solutions:
> 1) Add an argument "host" to the submitJob() method (similar to
> runJob()) to specify the resource manager
> 2) Specify the resource manager in the JobService constructor
> 3) Add a new required attribute to the JobDefinition class (e.g.
> "SAGA_ResouceManager")
> 
> I do not like solution 3 since job definition is conceptually distinct
> from the choice of resource manager, for example you may wish to define
> a single JobDefinition and submit that job to several different resource
> managers.
> 
> I do not have a strong preference between solutions 1 and 2. (Although
> constructors pose a small problem for the Java bindings)
> 
> Graeme
> 
> 
> 
> Andre Merzky wrote:
>> Hi Graeme, 
>> 
>> I am not much of an expert in resource thingies, so what I
>> say should be taken with a grain of salt.  Or rather with a
>> spoon I guess...
>> 
>> I assumed that run_job ("myhost", ...) would run the job on
>> myhost.  But what you say (and what the spec implies I
>> think, after reading again) is that myhost is specifying the
>> resource manager contact, not the target host.  Right?
>> 
>> Well, then we have the ability to specify a resource manager
>> contact for run_job, and a target resource on job_submit.
>> But no combination seems possible.
>> 
>> Frankly, I think that does not make sense.  For one the
>> semantics between job_run and job_submit should not be that
>> different (job_run was supposed to be a shortcut for
>> job_submit).  Secondly, there are clearly use cases for both
>> versions (resource manager contact and target resource).
>> 
>> One solution would be to specify the resource manager
>> contact in the constructor of the job_service, as that is
>> supposed to represent the resource and job manager anyway I
>> guess.  The host in run_job would then specify the target
>> resource ("" for 'any resource' I guess).
>> 
>> But again, I am not on firm ground in resource management,
>> and rather would not like to mess w/o feedback of someone
>> more knowledgeable.  What is your opinion?
>> 
>> Thanks, Andre.
>> 
>> 
>> 
>> Quoting [Graeme Pound] (Feb 21 2006):
>>> Andre,
>>> 
>>> I do not think that the attribute 'SAGA_HostList' of
>>> JobDefinition is appropriate either. The description of
>>> 'SAGA_HostList' in the strawman API reads:
>>> 
>>>   SAGA_HostList
>>>    - A list of host names, or host group names, which can be
>>>      considered by the resource manager as candidate hosts for
>>>      the job. Whether or not the job actually ends up running
>>>      on one of the hosts in the list, is solely at the
>>>      discretion of the resource manager. Vector of strings.
>>>      (JSDL, LSF)
>>> 
>>> This attribute should be used to pass information to the
>>> resource manager, NOT to specify the resource manager. For
>>> example it maps to the "CandidateHosts" element of a JSDL
>>> document.
>>> 
>>> The first argument of runJob() is the "host name or IP
>>> address of the endpoint which will accept and run the
>>> job". This argument is not defined for submitJob(), nor is
>>> it defined as an attribute of JobDefinition.
>>> 
>>> Graeme
>> 
>>> 
>>> Andre Merzky wrote:
>>>> Oops, you are right!  My wrong - mixed it up with
>>>> ExecutionHosts.  Well, then I really was off target:
>>>> SAGA_HostList is then indeed what you should use to specify
>>>> the target resource.
>>>> 
>>>> The run_job would, in its simpliest implementation, create a
>>>> job description with SAGA_HostList set to the specified
>>>> endpoint, and do a submit_job on that description.
>>>> 
>>>> Sorry for creating confusion...
>>>> 
>>>> Andre.
>>>> 
>>>> 
>>>> Quoting [Graeme Pound] (Feb 20 2006):
>>>>> Date: Mon, 20 Feb 2006 16:24:32 +0000
>>>>> From: Graeme Pound <G.E.POUND at soton.ac.uk>
>>>>> To: Andre Merzky <andre at merzky.net>
>>>>> CC: SAGA RG <saga-rg at ggf.org>
>>>>> Subject: Re: [saga-rg] JobService.submitJob() query
>>>>> 
>>>>> Andre,
>>>>> 
>>>>> 
>>>>>> 'SAGA_HostList' is on the job, and read only - it gives
>>>>>> information on where the job _is_ running, not where it
>>>>>> _should_ run.
>>>>> I do not follow you here. 'SAGA_HostList' is an attribute of
>>>>> JobDefinition (not JobInfo), therefore it should not be read only.
>>>>> 
>>>>> Graeme
>>>>> 
>>>>> 
>>>>> Andre Merzky wrote:
>>>>>> Uhm, I think you got us there.  I don't see any way to
>>>>>> specify the resource either, so its left completely up to
>>>>>> the backend to schedule the job.  I am not sure if that was
>>>>>> intended.
>>>>>> 
>>>>>> Chris, are we missing something?  Did we intend to leave
>>>>>> resource specification out?  Can't really be, as we have it
>>>>>> in run_job as Graeme points out...
>>>>>> 
>>>>>> 'SAGA_HostList' is on the job, and read only - it gives
>>>>>> information on where the job _is_ running, not where it
>>>>>> _should_ run.
>>>>>> 
>>>>>> Andre.
>>>>>> 
>>>>>> 
>>>>>> Quoting [Graeme Pound] (Feb 20 2006):
>>>>>>> Hi,
>>>>>>> 
>>>>>>> Can anybody clear up this issue for me? I may be missing something in
>>>>>>> the spec.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Graeme
>>>>>>> 
>>>>>>> 
>>>>>>> -3.31 How is the resource manager endpoint for JobService.submitJob()
>>>>>>> specified?
>>>>>>> 
>>>>>>> The endpoint is specified by an argument of JobService.runJob(), but is
>>>>>>> not _obvious_ for submitJob(). Is the endpoint a property of the
>>>>>>> instance of JobService (with runJob() a 'static' method of the class),
>>>>>>> or is the endpoint specified as an attribute of JobDefinition?
>>>>>>> 
>>>>>>> I assume that the contents of the JobDefinition attribute
>>>>>>> 'SAGA_HostList' is intended to be passed to the resource manager
>>>>>>> (rather than specify the resource manager itself); for example mapping
>>>>>>> to the 'CandidateHosts' element of a JSDL document. Is an additional
>>>>>>> attribute within JobDefinition required to specify the endpoint?
>>>>>> 
>>>> 
>>>> 
>> 
>> 
>> 
>