[saga-rg] JobService.submitJob() query

[Graeme Pound]

Andre Merzky wrote:
> Hi Chris, Graeme, 
> 
> if we go for 2, we should actually remove the RM contact
> from run_job, as that would conflict with the RM contact
> given on the job_service creation.
> 
> Cheers, Andre.
> 

Andre,

This provoked my question about job_service.run_job() being a 'static' 
Java method for which job_service need not be instantiated.

However this may be beyond the scope of the language independent SAGA API.

Graeme




> 
> Quoting [Christopher Smith] (Mar 06 2006):
>> I'll finally provide some context first ....
>>
>> As Andre mentioned in a previous email, we had left the choice of resource
>> manager up to the implementation of the library, and didn't expose it up
>> into the API layer. Feedback indicates that this was a mistake. :-)
>>
>> I can go for either 1 or 2, with a preference for 2. The reason is that to
>> me the JobService represents the service endpoint to a job scheduler or
>> resource manager. That said, I'm not religious about it.
>>
>> -- Chris
>>
>>
>> On 22/2/06 01:27, "Graeme Pound" <G.E.POUND at soton.ac.uk> wrote:
>>
>>> Andre,
>>>
>>> I may have confused the issue by mentioning the JobDefinition attribute
>>> "SAGA_HostList". This appears to have a valid role mapping to the
>>> "CandidateHosts" element of a JSDL document (and also in LSF?). The
>>> "SAGA_HostList" attribute should not be used to specify the resource
>>> manager.
>>>
>>> The problem rather is how is the resource manager (in the form of a URL
>>> or machine name) specified when the user invokes submitJob(). There are
>>> three possible solutions:
>>> 1) Add an argument "host" to the submitJob() method (similar to
>>> runJob()) to specify the resource manager
>>> 2) Specify the resource manager in the JobService constructor
>>> 3) Add a new required attribute to the JobDefinition class (e.g.
>>> "SAGA_ResouceManager")
>>>
>>> I do not like solution 3 since job definition is conceptually distinct
>>> from the choice of resource manager, for example you may wish to define
>>> a single JobDefinition and submit that job to several different resource
>>> managers.
>>>
>>> I do not have a strong preference between solutions 1 and 2. (Although
>>> constructors pose a small problem for the Java bindings)
>>>
>>> Graeme
>>>
>>>
>>>
>>> Andre Merzky wrote:
>>>> Hi Graeme, 
>>>>
>>>> I am not much of an expert in resource thingies, so what I
>>>> say should be taken with a grain of salt.  Or rather with a
>>>> spoon I guess...
>>>>
>>>> I assumed that run_job ("myhost", ...) would run the job on
>>>> myhost.  But what you say (and what the spec implies I
>>>> think, after reading again) is that myhost is specifying the
>>>> resource manager contact, not the target host.  Right?
>>>>
>>>> Well, then we have the ability to specify a resource manager
>>>> contact for run_job, and a target resource on job_submit.
>>>> But no combination seems possible.
>>>>
>>>> Frankly, I think that does not make sense.  For one the
>>>> semantics between job_run and job_submit should not be that
>>>> different (job_run was supposed to be a shortcut for
>>>> job_submit).  Secondly, there are clearly use cases for both
>>>> versions (resource manager contact and target resource).
>>>>
>>>> One solution would be to specify the resource manager
>>>> contact in the constructor of the job_service, as that is
>>>> supposed to represent the resource and job manager anyway I
>>>> guess.  The host in run_job would then specify the target
>>>> resource ("" for 'any resource' I guess).
>>>>
>>>> But again, I am not on firm ground in resource management,
>>>> and rather would not like to mess w/o feedback of someone
>>>> more knowledgeable.  What is your opinion?
>>>>
>>>> Thanks, Andre.
>>>>
>>>>
>>>>
>>>> Quoting [Graeme Pound] (Feb 21 2006):
>>>>> Andre,
>>>>>
>>>>> I do not think that the attribute 'SAGA_HostList' of
>>>>> JobDefinition is appropriate either. The description of
>>>>> 'SAGA_HostList' in the strawman API reads:
>>>>>
>>>>>   SAGA_HostList
>>>>>    - A list of host names, or host group names, which can be
>>>>>      considered by the resource manager as candidate hosts for
>>>>>      the job. Whether or not the job actually ends up running
>>>>>      on one of the hosts in the list, is solely at the
>>>>>      discretion of the resource manager. Vector of strings.
>>>>>      (JSDL, LSF)
>>>>>
>>>>> This attribute should be used to pass information to the
>>>>> resource manager, NOT to specify the resource manager. For
>>>>> example it maps to the "CandidateHosts" element of a JSDL
>>>>> document.
>>>>>
>>>>> The first argument of runJob() is the "host name or IP
>>>>> address of the endpoint which will accept and run the
>>>>> job". This argument is not defined for submitJob(), nor is
>>>>> it defined as an attribute of JobDefinition.
>>>>>
>>>>> Graeme
>>>>> Andre Merzky wrote:
>>>>>> Oops, you are right!  My wrong - mixed it up with
>>>>>> ExecutionHosts.  Well, then I really was off target:
>>>>>> SAGA_HostList is then indeed what you should use to specify
>>>>>> the target resource.
>>>>>>
>>>>>> The run_job would, in its simpliest implementation, create a
>>>>>> job description with SAGA_HostList set to the specified
>>>>>> endpoint, and do a submit_job on that description.
>>>>>>
>>>>>> Sorry for creating confusion...
>>>>>>
>>>>>> Andre.
>>>>>>
>>>>>>
>>>>>> Quoting [Graeme Pound] (Feb 20 2006):
>>>>>>> Date: Mon, 20 Feb 2006 16:24:32 +0000
>>>>>>> From: Graeme Pound <G.E.POUND at soton.ac.uk>
>>>>>>> To: Andre Merzky <andre at merzky.net>
>>>>>>> CC: SAGA RG <saga-rg at ggf.org>
>>>>>>> Subject: Re: [saga-rg] JobService.submitJob() query
>>>>>>>
>>>>>>> Andre,
>>>>>>>
>>>>>>>
>>>>>>>> 'SAGA_HostList' is on the job, and read only - it gives
>>>>>>>> information on where the job _is_ running, not where it
>>>>>>>> _should_ run.
>>>>>>> I do not follow you here. 'SAGA_HostList' is an attribute of
>>>>>>> JobDefinition (not JobInfo), therefore it should not be read only.
>>>>>>>
>>>>>>> Graeme
>>>>>>>
>>>>>>>
>>>>>>> Andre Merzky wrote:
>>>>>>>> Uhm, I think you got us there.  I don't see any way to
>>>>>>>> specify the resource either, so its left completely up to
>>>>>>>> the backend to schedule the job.  I am not sure if that was
>>>>>>>> intended.
>>>>>>>>
>>>>>>>> Chris, are we missing something?  Did we intend to leave
>>>>>>>> resource specification out?  Can't really be, as we have it
>>>>>>>> in run_job as Graeme points out...
>>>>>>>>
>>>>>>>> 'SAGA_HostList' is on the job, and read only - it gives
>>>>>>>> information on where the job _is_ running, not where it
>>>>>>>> _should_ run.
>>>>>>>>
>>>>>>>> Andre.
>>>>>>>>
>>>>>>>>
>>>>>>>> Quoting [Graeme Pound] (Feb 20 2006):
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Can anybody clear up this issue for me? I may be missing something in
>>>>>>>>> the spec.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Graeme
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -3.31 How is the resource manager endpoint for JobService.submitJob()
>>>>>>>>> specified?
>>>>>>>>>
>>>>>>>>> The endpoint is specified by an argument of JobService.runJob(), but is
>>>>>>>>> not _obvious_ for submitJob(). Is the endpoint a property of the
>>>>>>>>> instance of JobService (with runJob() a 'static' method of the class),
>>>>>>>>> or is the endpoint specified as an attribute of JobDefinition?
>>>>>>>>>
>>>>>>>>> I assume that the contents of the JobDefinition attribute
>>>>>>>>> 'SAGA_HostList' is intended to be passed to the resource manager
>>>>>>>>> (rather than specify the resource manager itself); for example mapping
>>>>>>>>> to the 'CandidateHosts' element of a JSDL document. Is an additional
>>>>>>>>> attribute within JobDefinition required to specify the endpoint?
>>>>>>
>>>>
>>>>
> 
> 
>