[saga-rg] JobService.submitJob() query

[Andre Merzky]

Hi Chris, Graeme, 

if we go for 2, we should actually remove the RM contact
from run_job, as that would conflict with the RM contact
given on the job_service creation.

Cheers, Andre.


Quoting [Christopher Smith] (Mar 06 2006):
> 
> I'll finally provide some context first ....
> 
> As Andre mentioned in a previous email, we had left the choice of resource
> manager up to the implementation of the library, and didn't expose it up
> into the API layer. Feedback indicates that this was a mistake. :-)
> 
> I can go for either 1 or 2, with a preference for 2. The reason is that to
> me the JobService represents the service endpoint to a job scheduler or
> resource manager. That said, I'm not religious about it.
> 
> -- Chris
> 
> 
> On 22/2/06 01:27, "Graeme Pound" <G.E.POUND at soton.ac.uk> wrote:
> 
> > Andre,
> > 
> > I may have confused the issue by mentioning the JobDefinition attribute
> > "SAGA_HostList". This appears to have a valid role mapping to the
> > "CandidateHosts" element of a JSDL document (and also in LSF?). The
> > "SAGA_HostList" attribute should not be used to specify the resource
> > manager.
> > 
> > The problem rather is how is the resource manager (in the form of a URL
> > or machine name) specified when the user invokes submitJob(). There are
> > three possible solutions:
> > 1) Add an argument "host" to the submitJob() method (similar to
> > runJob()) to specify the resource manager
> > 2) Specify the resource manager in the JobService constructor
> > 3) Add a new required attribute to the JobDefinition class (e.g.
> > "SAGA_ResouceManager")
> > 
> > I do not like solution 3 since job definition is conceptually distinct
> > from the choice of resource manager, for example you may wish to define
> > a single JobDefinition and submit that job to several different resource
> > managers.
> > 
> > I do not have a strong preference between solutions 1 and 2. (Although
> > constructors pose a small problem for the Java bindings)
> > 
> > Graeme
> > 
> > 
> > 
> > Andre Merzky wrote:
> >> Hi Graeme, 
> >> 
> >> I am not much of an expert in resource thingies, so what I
> >> say should be taken with a grain of salt.  Or rather with a
> >> spoon I guess...
> >> 
> >> I assumed that run_job ("myhost", ...) would run the job on
> >> myhost.  But what you say (and what the spec implies I
> >> think, after reading again) is that myhost is specifying the
> >> resource manager contact, not the target host.  Right?
> >> 
> >> Well, then we have the ability to specify a resource manager
> >> contact for run_job, and a target resource on job_submit.
> >> But no combination seems possible.
> >> 
> >> Frankly, I think that does not make sense.  For one the
> >> semantics between job_run and job_submit should not be that
> >> different (job_run was supposed to be a shortcut for
> >> job_submit).  Secondly, there are clearly use cases for both
> >> versions (resource manager contact and target resource).
> >> 
> >> One solution would be to specify the resource manager
> >> contact in the constructor of the job_service, as that is
> >> supposed to represent the resource and job manager anyway I
> >> guess.  The host in run_job would then specify the target
> >> resource ("" for 'any resource' I guess).
> >> 
> >> But again, I am not on firm ground in resource management,
> >> and rather would not like to mess w/o feedback of someone
> >> more knowledgeable.  What is your opinion?
> >> 
> >> Thanks, Andre.
> >> 
> >> 
> >> 
> >> Quoting [Graeme Pound] (Feb 21 2006):
> >>> Andre,
> >>> 
> >>> I do not think that the attribute 'SAGA_HostList' of
> >>> JobDefinition is appropriate either. The description of
> >>> 'SAGA_HostList' in the strawman API reads:
> >>> 
> >>>   SAGA_HostList
> >>>    - A list of host names, or host group names, which can be
> >>>      considered by the resource manager as candidate hosts for
> >>>      the job. Whether or not the job actually ends up running
> >>>      on one of the hosts in the list, is solely at the
> >>>      discretion of the resource manager. Vector of strings.
> >>>      (JSDL, LSF)
> >>> 
> >>> This attribute should be used to pass information to the
> >>> resource manager, NOT to specify the resource manager. For
> >>> example it maps to the "CandidateHosts" element of a JSDL
> >>> document.
> >>> 
> >>> The first argument of runJob() is the "host name or IP
> >>> address of the endpoint which will accept and run the
> >>> job". This argument is not defined for submitJob(), nor is
> >>> it defined as an attribute of JobDefinition.
> >>> 
> >>> Graeme
> >> 
> >>> 
> >>> Andre Merzky wrote:
> >>>> Oops, you are right!  My wrong - mixed it up with
> >>>> ExecutionHosts.  Well, then I really was off target:
> >>>> SAGA_HostList is then indeed what you should use to specify
> >>>> the target resource.
> >>>> 
> >>>> The run_job would, in its simpliest implementation, create a
> >>>> job description with SAGA_HostList set to the specified
> >>>> endpoint, and do a submit_job on that description.
> >>>> 
> >>>> Sorry for creating confusion...
> >>>> 
> >>>> Andre.
> >>>> 
> >>>> 
> >>>> Quoting [Graeme Pound] (Feb 20 2006):
> >>>>> Date: Mon, 20 Feb 2006 16:24:32 +0000
> >>>>> From: Graeme Pound <G.E.POUND at soton.ac.uk>
> >>>>> To: Andre Merzky <andre at merzky.net>
> >>>>> CC: SAGA RG <saga-rg at ggf.org>
> >>>>> Subject: Re: [saga-rg] JobService.submitJob() query
> >>>>> 
> >>>>> Andre,
> >>>>> 
> >>>>> 
> >>>>>> 'SAGA_HostList' is on the job, and read only - it gives
> >>>>>> information on where the job _is_ running, not where it
> >>>>>> _should_ run.
> >>>>> I do not follow you here. 'SAGA_HostList' is an attribute of
> >>>>> JobDefinition (not JobInfo), therefore it should not be read only.
> >>>>> 
> >>>>> Graeme
> >>>>> 
> >>>>> 
> >>>>> Andre Merzky wrote:
> >>>>>> Uhm, I think you got us there.  I don't see any way to
> >>>>>> specify the resource either, so its left completely up to
> >>>>>> the backend to schedule the job.  I am not sure if that was
> >>>>>> intended.
> >>>>>> 
> >>>>>> Chris, are we missing something?  Did we intend to leave
> >>>>>> resource specification out?  Can't really be, as we have it
> >>>>>> in run_job as Graeme points out...
> >>>>>> 
> >>>>>> 'SAGA_HostList' is on the job, and read only - it gives
> >>>>>> information on where the job _is_ running, not where it
> >>>>>> _should_ run.
> >>>>>> 
> >>>>>> Andre.
> >>>>>> 
> >>>>>> 
> >>>>>> Quoting [Graeme Pound] (Feb 20 2006):
> >>>>>>> Hi,
> >>>>>>> 
> >>>>>>> Can anybody clear up this issue for me? I may be missing something in
> >>>>>>> the spec.
> >>>>>>> 
> >>>>>>> Thanks,
> >>>>>>> Graeme
> >>>>>>> 
> >>>>>>> 
> >>>>>>> -3.31 How is the resource manager endpoint for JobService.submitJob()
> >>>>>>> specified?
> >>>>>>> 
> >>>>>>> The endpoint is specified by an argument of JobService.runJob(), but is
> >>>>>>> not _obvious_ for submitJob(). Is the endpoint a property of the
> >>>>>>> instance of JobService (with runJob() a 'static' method of the class),
> >>>>>>> or is the endpoint specified as an attribute of JobDefinition?
> >>>>>>> 
> >>>>>>> I assume that the contents of the JobDefinition attribute
> >>>>>>> 'SAGA_HostList' is intended to be passed to the resource manager
> >>>>>>> (rather than specify the resource manager itself); for example mapping
> >>>>>>> to the 'CandidateHosts' element of a JSDL document. Is an additional
> >>>>>>> attribute within JobDefinition required to specify the endpoint?
> >>>>>> 
> >>>> 
> >>>> 
> >> 
> >> 
> >> 
> > 



-- 
"So much time, so little to do..."  -- Garfield