[drmaa-wg] SGE, drmaa_job_ps and DRMAA sessions

[Daniel Templeton]

The reason SGE doesn't allow drmaa_job_ps() for jobs that aren't from 
the current session is that the function implementation is built around 
the internal state of the session.  I will have to look a little deeper 
to see why it is that way and if it can be changed.  Same goes for 
drmaa_wait() and drmaa_synchronize().  In the mean time, Ancor, could 
you submit a bug to the http://gridengine.sunsource.net site for the issue?

The drmaa_control() function does work on jobs that aren't from the 
current session.

Daniel

Áncor González Sosa wrote:

> El Viernes, 4 de Marzo de 2005 22:02, Tim Harsch escribió:
> 
>>Thanks Peter,
>>    I'm a little rusty on what DRMAA on SGE is doing for sure, perhaps
>>Ancor can be a bit more specific as to the behaviour he is seeing.
> 
> 
> Simple:
> 1) Open a DRMAA session, submit a job and store the returned jobid
> 
> 2) Close the sessión
> 
> 3) Open a new session and call drmaa_job_ps($myJobId)
>    The call fails because the job id is not recognized.
> 
> I have finally implemented it executing sge commands and parsing the output 
> and everything works fine (but I hate this solution, I clearly prefer 
> DRMAAc).
> 
> 
>>----- Original Message -----
>>From: "Peter Troeger" <peter.troeger at hpi.uni-potsdam.de>
>>To: "Tim Harsch" <harsch1 at llnl.gov>
>>Cc: "Áncor González Sosa" <ancorgs at softhome.net>; <drmaa-wg at ggf.org>
>>Sent: Friday, March 04, 2005 1:56 PM
>>Subject: Re: [drmaa-wg] SGE, drmaa_job_ps and DRMAA sessions
>>
>>
>>Sorry for the late answer. I'm not a SGE expert, but DRMAA section 3.1.2
>>is very clear about this:
>>
>>"... Job Id’s SHALL remain valid from one session to another. Job
>>control routines SHOULD work
>>correctly if a job Id came from a previous DRMAA session, provided the
>>current DRMAA session
>>knows how to resolve this job Id. The burden is on the user to match
>>previous job Id’s with
>>appropriate DRMAA sessions (i.e., DRMAA implementations). It is
>>RECOMMENDED that
>>restartable applications make job Id’s persistent in order to access the
>>already submitted jobs."
>>
>>In sum, you need to persist your job id's with an own mechanism. It
>>SHOULD be allowed to re-use old job id's in a new session. This approach
>>ensures that the DRMAA library is not enforced to do any kind of data
>>storage.
>>If SGE works as described, than the "SHOULD" keyword from the
>>specification is ignored. If this was intentional, then I would be
>>interested in the reasons (Andreas ? Dan ?). I really agree to the
>>argumentation of Tim, so maybe the "SHOULD" keyword has to be a "SHALL".
>>
>>Regards,
>>Peter.
>>
>>Tim Harsch schrieb:
>>
>>>Ancor,
>>>   Someone who knows the spec better than I should probably confirm but I
>>>believe drmaa_job_ps is indeed only tracking the jobs submitted from
>>>within the session.  In my opinion, for drmaa_job_ps, it is indeed how
>>>the behaviour should be from a sort of object oriented perspective...
>>>however, that doesn't meand there shouldn't perhaps be a sessionless set
>>>of API functions designed for manipulating jobs on the grid as a whole. 
>>>I don't know if this has been discussed for DRMAA before or not.
>>>   I can see how this would be a problem for a set of CGIs, since each
>>>are a new process you'll not be able to maintain the drmaa_session
>>>accross processes.  I don't know if there is a way around it...
>>>   You might have to consider maintaining your job ids in a persistent
>>>store of your own and backtick calls to qsub, etc.  Its ugly in that
>>>you'll have the process overhead each time, but you'll get more of the
>>>SGE
>>
>>specific
>>
>>
>>>features that way.
>>>
>>>   Perhaps the group can help tell us if there is a way to set up a
>>>persistent DRMAA session?
>>>
>>>   Wouldn't this problem also be a problem for Globus?  At one of the
>>>conferences they were saying that Globus would (in the future) perhaps use
>>>DRMAA as the tie in to the DRM, seems like without persistent sessions
>>>this might be a bit difficult in practice.
>>>
>>>HTH,
>>>Tim
>>>
>>>----- Original Message -----
>>>From: "Áncor González Sosa" <ancorgs at softhome.net>
>>>To: <drmaa-wg at ggf.org>
>>>Sent: Wednesday, March 02, 2005 3:34 PM
>>>Subject: [drmaa-wg] SGE, drmaa_job_ps and DRMAA sessions
>>>
>>>
>>>I'm developing a webmin module for submiting jobs and fetching the results
>>>in
>>>a cluster with Sun Grid Engine 6.0u1. I'm using Scheduler::DRMAAc, the
>>>Perl wrapper of the C binding of the DRMAA specification.
>>>
>>>My problem is that drmaa_job_ps() only works for jobs that have been
>>>submited
>>>in the current DRMAA session. Is this the expected behavior in any DRMAA
>>>implementation or is a feature/bug of SGE 6? I'm not sure, even after
>>>reading
>>>the DRMAA 1.0 specification paper. If it is the normal behavior of a DRMAA
>>>system, I find DRMAA really useless for "unattended" execution.
>>>
>>>Greetings
> 
>