[drmaa-wg] SGE, drmaa_job_ps and DRMAA sessions

[Tim Harsch]

So according to the spec Ancor's method should work right?

I know that SGE was way early to adopt DRMAA support, in fact DRMAA is not
yet at a 1.0 status correct?  What I'm getting at is that perhaps this
section of the spec was not complete by the time the SGE development team
had written the support.  Maybe we can just encourage SGE devlopment team to
look at the problem and hope it makes it into the next patch?

----- Original Message ----- 
From: "Áncor González Sosa" <ancorgs at softhome.net>
To: "Tim Harsch" <harsch1 at llnl.gov>
Cc: "Peter Troeger" <peter.troeger at hpi.uni-potsdam.de>; <drmaa-wg at ggf.org>
Sent: Friday, March 04, 2005 3:14 PM
Subject: Re: [drmaa-wg] SGE, drmaa_job_ps and DRMAA sessions


El Viernes, 4 de Marzo de 2005 22:02, Tim Harsch escribió:
> Thanks Peter,
>     I'm a little rusty on what DRMAA on SGE is doing for sure, perhaps
> Ancor can be a bit more specific as to the behaviour he is seeing.

Simple:
1) Open a DRMAA session, submit a job and store the returned jobid

2) Close the sessión

3) Open a new session and call drmaa_job_ps($myJobId)
   The call fails because the job id is not recognized.

I have finally implemented it executing sge commands and parsing the output
and everything works fine (but I hate this solution, I clearly prefer
DRMAAc).

> ----- Original Message -----
> From: "Peter Troeger" <peter.troeger at hpi.uni-potsdam.de>
> To: "Tim Harsch" <harsch1 at llnl.gov>
> Cc: "Áncor González Sosa" <ancorgs at softhome.net>; <drmaa-wg at ggf.org>
> Sent: Friday, March 04, 2005 1:56 PM
> Subject: Re: [drmaa-wg] SGE, drmaa_job_ps and DRMAA sessions
>
>
> Sorry for the late answer. I'm not a SGE expert, but DRMAA section 3.1.2
> is very clear about this:
>
> "... Job Id’s SHALL remain valid from one session to another. Job
> control routines SHOULD work
> correctly if a job Id came from a previous DRMAA session, provided the
> current DRMAA session
> knows how to resolve this job Id. The burden is on the user to match
> previous job Id’s with
> appropriate DRMAA sessions (i.e., DRMAA implementations). It is
> RECOMMENDED that
> restartable applications make job Id’s persistent in order to access the
> already submitted jobs."
>
> In sum, you need to persist your job id's with an own mechanism. It
> SHOULD be allowed to re-use old job id's in a new session. This approach
> ensures that the DRMAA library is not enforced to do any kind of data
> storage.
> If SGE works as described, than the "SHOULD" keyword from the
> specification is ignored. If this was intentional, then I would be
> interested in the reasons (Andreas ? Dan ?). I really agree to the
> argumentation of Tim, so maybe the "SHOULD" keyword has to be a "SHALL".
>
> Regards,
> Peter.
>
> Tim Harsch schrieb:
> >Ancor,
> >    Someone who knows the spec better than I should probably confirm but
I
> >believe drmaa_job_ps is indeed only tracking the jobs submitted from
> > within the session.  In my opinion, for drmaa_job_ps, it is indeed how
> > the behaviour should be from a sort of object oriented perspective...
> > however, that doesn't meand there shouldn't perhaps be a sessionless set
> > of API functions designed for manipulating jobs on the grid as a whole.
> > I don't know if this has been discussed for DRMAA before or not.
> >    I can see how this would be a problem for a set of CGIs, since each
> > are a new process you'll not be able to maintain the drmaa_session
> > accross processes.  I don't know if there is a way around it...
> >    You might have to consider maintaining your job ids in a persistent
> >store of your own and backtick calls to qsub, etc.  Its ugly in that
> > you'll have the process overhead each time, but you'll get more of the
> > SGE
>
> specific
>
> >features that way.
> >
> >    Perhaps the group can help tell us if there is a way to set up a
> >persistent DRMAA session?
> >
> >    Wouldn't this problem also be a problem for Globus?  At one of the
> >conferences they were saying that Globus would (in the future) perhaps
use
> >DRMAA as the tie in to the DRM, seems like without persistent sessions
> > this might be a bit difficult in practice.
> >
> >HTH,
> >Tim
> >
> >----- Original Message -----
> >From: "Áncor González Sosa" <ancorgs at softhome.net>
> >To: <drmaa-wg at ggf.org>
> >Sent: Wednesday, March 02, 2005 3:34 PM
> >Subject: [drmaa-wg] SGE, drmaa_job_ps and DRMAA sessions
> >
> >
> >I'm developing a webmin module for submiting jobs and fetching the
results
> >in
> >a cluster with Sun Grid Engine 6.0u1. I'm using Scheduler::DRMAAc, the
> > Perl wrapper of the C binding of the DRMAA specification.
> >
> >My problem is that drmaa_job_ps() only works for jobs that have been
> >submited
> >in the current DRMAA session. Is this the expected behavior in any DRMAA
> >implementation or is a feature/bug of SGE 6? I'm not sure, even after
> >reading
> >the DRMAA 1.0 specification paper. If it is the normal behavior of a
DRMAA
> >system, I find DRMAA really useless for "unattended" execution.
> >
> >Greetings

-- 
                                        .--.
  LINUX                                |o_o |
                                       |¡_/ |
  Usuario registrado #239475          //   \ \
                                     (|     | )
  Áncor González Sosa               /'\_   _/`\
  ancorgs at softhome.net              \___)=(___/

  Debian GNU/Linux 3.0 (Woody)