[DRMAA-WG] conf call minutes - Jan. 6th 2010

[Rajic, Hrabri]

To complement the bulk job submission interface, maybe bulk job paradigm could be further abstracted and extended to other interfaces.

At the first glance that would reduce the memory requirements, but also bring all sort of extensions the job control and status interfaces. 

-------
--Hrabri

-----Original Message-----
From: drmaa-wg-bounces at ogf.org [mailto:drmaa-wg-bounces at ogf.org] On Behalf Of Daniel Templeton
Sent: Thursday, January 07, 2010 8:54 AM
To: drmaa-wg at ogf.org
Subject: Re: [DRMAA-WG] conf call minutes - Jan. 6th 2010

I didn't noticed that I agreed to the Job/String duality.  The point 
about dealing with tens of thousands of jobs as Job objects being 
difficult may be true, but I'm not sure I like the idea of handling them 
as String objects instead.  A better approach might be to offer some 
kind of aggregate object, like those silly opaque iterators from the C 
binding.

I guess we need to look at use cases.  The only reason I can see that 
you'd want to wait for 10k jobs and actually look at the returned list 
of jobs, is if you have a list of jobs you launched, and you're waiting 
for them to finish.  Since the job ids are opaque, you can't shortcut 
the job ids with math.  In order to keep a list of 10k jobs, even as 
String objects, you're going to have to take a clever approach.  If 
you're already being clever, having String objects instead of Job 
objects probably doesn't help much.  For example, a clever solution 
might be to use numbered job names so that you can use math to track the 
jobs.  In that case, you want Job objects rather than String objects so 
that you can find out the names of the jobs.  Heck, maybe job name 
should be part of the Job object directly.  In that kind of situation, 
an iterator would be exactly what you want, and it could be smart enough 
not to keep all the Job objects alive at once.

Daniel

Daniel Gruber wrote:
> Attendees:
>
> Peter Troeger
> Dan Templeton
> Roger Brobst
> Marius
> Daniel Gruber
>
>
> 1. Meeting secretary for this meeting?
>
> Daniel Gruber
>
> 2. Report from SAGA / DRMAA F2F meeting, discussion of issues
>
> - 2 days 
> - how to align DRMAA for use within SAGA
> - SAGA has DRMAA1 adapter (interface for interacting with different DRMS)
>
> See DRMAAv2 draft 6:
>
> page 6: Job signaling: 
> -> we don't support that (we are interested in issue operations 
> and not signals)
>
> page 8: Thread safety (because all SAGA calls can be asynchronous)
> -> one distinguished section for each function about multithreading
>
> page 8: Event enumeration: OK
>
> page 8: History: OK (how to store?)?  
>
> page 9: Merge exceptions: OK
>
> page 10: Change of other state changes (not just job state changes)
> -> all should be optional 
> -> still open!!!
>
> page 10: Job Session: Identification by name: OK but we have to 
>                       discuss the semantics 
>
> page 10: getJob(String): OK (added)
>
> page 10: return not just job objects, but also jobs as Strings
>          (performance/scalability): OK
>
> page 11: EventNotification: stays session based because of performance/
>          scalability
>
> page 12: Job template for interactive jobs:
> -> no valid use case 
> -> out of scope 
> -> not all DRMS support that
>          
> File staging: SAGA supports that (sequence of strings)
> - security? 
> - is it DRM stuff? 
> - Condor -> supports file staging
> - LSF/PBS pro 
> - still open issue!!!
>
> 3. New WIKI for DRMAAv2 spec 
>
> http://wikis.sun.com/display/DRMAAv2/Home
>
> 4. Revised document process time plan
>
> - not discussed (timeout)
>
> --
>   drmaa-wg mailing list
>   drmaa-wg at ogf.org
>   http://www.ogf.org/mailman/listinfo/drmaa-wg
>   
--
  drmaa-wg mailing list
  drmaa-wg at ogf.org
  http://www.ogf.org/mailman/listinfo/drmaa-wg