[DRMAA-WG] drmaa python

[Cheng Soon Ong]

Hi all,

>> Also, I just recently moved institutions, so while my previous place 
>> was using gridengine, now my new place uses lsf (which unfortunately 
>> does not have a python binding to DRMAA yet). What do you think is the 
>> best strategy to implement a python binding? Is using swig on the C 
>> binding a good option?
> 
> Enrico's implementation will again use a given DRMAA C library, but in a 
> more dynamic fashion without SWIG. This makes it a generic solution for 
> every DRM providing a DRMAA C library.
> 
> If you have enough detailed knowledge of LSF, you could also access LSF 
> directly in a DRMAA Python implementation, either by command-line tools 
> or some proprietary API. From a performance perspective, this might be a 
> better solution than tunneling everything through DRMAA C. And it gives 
> us a second independent implementation, which OGF would love to see.
> 

I don't have detailed knowledge of LSF, or for that matter any implementation of 
cluster systems. I know that FedStage has released a C DRMAA implementation 
which works on our system here.
https://www.fedstage.com/wiki/FedStage_DRMAA_Guide

I'll take a look at Enrico's python DRMAA implementation (when it is out) and 
see whether it will work with FedStage's C library.

The suggestion about doing a direct implementation is interesting. I'm not sure 
I know enough about LSF or our customizations here to be able to do that. But it 
raises an issue that I've recently had. I imagine that many sites would have 
some sort of custom default resource requests (we definitely do here). My naive 
pythonic solution would be to have a base class which has the "basic" drmaa 
requests, and then a derived class which mirrors our local set of submit scripts 
so that jobs submitted via drmaa have identical rights (and default settings) to 
jobs submitted by command line. Would this be a good thing to do? I guess my 
question is more about "How does one customize drmaa for a particular site?". Or 
is it bad to have the class implementing "basic" drmaa requests, because the 
cluster system admins probably would like such jobs.

Cheng