[DRMAA-WG] Meeting Minutes OGF-42, Day 1

[Khalid Shakir]

Hi Bruno,

I've seen your blog posts on possibly using SWIG or JNI for DRMAA. Back
with DRMAAv1, with both a C <http://redmine.ogf.org/documents/10> and a Java
<http://redmine.ogf.org/documents/11> specs available, we implemented a
generic DRMAAv1 Java-to-C binding using JNA <https://github.com/twall/jna>'s
dynamic invocation. While I never had time to support our wrapper
<https://github.com/broadgsa/gatk/tree/9d15d241af5d4a3709e3c76249cff4c2a9053d60/public/gatk-tools-public/src/main/java/org/broadinstitute/gatk/utils/jna/drmaa/v1_0>
as a proper mavenized library, I'd love to see someone publish a similar
DRMAAv2 JNA wrapper as an artifact, of course after both specs are out.

Anecdotally, after a user contributed patch mapped other parts of our
software to PBS specific variables, the above wrapper seemed to work for at
least a another
<http://gatkforums.broadinstitute.org/discussion/2150/queue-and-pbs> without
further modification to our DRMAAv1 JNA code. While I haven't measured the
actual speed of GridEngine-via-JNA vs. GridEngine-via-JNI, this dynamic
wrapper "feels" fast enough for most users, as the JNA probably adds <0.5s
per call, ever. This known cost
<http://developers.opengamma.com/blog/2012/05/25/jna-jni-and-raw-java-performance>
has been worth it since we now only include a single DRMAAv1-JNA wrapper,
versus multiple JNI wrappers for DRMAAv1-PBS, DRMAAv1-GridEngine, etc.

Best wishes,
-k


On Wed, Sep 10, 2014 at 9:10 PM, Bruno P. Kinoshita <
brunodepaulak at yahoo.com.br> wrote:

> Hello Peter
>
> > We spent several years on defining an API that fits to the majority of
> cluster products today and tomorrow, so using the DRMAA2 spec may save you
> some thinking time. Please note also that many features of DRMAA2 are
> optional, so you can start small and extend your implementations
> step-by-step.
>
> That's our plan. And we also believe that using an industry-proven
> software, with formed community and documentation will make it simpler for
> contributors and new developers.
>
> > Univa Grid Engine 8.2 is now supporting DRMAA2 out of the box, which
> would allow you to implement a DRMA2 Java binding by a thin Java-to-C
> conversion layer for this product. If we get DRMAA2 C libraries for the
> other products, this layer would be re-usable (as we had it with DRMAAv1).
>
> That sounds great. I'll try to hack an ansible + vagrant/docker script to
> give it a shot.
>
> > Including your request, I have now a number of serious demands for the
> Java binding spec. I will start with an initial version during the next
> days and keep the list informed. Testing and implementations for that would
> be great anyways.
>
> Thanks! Are you going to use GitHub? If so I can probably bother you there
> with issues and maybe PR's :)
>
> All the best,
> Bruno
>
>   ------------------------------
>  *From:* Peter Tröger <peter at troeger.eu>
> *To:* Bruno P. Kinoshita <brunodepaulak at yahoo.com.br>
> *Cc:* drmaa-wg <drmaa-wg at ogf.org>
> *Sent:* Wednesday, September 10, 2014 4:14 AM
> *Subject:* Re: [DRMAA-WG] Meeting Minutes OGF-42, Day 1
>
> Dear Bruno,
>
> thanks for your feedback, we always need encouraged people to join the
> DRMAA working group.
>
> I am the maintainer of a PBS Plug-in for Jenkins. It uses a custom PBS
> Java API at the moment, but users asked for a generic grid computing
> plug-in for Jenkins, supporting different vendors.
>
> Maybe I can help with the Java binding if necessary, with testing,
> reviewing, writing docs, etc. We are deciding whether to wait for DRMAA v2,
> or write our own little common API in Java for PBS, SGE, etc.
>
>
> We spent several years on defining an API that fits to the majority of
> cluster products today and tomorrow, so using the DRMAA2 spec may save you
> some thinking time. Please note also that many features of DRMAA2 are
> optional, so you can start small and extend your implementations
> step-by-step.
>
> Univa Grid Engine 8.2 is now supporting DRMAA2 out of the box, which would
> allow you to implement a DRMA2 Java binding by a thin Java-to-C conversion
> layer for this product. If we get DRMAA2 C libraries for the other
> products, this layer would be re-usable (as we had it with DRMAAv1).
>
> Including your request, I have now a number of serious demands for the
> Java binding spec. I will start with an initial version during the next
> days and keep the list informed. Testing and implementations for that would
> be great anyways.
>
> Best regards,
> Peter.
>
>
>
>
>
> --
>   drmaa-wg mailing list
>   drmaa-wg at ogf.org
>   https://www.ogf.org/mailman/listinfo/drmaa-wg
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ogf.org/pipermail/drmaa-wg/attachments/20140911/5e6c116f/attachment.html>