[DRMAA-WG] Java DRMAA implementation for Condor

[Felix, Victor]

Hi Rayson,

Thanks for your email. What I meant by pure Java in this case was a
DRMAA library that does not rely on setting the java.library.path to
load any compiled C/C++ shared objects. In the implementation I'm
working on, I still have to call the Condor binaries (condor_submit,
condor_q) in certain places. In this sense, it would work just the like
Peter's C implementation, which does the same thing. One could argue
that this isn't really "pure" Java since I am relying on the binaries,
but what I'm mainly interested in is avoiding JNI and the associated
complexity that goes along with it (multi-language issues, deployment
issues, etc). Further, there are some scenarios in which JNI usage is
problematic. For instance, let's say you want to use the DRMAA library
in Tomcat for example. Recent versions of Tomcat had problems with
libraries that use JNI (a loading/unloading issue). This may have been
fixed in Tomcat 6, but I'm not sure.

The other problem I ran into was some issues with methods that were not
implemented in the Condor C implementation that became issues for me
when trying to run array jobs. Peter mentioned that these have been
corrected (thanks Peter!), so I should have another look...

-Victor

-----Original Message-----
From: Rayson Ho [mailto:rayrayson at gmail.com] 
Sent: Tuesday, September 09, 2008 2:09 PM
To: Felix, Victor
Cc: drmaa-wg at ogf.org
Subject: Re: [DRMAA-WG] Java DRMAA implementation for Condor

Victor,

(I missed the first part of today's meeting, and only heard the later
part of your intro to the condor-jdrmaa project. May be you mentioned
something that I am missing here)

The condor-jdrmaa SF page says "future versions will reduce the amount
of JNI used, so that eventually, a pure Java..."

IMO, it is hard to implement a pure Java DRMAA library, as the only
way to talk to the Central Manager in Condor, or pbs_server in
PBS/PBSPro/Torque, or mbatchd in LSF is via a TCP/IP socket
connection, yet each DRM has its own protocol.

It's a bit better on the SGE side, as it has JGDI (Java Grid engine
Database Interface) and JMX, but it requires running a JVM in the
qmaster, and it's not really for DRMAA yet:

http://gridengine.sunsource.net/download/workshop10-12_09_07/SGE-WS2007-
FutureSunGridEngineCoreEnhancements_SMI_AndySchwierskott.pdf

And my question is, what's the benefit of getting a pure Java
implementation, and what are the problems that cannot be fixed in the
current implementation of using Java wrappers+JNI+C DRMAA?

Writing new Java code to talk to the master machine from scratch is a
big effort, and doing that for each DRM may introduce compatibility
issues -- SGE developers mentioned before that GDI compatibility is
not guaranteed across versions.

FYI, in 2006 the XGrid DRMAA Summer of Code project uses the Java
wrappers+JNI+C DRMAA, and there were a number of things that Ed did to
get it to work:
http://code.edbaskerville.com/2006/07/11/java-bindings-working/

And you can see the background info here:
http://www.ogf.org/pipermail/drmaa-wg/2006-June/000470.html

Rayson



On 9/2/08, Felix, Victor <vfelix at som.umaryland.edu> wrote:
> Hello,
>
> Just wanted to point out that a new project has been set up on
> SourceForge to provide a Java DRMAA implementation for Condor, which
has
> been missing. It is called Condor-JDRMAA. The current release is
simply
> a wrapper around the C implementation, however future releases will
work
> toward a pure Java implementation...
>
> http://sourceforge.net/projects/condor-jdrmaa/
>
> Best regards,
>
> -Victor
>
> --
>  drmaa-wg mailing list
>  drmaa-wg at ogf.org
>  http://www.ogf.org/mailman/listinfo/drmaa-wg
>