[DRMAA-WG] Treat job templates as value types ?

Mariusz Mamoński mamonski at man.poznan.pl
Fri Mar 26 09:23:04 CDT 2010 

Hi,

 I have tried to collect all the problems, suggestions, and also my
experience with different APIs and created a scratch page on my wiki:
"How the JobTamplate could look if.. ") with several different
possible options:

http://fury.man.poznan.pl/~mmamonski/wiki/index.php/DRMAAv2/JobTemplate_-_Bindings

Please review every of the 4 options and provide your comments! We can
always combine some ideas.

After writing this summary i asked one dummy question to myself: "What
is the benefit of having vendor specific attributes if we already have
nativeOptions (nativeSpecification)"?

Cheers,

On 24 March 2010 14:28, Andre Merzky <andre at merzky.net> wrote:
> Quoting [Daniel Templeton] (Mar 24 2010):
>>
>> Andre,
>>
>> Don't worry about being biased.  I'm unabashedly SGE-biased. :)
>
> :-)
>
>
>> Unfortunately, though, what you proposed is what we're already
>> discussing.  It's the problem, not the solution.  We're trying to have a
>> consistent API across OO and non-OO languages.  The issue goes away if
>> we just add "except in C" to the description.
>
> Hmm, I thought the problem was:
>
>>>>>>> just only one concern: what if one of the DRM vendor wish to
>>>>>>> provide some JobTemplate attributes additional to those
>>>>>>> specified in DRMAA (as i remember SGE was an example)? I
>>>>>>> expect having different struct definition  (different
>>>>>>> drmaa2.h ) leads to serious problems. I'm not saying no
>>>>>>> (actually having a self allocated struct in C is more
>>>>>>> convenient than using getter/setters - e.g. error handling),
>>>>>>> but this, for me, requires addressing the extension
>>>>>>> methodology on the IDL level.
>
> The SAGA scheme would still map to C in a very simple way
>
>  // create job template
>  drmaa_job_template_t jt = drmaa_job_template_create ();
>
>  // set a predefined DRMAA attribute
>  // these calls would always succeed, basically.
>  drmaa_job_template_set_attribute  (jt, DRMAA_JOB_TEMPLATE_EXECUTABLE, "/bin/date");
>  drmaa_job_template_set_attribute  (jt, "Executable", "/bin/date");
>  drmaa_job_template_set_executable (jt, "/bin/date");
>
>  // set a vendor specific attribute
>  // this call would return an error if the attribute is not
>  // supported
>  drmaa_job_template_set_attribute  (jt, "SGE:Executable-Type", "UniversalBinary");
>
>  // free job template
>  drmaa_job_template_destroy (jt);
>
> So, no special rule for C, and the same drmaa2.h for all vendors.
>
> Well, the special rule for C is to have a Create and Destrpy method
> - but you have that for all classes/types, job template is not
> special in that respect...
>
> Cheers, Andre.
>
>
>> Andre does bring up a valid point, though, which is that we really could
>> just special-case the non-OO languages, just like we did in v1.  Or,
>> like I said before, we could leave the template management to the
>> implementation, scoped to the session, with an overflow value of being
>> allowed to free templates as needed.
>>
>> Daniel
>>
>> On 03/24/10 06:03, Andre Merzky wrote:
>> >my biased point of view: use the saga model:
>> >
>> >   {
>> >     // create job template on the stack
>> >     saga::job::description jd;
>> >
>> >     // 3 equivalent ways for predefined attribute
>> >     jd.set_attribute (saga::job::description_executable, "/bin/date");
>> >     jd.set_attribute ("Executable", "/bin/date");
>> >     jd.set_executable ("/bin/date");
>> >
>> >     // vendor-defined attributes:
>> >     jd.set_attribute ("SGE:Executable-Type", "UniversalBinary");
>> >
>> >     // job template gets deleted from the stack when leaving the scope
>> >   }
>> >
>> >Memory management of the hash table is done internally.  The
>> >implementation also needs to maintain a list of predefined
>> >attributes - but that is anyway the case I think.  C language
>> >bindings could of course still fall back to the explicit
>> >create/delete methods.
>> >
>> >Again, sorry for being biased...
>> >
>> >Best, Andre.
>> >
>> >
>> >Quoting [Daniel Templeton] (Mar 23 2010):
>> >>
>> >>It's not a bad idea.  I would really like to find a way to make it work,
>> >>because it would make life quite a bit simpler.  Is there a way to make
>> >>it work?
>> >>
>> >>Daniel
>> >>
>> >>On 03/23/10 13:56, Peter Tröger wrote:
>> >>>Ok, it was a bad idea. Thanks for commenting.
>> >>>
>> >>>/Peter.
>> >>>
>> >>>Am 22.03.2010 um 16:14 schrieb Daniel Templeton:
>> >>>
>> >>>>And that may be the original reason why the SGE C binding uses that
>> >>>>pseudo-OO structure.  It is more or less a hash map, allowing
>> >>>>arbitrary
>> >>>>name-value pairs to be stored, facilitating vendor-defined job
>> >>>>template
>> >>>>attributes.  If we can't find a way around that issue, it's a
>> >>>>show-stopper.  We have to allow for vendor-defined attributes.
>> >>>>
>> >>>>Daniel
>> >>>>
>> >>>>On 03/22/10 07:29, Mariusz Mamo??ski wrote:
>> >>>>>Hi Peter, all,
>> >>>>>
>> >>>>>2010/3/22 Peter Tröger<peter at troeger.eu>:
>> >>>>>>Dear all,
>> >>>>>>
>> >>>>>>we have an ongoing discussion about the possible removal of
>> >>>>>>"createJobTemplate" and "deleteJobTemplate". The last proposal was
>> >>>>>>to move this functions to the language bindings that need it. This
>> >>>>>>is currently only C - all other languages have their own way of
>> >>>>>>performing instantiation and termination explicitly.
>> >>>>>>
>> >>>>>>After some more thinking, I think I got the true underlying issue.
>> >>>>>>So far, we are treating job templates as instances with state and
>> >>>>>>behavior - objects, in most languages. The only reason for doing
>> >>>>>>this (so far) is the ability to throw errors on attribute access,
>> >>>>>>since we need that for DRMAA's understanding of optional job
>> >>>>>>template attributes. However, from all other perspectives, job
>> >>>>>>templates are just value types. If you got one, you fill it, and
>> >>>>>>then you pass the thing as a whole. In a RPC scenario, you would
>> >>>>>>also expect to send filled job templates as a whole, similar to a
>> >>>>>>filled JSDL document.
>> >>>>>>
>> >>>>>>So if we change that, what would that mean:
>> >>>>>>
>> >>>>>>C:  Generic getter / setter functions for job template attributes
>> >>>>>>would go away. Instead, you would create / delete JobTemplate
>> >>>>>>structs directly. runJob() would take a pointer to this struct.
>> >>>>>>Explicit removal is no longer needed, since the stack is cleared
>> >>>>>>automatically.
>> >>>>>>
>> >>>>>>C# : JobTemplate class would become JobTemplate struct.
>> >>>>>>
>> >>>>>>Java / Python: JobTemplate class remains JobTemplate class, since
>> >>>>>>they have no struct concept. Creation and destruction can be
>> >>>>>>managed with OO mechanisms.
>> >>>>>>
>> >>>>>>Another effect would be a change in the semantics of optional
>> >>>>>>attributes. We already demand the syntactical inclusion of
>> >>>>>>optional attribute names in the class definition. In C, the usage
>> >>>>>>of a non-implemented optional attribute names gives a specialized
>> >>>>>>error. With the change, we would have struct members that you are
>> >>>>>>not allowed to fill in with some DRMAA libraries. But this would
>> >>>>>>be detected only at submission time, since struct changes do not
>> >>>>>>involve library code.
>> >>>>>>
>> >>>>>just only one concern: what if one of the DRM vendor wish to provide
>> >>>>>some JobTemplate attributes additional to those specified in DRMAA
>> >>>>>(as
>> >>>>>i remember SGE was an example)? I expect having different struct
>> >>>>>definition  (different drmaa2.h ) leads to serious problems. I'm not
>> >>>>>saying no (actually having a self allocated struct in C is more
>> >>>>>convenient than using getter/setters - e.g. error handling), but
>> >>>>>this,
>> >>>>>for me, requires addressing the extension methodology on the IDL
>> >>>>>level.
>> >>>>>
>> >>>>>>And "createJobTemplate" resp. "deleteJobTemplate" ? Not needed at
>> >>>>>>all then, regardless of the language.
>> >>>>>>
>> >>>>>>Best,
>> >>>>>>Peter.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>--
>> >>>>>>   drmaa-wg mailing list
>> >>>>>>   drmaa-wg at ogf.org
>> >>>>>>   http://www.ogf.org/mailman/listinfo/drmaa-wg
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>Cheers,
>> >>>>--
>> >>>>   drmaa-wg mailing list
>> >>>>   drmaa-wg at ogf.org
>> >>>>   http://www.ogf.org/mailman/listinfo/drmaa-wg
>> >>>
>> >>>--
>> >>>    drmaa-wg mailing list
>> >>>    drmaa-wg at ogf.org
>> >>>    http://www.ogf.org/mailman/listinfo/drmaa-wg
>>
>
>
>
> --
> Nothing is ever easy.
> --
>  drmaa-wg mailing list
>  drmaa-wg at ogf.org
>  http://www.ogf.org/mailman/listinfo/drmaa-wg
>



-- 
Mariusz

More information about the drmaa-wg mailing list