[SAGA-RG] Fwd (andre at merzky.net): Re: JSDL - SAGA

Fri Feb 16 07:33:56 CST 2007

Andre Merzky wrote: 

> Quoting [Thilo Kielmann] (Feb 16 2007):

Let me state the obvious first: I kind of agree with Thilo. We should
support whatever wide spread used (read: standardized) job description
language future may bring. SAGA is all about standards and we should align
SAGA with other standards, especially if these are produced by the same body
as SAGA (OGF).

> > I think we are having conflicting goals here.
> > (Technical goals, not personal ones ;-)
> > 
> > On one hand, we have the "S for simplicity" in SAGA, and we 
> must keep it.
> > 
> > On the other hand, we have the necessity to support JSDL, 
> future JSDL 
> > extensions, or any other types of job decscriptions that 
> people want to use.
> > (And still might to be invented.)
> 
> That is one of the points I kind of disagree with:  Do we 
> really have the necessity to support native JSDL, or even
> JSDL++?
> 
> The _only_ point in our use cases where JSDL is explicitely 
> mentioned is the RealityGrid use case
> 
> 11. Grid Technologies currently used:
> -------------------------------------
> 
>   If you are currently using or developing this scenario, which
>   grid technologies are you using or considering?
> 
>     Currently using: gsissh, globus-url-copy, globus-job-run,
>     GSI, WSDL, Grid Services (OGSI::Lite), SOAP, XML, GridSphere,
>     Access Grid, VizServer, Chromium, gsoap.
> 
>     Currently considering WS-RF, WS-Notification, WS-Security,
>     SRB, JSDL, UNICORE, Sakai.
> 
> 
> '_consider_' using JSDL, and after listing WS-RF and other 
> WS-Standards. 
> 
> I also think that this is the level where JSDL is aiming at:
> the JSDL spec specifies the target use cases of JSDL 
> documents to be exchanged between WS clients and WS services
> - it is, IMHO, not intended to be a human readable or API 
> level language (see paragraph 2,3 of introduction, and figure 
> 1 in the JSDL spec).

Agreed. JSDL is not meant to be human readable. But who cares? 

> > Simultaneously, users are likely to use JSDL++ themselves, and may 
> > wnat to use JSDL++ to express their resource needs.
> 
> That is what I doubt: users will in general NOT write XML 
> documents manually.  Even simple JSDL docs are not 
> neccessarily human readable (IMHO), and the extensions will 
> be, well, more interesting (from what I have seen about them 
> so far).  So that is why I think that...

As I said: who cares? If JSDL++ will be widely adopted we can be reasonably
sure we will get a lot of tooling support to generate these XML documents
without (direct) human intervention. 

> > At this point in time, SAGA will sit in the middle, and it 
> may be very 
> > clumsy to first translate from JSDL++ to a SAGA job 
> description, and 
> > then back to
> > JSDL++ somewhere "down under" in the implementation.
> 
> ... this situation will be the exception, not the rule.

Who are you to know? We don't know yet, what people will do with SAGA and I
think this is part of the problem. We have the use cases - yes. But this was
only a starting point, only a first impression we got from a couple of
groups of interested people. Wide adoption of SAGA will produce a lot more
use cases we need to be prepared for and not constraining our selfs against
common sense is the only thing we can do now.

> > For the very purpose of SAGA as a universal and simple grid 
> API, it has to: 
> > - be independent of job description standards (mostly simpler than 
> > them)
> 
> We are!  That is how our JD was derived/defined.
> 
> 
> > - support job description standards
> 
> We do!  Our JD is easily translatable into JSDL.

But no v.v.

> > My suggestion:
> > 
> > SAGA should have one class of job descriptions, and the 
> possibility to 
> > create subclasses for more specific job descriptions (like JSDL).
> > Such subclasses could be defined as separate extension 
> documents (just 
> > like gridcpr). (Or was it "resource descriptions"???)
> 
> The problem with that is that you'd allow, at some point, 
> also Globus-RSL job descriptions, or Naregi-RSL job 
> descriptions, which bind your application to the specific 
> back ends, and hence break portability. (BTW, I think the 
> number of users which would like to re-use RSL is certainly 
> larger than thos who want to reuse JSDL...)

No! RSL will not be standardized by OGF as a job description standard.

> > With this approach, users could still write programs that are 
> > independent of the underlying job submission machinery, having a 
> > simplified view on jobs and resource attributes etc. etc.
> > At the same time, subclassed job/resource descriptions could be 
> > "passed through" transparently from the API to the implementeation, 
> > without being converted back and forth, a process that is 
> very likely 
> > to loose some important details.
> 
> passing things through is a bad idea - that is what we 
> learned with GAT.  I hoestly think we should stay away from 
> that, as far as possible.  

Passing things through is a bad idea if you do now specify, what you're
going to pass through. Otherwise it's just fine and part of the interface.
We are passing through our job description to the adaptors as well...

> My suggestion:
> 
>   If there emerge any use cases which are not satisfied with
>   the job description we have in SAGA, then the job
>   description in SAGA needs to be fixed.  And yes, we should
>   make sure that this fix is translatable into JSDL, and
>   possibly others.  Until we have such use cases, stay
>   Simple! (i.e.  change nothing).

Why create two different standards out of the OGF for the same thing? Didn't
we integrate the GridRPC into SAGA almost unchanged as well - just
repackaged it into a fancier interface? 

In my opinion the SAGA job description can only provide a nice interface to
the main OGF job description format (which is as it turned out JSDL) as a
surplus. It should not replace it.

Just my 2c.
Regards Hartmut

> 
> Cheers, Andre.
> 
> 
> > Is this a route to go?
> > 
> > 
> > Thilo
> > 
> > On Tue, Feb 13, 2007 at 06:28:58PM +0100, 'Andre Merzky' wrote:
> > > From: 'Andre Merzky' <andre at merzky.net>
> > > To: SAGA RG <saga-rg at ogf.org>
> > > Subject: [SAGA-RG] Fwd (andre at merzky.net): Re: JSDL - SAGA
> > > 
> > > Hi group(s),
> > > 
> > > a couple of us had a recent discussion (f2f and email) about JSDL 
> > > and SAGA.  The question is: should we support JSDL fully on API 
> > > level?  E.g., should we allow the application programmer to 
> > > specify/use JSDL documents for job creation?
> > > 
> > > The reasons for doing that are compelling: JSDL is one of 
> the most 
> > > acknowledged standards in OGF, and the number of backends 
> supporting 
> > > JSDL is rapidly increasing it seems.
> > > Supporting JSDL directly would allow to interface with 
> other tools 
> > > using JSDL, and would allow to reuse JSDL documents where 
> these are 
> > > already available.
> > > 
> > > Well, I have however some problems with that approach, which are 
> > > outlined in the cited email below.
> > > 
> > > Do you guys have any other thoughts, and what solution would you 
> > > prefer?
> > > 
> > > Cheers, Andre.
> > > 
> > > 
> > > ----- Forwarded message from 'Andre Merzky' 
> <andre at merzky.net> -----
> > > 
> > > > From: 'Andre Merzky' <andre at merzky.net>
> > > > To: Shantenu Jha <sjha at cct.lsu.edu>
> > > > Cc: Hartmut Kaiser <hkaiser at cct.lsu.edu>,
> > > > 	'Thilo Kielmann' <kielmann at cs.vu.nl>,
> > > > 	'Andre Merzky' <andre at merzky.net>
> > > > Subject: Re: JSDL - SAGA
> > > > 
> > > > Hartmut and I discussed that somewhat last week.  So he 
> knows I am 
> > > > not wholehartedly for that option.  SAGA is supposed to 
> abstract 
> > > > the low level details, not to expose them.
> > > > 
> > > > JSDL is going to define a number of extensions now.  
> Some of these 
> > > > extensions are very useful for us, others not.
> > > > Mostly, they will be more complex than JSDL itself.
> > > > 
> > > > Are we going to support the extensions?  Which?  All/some?
> > > > How to select?  What error do we report on unsupported 
> extensions?  
> > > > Do we mandate that extensions are supported by the backends?  
> > > > Which?
> > > > 
> > > > Even w/o extensions: is the job description updated 
> after an JSDL 
> > > > attrib is set?  What about those attribs which are not 
> JSDL keys?  
> > > > Assume an implementation which implements the existing SAGA job 
> > > > description keys: MUST it support complete JSDL now?  
> What error 
> > > > whould it report?
> > > > 
> > > > These are probably all solvable problems, and I do agree that 
> > > > there are advantages, i.e. the re-use of existing JSDL 
> documents.  
> > > > Anyway, IMHO we should be careful, consider if we really have 
> > > > enough use cases etc.  Also, a free function 
> > > > jsdl_to_job_description may do the trick, w/o 
> complicating the job 
> > > > package itself.
> > > > 
> > > > Cheers, Andre.
> > > > 
> > > 
> > > Another point I'd like to raise is: if HPC profile 
> bekomes a widely 
> > > accepted OGF standard, do we support it directly, too?  Or 
> > > OGSA-Workflow?  Where to stop, and where is the 'S'
> > > in that approach?
> > > 
> > > Andre.
> > > 
> > > 
> > > > Quoting [Shantenu Jha]
> > > >> 
> > > >> What little I know, I think so to.
> > > >> 
> > > >>
> > > >> Quoting [Hartmut Kaiser]
> > > >>>
> > > >>> Agree 100%
> > > >>> The easiest way is probably just to add a attribute 
> in the job 
> > > >>> description taking the whole JSDL.
> > > >>>
> > > >>
> > > >> Quoting [Thilo Kielmann]
> > > >>>
> > > >>> Yes!
> > > >>>
> > > >>> 
> > > >>> Quoting [Shantenu Jha]
> > > >>>>
> > > >>>> Shouldn't we ensure that SAGA consumes JSDL w/o any 
> > > >>>> problem/changes?
> > > >>>>
> > > --
> > > "So much time, so little to do..."  -- Garfield
> > > --
> > >   saga-rg mailing list
> > >   saga-rg at ogf.org
> > >   http://www.ogf.org/mailman/listinfo/saga-rg
> --
> "So much time, so little to do..."  -- Garfield
> --
>   saga-rg mailing list
>   saga-rg at ogf.org
>   http://www.ogf.org/mailman/listinfo/saga-rg
> 