[SAGA-RG] Fwd (andre at merzky.net): Re: JSDL - SAGA
Hartmut Kaiser
hkaiser at cct.lsu.edu
Mon Feb 19 14:50:08 CST 2007
Hi all,
After thinking about this issue a bit more I would like to make a even more
'radical' suggestion.
Let's recap:
- JSDL is an OGF Standard now and there is no way of ignoring it in SAGA
- the current SAGA JD is more constrained in terms of scope if compared to
JSDL
- there will be a lot of middleware supporting JSDL in the future
- there wll be a lot of tooling support to create JSDL in the future
A viable solution to this in SAGA could be (duck):
- redo the SAGA job_service and job interfaces so that these work solely
based on JSDL strings
- supply a SAGA JD component providing a _simple_ interface for JSDL
construction:
job_description::construct(string jsdl);
job_description::as_jsdl();
job_service::create_job(string jsdl);
job::migrate(..., string jsdl, ...);
where we clearly could stick to JSDL V1 for this conversion and the allowed
attribute set on our job descriptions.
Regards Hartmut
> -----Original Message-----
> From: saga-rg-bounces at ogf.org
> [mailto:saga-rg-bounces at ogf.org] On Behalf Of Pascal Kleijer
> Sent: Sunday, February 18, 2007 8:51 PM
> To: SAGA RG
> Subject: Re: [SAGA-RG] Fwd (andre at merzky.net): Re: JSDL - SAGA
>
> Dear all,
>
> I have given me the week-end to get some reflection on the
> problem. As mentioned it is not a trivial issue. If things
> goes out of hand, we might as well rename the project CAGA of
> not go gaga :P
>
> The necessity to support JSDL 1.0.x is a necessity, since it
> is an OFG standard. I do not think JSDL 1.0 is overly complex
> right now. The Job Description should be made such that
> passing from one form to another is automatic (JD -> JSDL).
> The support of a property key for JSDL isn't bad; if used,
> all other properties can be potentially ignored. The JD would
> not have to parse it. The current specification is simple
> enough to not violate the "S" rule of SAGA.
>
> The problem we have is with all the up coming extensions. In
> my case, I clearly violated the SAGA API rules in order to
> make the system run with NAREGI. I tweaked the problem to
> solve my case. In the 80/20 rule, it probably means that the
> SAGA job submission won't be usable. I will have to write
> code that does not deal with SAGA and goes directly to the
> underwear. Now I am speaking as a Grid "expert" and not as a
> Grid newbie (which is the SAGA user base). If you want to use
> complex Grid programming and do complex things, SAGA might
> not be the right solution.
>
> The proposal made by Dan is not elegant in my opinion because
> it will force developer to know the underlying technology
> they want to use; it will also clutter the JD. The proposal
> from Thilo might work for OO languages because you can hide a
> lot behind patterns, but it won't mean a specific
> implementation will support such scheme.
>
> The passing through solution seems to be mildly seen by the
> some GAT people here. I would say no, if the error handling
> is explicit enough to return a message telling that this JSDL
> cannot be interpreted by the underlying system. I will not
> see this as a bad solution either.
>
> --
>
> Best regards,
> Pascal Kleijer
>
> ----------------------------------------------------------------
> HPC Marketing Promotion Division, NEC Corporation
> 1-10, Nisshin-cho, Fuchu, Tokyo, 183-8501, Japan.
> Tel: +81-(0)42/333.6389 Fax: +81-(0)42/333.6382
>
>
> Daniel S. Katz wrote:
> > Another related idea is for SAGA to have one class for job
> > descriptions with a quite loose ability to add "hints",
> such as in MPI
> > I/O. The hints would not be required and might not be used, but in
> > some cases would be used, depending on what SAGA is sitting
> on top of.
> >
> > Dan
> >
> > Thilo Kielmann wrote (on 2/16/07 5:54 AM):
> >> Folks,
> >>
> >> I think we are having conflicting goals here.
> >> (Technical goals, not personal ones ;-)
> >>
> >> On one hand, we have the "S for simplicity" in SAGA, and
> we must keep it.
> >>
> >> On the other hand, we have the necessity to support JSDL,
> future JSDL
> >> extensions, or any other types of job decscriptions that
> people want to use.
> >> (And still might to be invented.)
> >>
> >> Assume, JSDL++ (whatever it will look like in near future) will
> >> become a widely adopted standard. (Or anything else,
> doesn't matter
> >> in the following.) Then, SAGA implementations will have to use
> >> JSDL++, and to form JSDL++ from SAGA job descriptions.
> >> Simultaneously, users are likely to use JSDL++ themselves,
> and may wnat to use JSDL++ to express their resource needs.
> >> At this point in time, SAGA will sit in the middle, and it may be
> >> very clumsy to first translate from JSDL++ to a SAGA job
> description,
> >> and then back to
> >> JSDL++ somewhere "down under" in the implementation.
> >>
> >> For the very purpose of SAGA as a universal and simple
> grid API, it has to:
> >> - be independent of job description standards (mostly simpler than
> >> them)
> >> - support job description standards
> >>
> >>
> >> My suggestion:
> >>
> >> SAGA should have one class of job descriptions, and the
> possibility
> >> to create subclasses for more specific job descriptions
> (like JSDL).
> >> Such subclasses could be defined as separate extension documents
> >> (just like gridcpr). (Or was it "resource descriptions"???)
> >>
> >> With this approach, users could still write programs that are
> >> independent of the underlying job submission machinery, having a
> >> simplified view on jobs and resource attributes etc. etc.
> >> At the same time, subclassed job/resource descriptions could be
> >> "passed through" transparently from the API to the
> implementeation,
> >> without being converted back and forth, a process that is
> very likely
> >> to loose some important details.
> >>
> >>
> >> Is this a route to go?
> >>
> >>
> >> Thilo
> >>
> >> On Tue, Feb 13, 2007 at 06:28:58PM +0100, 'Andre Merzky' wrote:
> >>
> >>> From: 'Andre Merzky' <andre at merzky.net> <mailto:andre at merzky.net>
> >>> To: SAGA RG <saga-rg at ogf.org> <mailto:saga-rg at ogf.org>
> >>> Subject: [SAGA-RG] Fwd (andre at merzky.net
> <mailto:andre at merzky.net>):
> >>> Re: JSDL - SAGA
> >>>
> >>> Hi group(s),
> >>>
> >>> a couple of us had a recent discussion (f2f and email) about JSDL
> >>> and SAGA. The question is: should we support JSDL fully on API
> >>> level? E.g., should we allow the application programmer to
> >>> specify/use JSDL documents for job creation?
> >>>
> >>> The reasons for doing that are compelling: JSDL is one of
> the most
> >>> acknowledged standards in OGF, and the number of backends
> supporting
> >>> JSDL is rapidly increasing it seems.
> >>> Supporting JSDL directly would allow to interface with
> other tools
> >>> using JSDL, and would allow to reuse JSDL documents where
> these are
> >>> already available.
> >>>
> >>> Well, I have however some problems with that approach, which are
> >>> outlined in the cited email below.
> >>>
> >>> Do you guys have any other thoughts, and what solution would you
> >>> prefer?
> >>>
> >>> Cheers, Andre.
> >>>
> >>>
> >>> ----- Forwarded message from 'Andre Merzky' <andre at merzky.net>
> >>> <mailto:andre at merzky.net> -----
> >>>
> >>>
> >>>> From: 'Andre Merzky' <andre at merzky.net> <mailto:andre at merzky.net>
> >>>> To: Shantenu Jha <sjha at cct.lsu.edu> <mailto:sjha at cct.lsu.edu>
> >>>> Cc: Hartmut Kaiser <hkaiser at cct.lsu.edu>
> <mailto:hkaiser at cct.lsu.edu>,
> >>>> 'Thilo Kielmann' <kielmann at cs.vu.nl> <mailto:kielmann at cs.vu.nl>,
> >>>> 'Andre Merzky' <andre at merzky.net> <mailto:andre at merzky.net>
> >>>> Subject: Re: JSDL - SAGA
> >>>>
> >>>> Hartmut and I discussed that somewhat last week. So he
> knows I am
> >>>> not wholehartedly for that option. SAGA is supposed to abstract
> >>>> the low level details, not to expose them.
> >>>>
> >>>> JSDL is going to define a number of extensions now.
> Some of these
> >>>> extensions are very useful for us, others not.
> >>>> Mostly, they will be more complex than JSDL itself.
> >>>>
> >>>> Are we going to support the extensions? Which? All/some?
> >>>> How to select? What error do we report on unsupported
> extensions?
> >>>> Do we mandate that extensions are supported by the backends?
> >>>> Which?
> >>>>
> >>>> Even w/o extensions: is the job description updated
> after an JSDL
> >>>> attrib is set? What about those attribs which are not
> JSDL keys?
> >>>> Assume an implementation which implements the existing SAGA job
> >>>> description keys: MUST it support complete JSDL now? What error
> >>>> whould it report?
> >>>>
> >>>> These are probably all solvable problems, and I do agree
> that there
> >>>> are advantages, i.e. the re-use of existing JSDL documents.
> >>>> Anyway, IMHO we should be careful, consider if we really have
> >>>> enough use cases etc. Also, a free function
> >>>> jsdl_to_job_description may do the trick, w/o
> complicating the job
> >>>> package itself.
> >>>>
> >>>> Cheers, Andre.
> >>>>
> >>>>
> >>> Another point I'd like to raise is: if HPC profile
> bekomes a widely
> >>> accepted OGF standard, do we support it directly, too? Or
> >>> OGSA-Workflow? Where to stop, and where is the 'S'
> >>> in that approach?
> >>>
> >>> Andre.
> >>>
> >>>
> >>>
> >>>> Quoting [Shantenu Jha]
> >>>>
> >>>>> What little I know, I think so to.
> >>>>>
> >>>>>
> >>>>> Quoting [Hartmut Kaiser]
> >>>>>
> >>>>>> Agree 100%
> >>>>>> The easiest way is probably just to add a attribute in the job
> >>>>>> description taking the whole JSDL.
> >>>>>>
> >>>>>>
> >>>>> Quoting [Thilo Kielmann]
> >>>>>
> >>>>>> Yes!
> >>>>>>
> >>>>>>
> >>>>>> Quoting [Shantenu Jha]
> >>>>>>
> >>>>>>> Shouldn't we ensure that SAGA consumes JSDL w/o any
> >>>>>>> problem/changes?
> >>>>>>>
> >>>>>>>
> >>> --
> >>> "So much time, so little to do..." -- Garfield
> >>> --
> >>> saga-rg mailing list
> >>> saga-rg at ogf.org <mailto:saga-rg at ogf.org>
> >>> http://www.ogf.org/mailman/listinfo/saga-rg
> >>>
> >>
> >>
> >>
> >
> >
> ----------------------------------------------------------------------
> > --
> >
> > --
> > saga-rg mailing list
> > saga-rg at ogf.org
> > http://www.ogf.org/mailman/listinfo/saga-rg
>
>
More information about the saga-rg
mailing list