[saga-rg] Task model...
Hartmut Kaiser
hkaiser at cct.lsu.edu
Tue Nov 8 19:49:50 CST 2005
Andre Merzky wrote:
> Hmm, that would make implementation actually very easy:
>
> class saga::file
> {
> ...
> saga::task mkdir (string name)
> {
> saga::task t = impl->mkdir (name);
>
> // if sync call (task runs and finishes)
> if ( flag == saga::task::Sync )
> {
> t.run ();
> t.wait ();
>
> if ( t.status == saga::task::Failed )
> {
> t.throw ();
> }
>
> }
>
> // if async call (task runs)
> else if ( flag == saga::task::ASync )
> {
> t.run ( );
> return (t);
> }
>
> // if task should not be run
> else // if ( flag == saga::task::Task )
> {
> return (t);
> }
>
> return (t);
> }
> };
>
>
> Nice. So, question is if that appeals to application
> programmers as an API. Lets rename the versiosn we have for
> easier discussion:
>
> ------------------------------------------------------
> A: d.mkdir ("test/");
> d.mkdir_sync ("test/"); // same
> saga::task t_1 = d.mkdir_async ("test/");
> saga::task t_2 = d.mkdir_task ("test/");
>
> B: d.mkdir ("test/");
> d.sync ::mkdir ("test/"); // same
> saga::task t_1 = d.async::mkdir ("test/");
> saga::task t_2 = d.task ::mkdir ("test/");
>
> C: d.mkdir ("test/");
> d.mkdir <sync> ("test/"); // same
> saga::task t_1 = d.mkdir <async> ("test/");
> saga::task t_2 = d.mkdir <task> ("test/");
>
> D: d.mode ( Sync); d.mkdir ("test/");
> d.mode ( Sync); saga::task t = d.mkdir ("test/"); // same
> d.mode (ASync); saga::task t = d.mkdir ("test/");
> d.mode ( Task); saga::task t = d.mkdir ("test/");
> ------------------------------------------------------
>
> Ok, your model D seems to double the number of required
> calls, but one should keep in mind that the flag needs to be
> changed only if the model needs to be changed, not for each call.
>
> What do others think about this?
>
> Two comments from my side:
>
> - it would be possible to have that behavious very similar
> in most langiage bindings
>
> - as one would neet to keep track of the actual flag
> setting before calling a method, lazy programmers would
> probably set the flag either only once (forever), or
> would set it before each call (to be safe). So it might
> be considerd similar to a method flag, which was
> discarded earlier as beeing too ugly... ;-)
The initial flag could be passed as an additional constructor parameter
then.
The main question is, do we need to have different return values from
sync/async operations or not. It's kind of unusual to get a task from a
'normal' sync operation, even more because this task is usable for return
value retrieval only.
Regards Hartmut
>
> Cheers, Andre.
>
>
> Quoting [Hidemoto Nakada] (Nov 08 2005):
> > Date: Wed, 09 Nov 2005 10:01:05 +0900
> > From: Hidemoto Nakada <hide-nakada at aist.go.jp>
> > To: Andre Merzky <andre at merzky.net>
> > CC: Pascal Kleijer <k-pasukaru at ap.jp.nec.com>,
> > pascal.kleijer at knd.biglobe.ne.jp, goodale at cct.lsu.edu,
> > jshalf at lbl.gov, saga-rg at ggf.org, gridrpc-wg at gridforum.org
> > Subject: Re: [saga-rg] Task model...
> >
> > Hi,
> >
> > Andre Merzky wrote:
> > >Ah, right, sorry, I forgot about that model...
> > >
> > >There are two points to it, really:
> > >
> > > a) synchroneous calls alse return a (finished) task
> > > b) a object can be either synchroneous or asynchroneous
> > >
> > >(a) is actually a more general point: it would be
> interesting for the
> > >other models we listed earlier as well.
> > >I kind of like it, as it makes implementation simplier,
> but OTOH it
> > >makes error handlin somewhat more complex for the
> synchroneous case,
> > >as you need to query the task state and error condition
> explicitely.
> > >
> > >(b) was indeed discussed, but if my memory does not trick
> me, it was
> > >discarded as it does not allow to
> > >
> > > - create a directory object
> > > - do a synchroneous mkdir
> > > - copy 100 files asynchroneously into that dir
> >
> > Why don't we simply have a 'sync' flag in the object, so
> that we can
> > change the mode of a object dynamically.
> >
> > Directory d = new Directory("\");
> > d.sync = false;
> > Task t = d.mkdir("test\");
> > t.wait();
> >
> > - hidemoto
> >
> >
> > >
> > >you cannot mix sync and async calls in that model. Is
> that something
> > >you would think is no problem?
> > >
> > >We don't have very detailed use cases on asynchroneous
> calls, apart
> > >from what we learned from the GridRPC folx. It might be worth to
> > >write some down explicitely...
> > >
> > >Cheers, Andre.
> > >
> > >
> > >Quoting [Pascal Kleijer] (Nov 07 2005):
> > >
> > >>Hello,
> > >>
> > >>Here is an example of what was discussed in Boston.
> > >>
> > >>// Type g1: No factory
> > >>// The class has a flag in the constructor to tell if it work //
> > >>synchronous or not.
> > >>Directory d= new Directory("\", true);
> > >>Task t= d.mkdir("test\");
> > >>t.wait();
> > >>
> > >>//Type g2: with factory
> > >>DirectoryFactory f= new SynchronousDirectoryFactory();
> > >>Directory d= f.newInstance("\");
> > >>Task t= d.mkdir("test");
> > >>t.wait();
> > >>
> > >>Whatever the nature of the call, synchronous or not, a
> task is returned.
> > >>A synchronous call will return a finished task. By
> providing a tight
> > >>encapsulation it is possible to sub-class easily to add
> new features.
> > >>For example you can implement a DebugDirectory class
> easily adding
> > >>some extra debugging information without having to change
> the main code.
> > >>
> > >>My preference goes to the g1 case since it is the most
> direct and no
> > >>factory is needed. Pattern design (g2) is nice, but I'm
> not sure if
> > >>this is really necessary for this model.
> > >>
> > >>I know this solution is really OO designed, but that?s
> just because
> > >>I always think in terms of objects.
> > >>
> > >>
> > >>Best regards,
> > >>Pascal Kleijer
> > >>
> > >>
> > >>
> > >>>Hi Pascal,
> > >>>
> > >>>there have been more models in discussion in Boston,
> true enough.
> > >>>However, I can't really figure out what model you refer
> to. Some
> > >>>example code would be helpful :-)
> > >>>
> > >>>Cheers, Andre.
> > >>>
> > >>>
> > >>>Quoting [pascal.kleijer at knd.biglobe.ne.jp] (Nov 03 2005):
> > >>>
> > >>>>Hello all,
> > >>>>
> > >>>>Lets enter the arena if it is not too late, sorry for that.
> > >>>>I do agree with many things that have been discussed here.
> > >>>>Being myself a POP (pattern oriented programming) and
> OOP (Object
> > >>>>Oriented programming) guy I tend to favor nicely encapsulated
> > >>>>models.
> > >>>>
> > >>>>As mentioned by Tom, the template is possible in Java
> as well. But
> > >>>>that forces the usage of Java 5.0 (aka 1.5) at least.
> This isn't
> > >>>>recommended since many programs still run with 1.4. Using them
> > >>>>would probably create a very different binding in Java, C++ or
> > >>>>Python, which would end up in 2 different SAGA
> > >>>>implementations/usage, one for OO languages and one for old
> > >>>>fashioned procedurals.
> > >>>>
> > >>>>As to exposing the different implementation to the user I think
> > >>>>this could be avoided in OO languages, or at least it
> should not
> > >>>>be as obvious as in 4a2. During the GGF meeting their
> was a last
> > >>>>minute discussion about a `g' solution which is pretty
> much pure
> > >>>>OO design with a simple method call whatever model is
> used. This
> > >>>>is I think a good choice for an OO point of view. You
> can derivate
> > >>>>an abstract class to specialized classes with synchronous or
> > >>>>asynchronous implementation. This would even open the
> room for sub
> > >>>>classing it to add custom code for debugging for
> example or other
> > >>>>treatment without affecting the initial code.
> > >>>>Depending on how the programming is tackled factories aren't
> > >>>>necessary. The problem might come when implementation
> is made in
> > >>>>procedural languages, it might be necessary to add additional
> > >>>>parameters to the constructor or the calls.
> > >>>>
> > >>>>I don't have the time this morning to make a sample code. But I
> > >>>>hope you got the idea.
> > >>>>
> > >>>>Best Regards,
> > >>>>Pascal Kleijer
> > >>>>
> > >>>>
> > >>>>>My vote is for 4a1. (Which was option D in our original
> > >>>>>discussion I
> > >>>>>believe.)
> > >>>>>
> > >>>>>Cheers,
> > >>>>>
> > >>>>>Tom
> > >>>>>
> > >>>>>On Thu, 3 Nov 2005, Andre Merzky wrote:
> > >>>>>
> > >>>>>
> > >>>>>>Hi John, Thilo, Group,
> > >>>>>>
> > >>>>>>Hartmut and I would like to argue about the C++ bindings.
> > >>>>>>We would like to come to an agreement soon, as our
> > >>>>>>implementation is dealing with the async part right now.
> > >>>>>>
> > >>>>>>As reminder:
> > >>>>>>
> >
> >>>>>>----------------------------------------------------------------
> > >>>>>>----- Example 4a: more versions in C++
> > >>>>>>
> > >>>>>>4a1: d.mkdir ("test/");
> > >>>>>> d.mkdir_sync ("test/"); // same
> > >>>>>>saga::task t_1 = d.mkdir_async ("test/");
> > >>>>>>saga::task t_2 = d.mkdir_task ("test/");
> > >>>>>>
> > >>>>>>4a2: d.mkdir ("test/");
> > >>>>>> d.sync .mkdir ("test/"); // same
> > >>>>>>saga::task t_1 = d.async.mkdir ("test/");
> > >>>>>>saga::task t_2 = d.task .mkdir ("test/");
> > >>>>>>
> > >>>>>>4a3: d.mkdir ("test/");
> > >>>>>> d.mkdir <sync> ("test/"); // same saga::task
> > >>>>>>t_1 = d.mkdir <async> ("test/"); saga::task t_2 =
> d.mkdir <task>
> > >>>>>>("test/");
> >
> >>>>>>----------------------------------------------------------------
> > >>>>>>-----
> > >>>>>>
> > >>>>>>
> > >>>>>>Quoting [John Shalf] (Nov 02 2005):
> > >>>>>>
> > >>>>>>>>Q5) Any comments to 4a1, 4a2 or 4a3? (not part of the
> > >>>>>>>>Strawman!)
> > >>>>>>>
> > >>>>>>>I prefer 4a1 because it is more readable and the
> implementation
> > >>>>>>>would be quite straightforward.
> > >>>>>>
> > >>>>>>You favour 4a1. We think that implementation is very
> straight
> > >>>>>>forward for all three versions. Readability seems not much
> > >>>>>>different to me, and might largely be a matter of taste.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>It is also a familiar paradigm for any MPI programmers and
> > >>>>>>>anyone who has played with various proprietary Async I/O
> > >>>>>>>implementations. (its a very familiar and conventional
> > >>>>>>>approach)
> > >>>>>>
> > >>>>>>Well, these are C-API's. Is there a C++ binding for MPI, and
> > >>>>>>does it look the same?
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>I kind of like 4a2 as well from the standpoint of a C++
> > >>>>>>>programmer (even with Andre's syntax corrections). However,
> > >>>>>>>the resulting bindings will not be very consistent with the
> > >>>>>>>approach we would take for Fortran or C bindings (eg. those
> > >>>>>>>would likely look more like 4a1).
> > >>>>>>
> > >>>>>>But, well, that is the idea of the language binding! I agree
> > >>>>>>that C and Fortran would look more like 4a1, of course. But
> > >>>>>>that is no reason that C++ should look like that as well, or
> > >>>>>>Java, Perl etc.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>It is not really much more readable than 4a1.
> Therefore, I'm
> > >>>>>>>uncertain if it is worth fragmenting our approach to
> bindings
> > >>>>>>>in different languages when there is not a clear benefit in
> > >>>>>>>terms of readability or implementation complexity.
> > >>>>>>
> > >>>>>>I think that 4a2/4a3 actually allow nicer
> implementations, as it
> > >>>>>>allows to have the different async parts somewhat
> separate from
> > >>>>>>the sync parts. We think its nicer :-)
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>I do a lot of C++ programming, but I find the 4a3
> option a bit
> > >>>>>>>obscure both in terms of readability and any advantages it
> > >>>>>>>might confer in terms of implementation.
> > >>>>>>
> > >>>>>>Hehe - I thought the same :-) Hartmut likes that
> version very
> > >>>>>>much. To me it appealed after pondering over it for
> a couple of
> > >>>>>>days. Now I think it is cute, and quite expressive.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>It would certainly be easier to create
> multi-language bindings
> > >>>>>>>for a single code base if we stick with something more
> > >>>>>>>conventional like 4a1.
> > >>>>>>
> > >>>>>>I think that C and Fortran bindings for this part are
> straight
> > >>>>>>forward anyway, there is no need to reflect that in C++...
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>Each approach is equally readable to me (less so for
> 4a3). I'm
> > >>>>>>>certainly open to any additional information on how
> the 4a2 and
> > >>>>>>>4a3 approaches could simplify implementation.
> > >>>>>>
> > >>>>>>The main point really is that the object itself has only the
> > >>>>>>sync method calls, and the async calls can be put
> into separate
> > >>>>>>header/sources easily, and build (by default) on top
> of the sync
> > >>>>>>calls. Of course, you can do that with all three
> versions, but
> > >>>>>>its not as obvious in 4a1.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>If the other approaches
> > >>>>>>>can offer some implementation benefits, then maybe I'd give
> > >>>>>>>them extra consideration, but otherwise, I would
> prefer a more
> > >>>>>>>conventional approach like 4a1.
> > >>>>>>
> > >>>>>>I would vote for 4a2 or 4a3 (in that order), but 4a1
> would be
> > >>>>>>ok if the majority likes that most, of course.
> Basically its a
> > >>>>>>matter of taste I think. I am happy that the general point
> > >>>>>>seems accaptable to all so far: having sync, async, and task
> > >>>>>>versions of the calls, w/o explicit task factory.
> > >>>>>>
> > >>>>>>
> > >>>>>>>The only implementation I'm outright against is the
> 4b example.
> > >>>>>>
> > >>>>>>Good! :-)
> > >>>>>>
> > >>>>>>Thanks, Andre.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>-john
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>
> > >>>
> > >
> > >
> > >
>
>
>
> --
> +-----------------------------------------------------------------+
> | Andre Merzky | phon: +31 - 20 - 598 - 7759 |
> | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 |
> | Dept. of Computer Science | mail: merzky at cs.vu.nl |
> | De Boelelaan 1083a | www: http://www.merzky.net |
> | 1081 HV Amsterdam, Netherlands | |
> +-----------------------------------------------------------------+
>
More information about the saga-rg
mailing list