[saga-rg] Task model...
Andre Merzky
andre at merzky.net
Tue Nov 8 19:40:10 CST 2005
Hmm, that would make implementation actually very easy:
class saga::file
{
...
saga::task mkdir (string name)
{
saga::task t = impl->mkdir (name);
// if sync call (task runs and finishes)
if ( flag == saga::task::Sync )
{
t.run ();
t.wait ();
if ( t.status == saga::task::Failed )
{
t.throw ();
}
}
// if async call (task runs)
else if ( flag == saga::task::ASync )
{
t.run ( );
return (t);
}
// if task should not be run
else // if ( flag == saga::task::Task )
{
return (t);
}
return (t);
}
};
Nice. So, question is if that appeals to application
programmers as an API. Lets rename the versiosn we have for
easier discussion:
------------------------------------------------------
A: d.mkdir ("test/");
d.mkdir_sync ("test/"); // same
saga::task t_1 = d.mkdir_async ("test/");
saga::task t_2 = d.mkdir_task ("test/");
B: d.mkdir ("test/");
d.sync ::mkdir ("test/"); // same
saga::task t_1 = d.async::mkdir ("test/");
saga::task t_2 = d.task ::mkdir ("test/");
C: d.mkdir ("test/");
d.mkdir <sync> ("test/"); // same
saga::task t_1 = d.mkdir <async> ("test/");
saga::task t_2 = d.mkdir <task> ("test/");
D: d.mode ( Sync); d.mkdir ("test/");
d.mode ( Sync); saga::task t = d.mkdir ("test/"); // same
d.mode (ASync); saga::task t = d.mkdir ("test/");
d.mode ( Task); saga::task t = d.mkdir ("test/");
------------------------------------------------------
Ok, your model D seems to double the number of required
calls, but one should keep in mind that the flag needs to be
changed only if the model needs to be changed, not for each
call.
What do others think about this?
Two comments from my side:
- it would be possible to have that behavious very similar
in most langiage bindings
- as one would neet to keep track of the actual flag
setting before calling a method, lazy programmers would
probably set the flag either only once (forever), or
would set it before each call (to be safe). So it might
be considerd similar to a method flag, which was
discarded earlier as beeing too ugly... ;-)
Cheers, Andre.
Quoting [Hidemoto Nakada] (Nov 08 2005):
> Date: Wed, 09 Nov 2005 10:01:05 +0900
> From: Hidemoto Nakada <hide-nakada at aist.go.jp>
> To: Andre Merzky <andre at merzky.net>
> CC: Pascal Kleijer <k-pasukaru at ap.jp.nec.com>,
> pascal.kleijer at knd.biglobe.ne.jp, goodale at cct.lsu.edu,
> jshalf at lbl.gov, saga-rg at ggf.org, gridrpc-wg at gridforum.org
> Subject: Re: [saga-rg] Task model...
>
> Hi,
>
> Andre Merzky wrote:
> >Ah, right, sorry, I forgot about that model...
> >
> >There are two points to it, really:
> >
> > a) synchroneous calls alse return a (finished) task
> > b) a object can be either synchroneous or asynchroneous
> >
> >(a) is actually a more general point: it would be
> >interesting for the other models we listed earlier as well.
> >I kind of like it, as it makes implementation simplier, but
> >OTOH it makes error handlin somewhat more complex for the
> >synchroneous case, as you need to query the task state and
> >error condition explicitely.
> >
> >(b) was indeed discussed, but if my memory does not trick
> >me, it was discarded as it does not allow to
> >
> > - create a directory object
> > - do a synchroneous mkdir
> > - copy 100 files asynchroneously into that dir
>
> Why don't we simply have a 'sync' flag in the object, so that
> we can change the mode of a object dynamically.
>
> Directory d = new Directory("\");
> d.sync = false;
> Task t = d.mkdir("test\");
> t.wait();
>
> - hidemoto
>
>
> >
> >you cannot mix sync and async calls in that model. Is that
> >something you would think is no problem?
> >
> >We don't have very detailed use cases on asynchroneous
> >calls, apart from what we learned from the GridRPC folx. It
> >might be worth to write some down explicitely...
> >
> >Cheers, Andre.
> >
> >
> >Quoting [Pascal Kleijer] (Nov 07 2005):
> >
> >>Hello,
> >>
> >>Here is an example of what was discussed in Boston.
> >>
> >>// Type g1: No factory
> >>// The class has a flag in the constructor to tell if it work
> >>// synchronous or not.
> >>Directory d= new Directory("\", true);
> >>Task t= d.mkdir("test\");
> >>t.wait();
> >>
> >>//Type g2: with factory
> >>DirectoryFactory f= new SynchronousDirectoryFactory();
> >>Directory d= f.newInstance("\");
> >>Task t= d.mkdir("test");
> >>t.wait();
> >>
> >>Whatever the nature of the call, synchronous or not, a task is returned.
> >>A synchronous call will return a finished task. By providing a tight
> >>encapsulation it is possible to sub-class easily to add new features.
> >>For example you can implement a DebugDirectory class easily adding some
> >>extra debugging information without having to change the main code.
> >>
> >>My preference goes to the g1 case since it is the most direct and no
> >>factory is needed. Pattern design (g2) is nice, but I'm not sure if this
> >>is really necessary for this model.
> >>
> >>I know this solution is really OO designed, but that?s just because I
> >>always think in terms of objects.
> >>
> >>
> >>Best regards,
> >>Pascal Kleijer
> >>
> >>
> >>
> >>>Hi Pascal,
> >>>
> >>>there have been more models in discussion in Boston, true
> >>>enough. However, I can't really figure out what model you
> >>>refer to. Some example code would be helpful :-)
> >>>
> >>>Cheers, Andre.
> >>>
> >>>
> >>>Quoting [pascal.kleijer at knd.biglobe.ne.jp] (Nov 03 2005):
> >>>
> >>>>Hello all,
> >>>>
> >>>>Lets enter the arena if it is not too late, sorry for that.
> >>>>I do agree with many things that have been discussed here.
> >>>>Being myself a POP (pattern oriented programming) and OOP
> >>>>(Object Oriented programming) guy I tend to favor nicely
> >>>>encapsulated models.
> >>>>
> >>>>As mentioned by Tom, the template is possible in Java as
> >>>>well. But that forces the usage of Java 5.0 (aka 1.5) at
> >>>>least. This isn't recommended since many programs
> >>>>still run with 1.4. Using them would probably create a very
> >>>>different binding in Java, C++ or Python, which would end up
> >>>>in 2 different SAGA implementations/usage, one for OO
> >>>>languages and one for old fashioned procedurals.
> >>>>
> >>>>As to exposing the different implementation to the user I
> >>>>think this could be avoided in OO languages, or at least it
> >>>>should not be as obvious as in 4a2. During the GGF meeting
> >>>>their was a last minute discussion about a `g' solution
> >>>>which is pretty much pure OO design with a simple method
> >>>>call whatever model is used. This is I think a good choice
> >>>>for an OO point of view. You can derivate an abstract class
> >>>>to specialized classes with synchronous or asynchronous
> >>>>implementation. This would even open the room for sub
> >>>>classing it to add custom code for debugging for example or
> >>>>other treatment without affecting the initial code.
> >>>>Depending on how the programming is tackled factories aren't
> >>>>necessary. The problem might come when implementation is
> >>>>made in procedural languages, it might be necessary to add
> >>>>additional parameters to the constructor or the calls.
> >>>>
> >>>>I don't have the time this morning to make a sample
> >>>>code. But I hope you got the idea.
> >>>>
> >>>>Best Regards,
> >>>>Pascal Kleijer
> >>>>
> >>>>
> >>>>>My vote is for 4a1. (Which was option D in our original discussion I
> >>>>>believe.)
> >>>>>
> >>>>>Cheers,
> >>>>>
> >>>>>Tom
> >>>>>
> >>>>>On Thu, 3 Nov 2005, Andre Merzky wrote:
> >>>>>
> >>>>>
> >>>>>>Hi John, Thilo, Group,
> >>>>>>
> >>>>>>Hartmut and I would like to argue about the C++ bindings.
> >>>>>>We would like to come to an agreement soon, as our
> >>>>>>implementation is dealing with the async part right now.
> >>>>>>
> >>>>>>As reminder:
> >>>>>>
> >>>>>>---------------------------------------------------------------------
> >>>>>>Example 4a: more versions in C++
> >>>>>>
> >>>>>>4a1: d.mkdir ("test/");
> >>>>>> d.mkdir_sync ("test/"); // same
> >>>>>>saga::task t_1 = d.mkdir_async ("test/");
> >>>>>>saga::task t_2 = d.mkdir_task ("test/");
> >>>>>>
> >>>>>>4a2: d.mkdir ("test/");
> >>>>>> d.sync .mkdir ("test/"); // same
> >>>>>>saga::task t_1 = d.async.mkdir ("test/");
> >>>>>>saga::task t_2 = d.task .mkdir ("test/");
> >>>>>>
> >>>>>>4a3: d.mkdir ("test/");
> >>>>>> d.mkdir <sync> ("test/"); // same
> >>>>>>saga::task t_1 = d.mkdir <async> ("test/");
> >>>>>>saga::task t_2 = d.mkdir <task> ("test/");
> >>>>>>---------------------------------------------------------------------
> >>>>>>
> >>>>>>
> >>>>>>Quoting [John Shalf] (Nov 02 2005):
> >>>>>>
> >>>>>>>>Q5) Any comments to 4a1, 4a2 or 4a3? (not part of the Strawman!)
> >>>>>>>
> >>>>>>>I prefer 4a1 because it is more readable and the implementation would
> >>>>>>>be quite straightforward.
> >>>>>>
> >>>>>>You favour 4a1. We think that implementation is very straight
> >>>>>>forward for all three versions. Readability seems not much
> >>>>>>different to me, and might largely be a matter of taste.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>It is also a familiar paradigm for any MPI
> >>>>>>>programmers and anyone who has played with various proprietary Async
> >>>>>>>I/O implementations. (its a very familiar and conventional approach)
> >>>>>>
> >>>>>>Well, these are C-API's. Is there a C++ binding for MPI,
> >>>>>>and does it look the same?
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>I kind of like 4a2 as well from the standpoint of a C++ programmer
> >>>>>>>(even with Andre's syntax corrections). However, the resulting
> >>>>>>>bindings will not be very consistent with the approach we would take
> >>>>>>>for Fortran or C bindings (eg. those would likely look more like
> >>>>>>>4a1).
> >>>>>>
> >>>>>>But, well, that is the idea of the language binding! I agree
> >>>>>>that C and Fortran would look more like 4a1, of course. But that
> >>>>>>is no reason that C++ should look like that as well, or Java,
> >>>>>>Perl etc.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>It is not really much more readable than 4a1. Therefore, I'm
> >>>>>>>uncertain if it is worth fragmenting our approach to bindings in
> >>>>>>>different languages when there is not a clear benefit in terms of
> >>>>>>>readability or implementation complexity.
> >>>>>>
> >>>>>>I think that 4a2/4a3 actually allow nicer implementations, as it
> >>>>>>allows to have the different async parts somewhat separate from
> >>>>>>the sync parts. We think its nicer :-)
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>I do a lot of C++ programming, but I find the 4a3 option a bit
> >>>>>>>obscure both in terms of readability and any advantages it might
> >>>>>>>confer in terms of implementation.
> >>>>>>
> >>>>>>Hehe - I thought the same :-) Hartmut likes that version very
> >>>>>>much. To me it appealed after pondering over it for a couple of
> >>>>>>days. Now I think it is cute, and quite expressive.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>It would certainly be easier to
> >>>>>>>create multi-language bindings for a single code base if we stick
> >>>>>>>with something more conventional like 4a1.
> >>>>>>
> >>>>>>I think that C and Fortran bindings for this part are straight
> >>>>>>forward anyway, there is no need to reflect that in C++...
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>Each approach is equally readable to me (less so for 4a3). I'm
> >>>>>>>certainly open to any additional information on how the 4a2 and 4a3
> >>>>>>>approaches could simplify implementation.
> >>>>>>
> >>>>>>The main point really is that the object itself has only the sync
> >>>>>>method calls, and the async calls can be put into separate
> >>>>>>header/sources easily, and build (by default) on top of the sync
> >>>>>>calls. Of course, you can do that with all three versions, but
> >>>>>>its not as obvious in 4a1.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>If the other approaches
> >>>>>>>can offer some implementation benefits, then maybe I'd give them
> >>>>>>>extra consideration, but otherwise, I would prefer a more
> >>>>>>>conventional approach like 4a1.
> >>>>>>
> >>>>>>I would vote for 4a2 or 4a3 (in that order), but 4a1 would be ok
> >>>>>>if the majority likes that most, of course. Basically its
> >>>>>>a matter of taste I think. I am happy that the general point
> >>>>>>seems accaptable to all so far: having sync, async, and task
> >>>>>>versions of the calls, w/o explicit task factory.
> >>>>>>
> >>>>>>
> >>>>>>>The only implementation I'm outright
> >>>>>>>against is the 4b example.
> >>>>>>
> >>>>>>Good! :-)
> >>>>>>
> >>>>>>Thanks, Andre.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>-john
> >>>>>>
> >>>>>>
> >>>>>>
> >>>
> >>>
> >
> >
> >
--
+-----------------------------------------------------------------+
| Andre Merzky | phon: +31 - 20 - 598 - 7759 |
| Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 |
| Dept. of Computer Science | mail: merzky at cs.vu.nl |
| De Boelelaan 1083a | www: http://www.merzky.net |
| 1081 HV Amsterdam, Netherlands | |
+-----------------------------------------------------------------+
More information about the saga-rg
mailing list