Subject: Re: [Fwd: [saga-rg] monitoring, again...]

[Raül Sirvent]

Andre Merzky wrote:
> Hi Raul, 
> 
> Quoting [Raul Sirvent] (Nov 07 2005):
> 
>>Hi. I would like to give our opinion (from GRID superscalar's point of 
>>view, as a use case of SAGA) about this monitoring API.
>>
>>The way it should be used is almost clear to me (it is very similar to 
>>the one defined in GAT). What I find missing is a way of waiting for 
>>callbacks to arrive, without having to wait for the task to finish. 
> 
> 
> you can do a
> 
>   while (1)
>   {
>     // callbacks will eventually arrive
>   }
> 
> or in fact anything else, as the callbacks are to be
> deliverd asynchroneously.  No need to care about if the task
> is really finished
> 
> 
> 
>>Of course I'm thinking in a Globus fashion (calls like
>>globus_poll and globus_poll_blocking).
> 
> 
> Ah, I am having a Deja Vu here :-)  I found in one of our
> earlier discussions:

You mean we are always talking about the same thing? That's not good... :)

> 
> 
> Quoting [Raul Sirvent] :
> 
>>We want to wake up every time a state change comes.  In your
>>case you are implementing a polling model but we are
>>interested in having a event-driven model. Something like:
>>
>>  submit_first_jobs ();
>>  
>>  while ( ! all_jobs_done )
>>  {
>>    wait_for_notification ();
>>
>>    // will submit new jobs if 
>>    // dependencies are solved
>>    treat_notification (); 
>>  }
>>
>>I don't see how to do that with the SAGA Task interface.
> 
> 
> I think, in SAGA that would look like:
> 
>     // run jobs, and add notification callbacks to all of
>     // them
>     submit_first_jobs ();
> 
>     while ( ! all_jobs_done )
>     {
>       // idle somewhat.  Maybe a callback gets called :-)
>       nanosleep (1);
> 
>       // A callback gets called in a different therad, so we
>       // don't see it here, unless it sets some flags.
>       // Well, assume that this it was the CB does, and that
>       // treat_notification () is checking these flags, and
>       // acts accordingly, e.g. submits new jobs
>       treat_notification ();
>     }
> 
> So the code looks actually exactly like yours, w/o an
> explicit wait/poll.  the CB is asynchroneous and in a
> different thread, and you can set a flag there, and poll for
> that, to notify your main thread.
> 
> Now, that description obviously hold only for
> implementations which implement CB really asynchroneously,
> meaning in threads.  If that is not the case, we would
> indeed need a explicit polling point, such as in GAT.
> 
> 
> 
> 
>>I remember something similar in GAT named.. 
>>ServiceActions? 
> 
> 
> exactly.  That is nothing else than the
> wait_for_notification, or the globus poll.
> 
> 
> 
>>Its purpose was to give the CPU to the GAT environment 
>>in order to do this kind of things (executing the functions to handle de 
>>callbacks, for instance). Is there something similar in SAGA?
> 
> 
> Its not needed, as the second thread used for the
> notification can get CPU as needed.
> 
> Hmm, I hope that makes sense :-)

Yes. It is clear now. You support a threaded model with asynchronous 
notifications, and you don't support a non-threaded model with 
synchronous notifications. I'm just a bit worried about the CPU wasted 
with active waits (to receive callbacks) in the main thread, or even 
having to insert sleeps (as you do) in the code. Do you think it is 
really so difficult to support both? (as Globus does, for instance).

Best regards,

Raul.

> 
> Cheers, Andre.
> 
> 
> 
>>Cheers. Raul.
>>