[saga-rg] SAGA spec proof read

[Andre Merzky]

Hi Pascal, 

thanks for the agreement!  I hope my comments have not been
too confusing... :-)

We should keep all the points you raised in mind, and cross
check if the message API indeed solves these issues.

Cheers, Andre.


Quoting [Pascal Kleijer] (Jun 09 2006):
> Date: Fri, 09 Jun 2006 17:35:11 +0900
> From: Pascal Kleijer <k-pasukaru at ap.jp.nec.com>
> To: Andre Merzky <andre at merzky.net>
> CC: saga-rg at ggf.org
> Subject: Re: [saga-rg] SAGA spec proof read
> 
> Hi Andre,
> 
> I have taken some time to digest your comments and assess the situation. 
> Your arguments are reasonable and accurate and see where you want to go. 
> I would then agree on the current proposal and defer these issues to the 
> messaging API. The messaging API should ensure the requirements that I 
> could imagine.
> 
> >Hi Pascal, 
> >
> >comments to the comments... :-)
> >
> >(removed some topics, agreed with them, and will change
> >accordingly)
> >
> >
> >Quoting [Pascal Kleijer] (Jun 09 2006):
> >>Hi,
> >>
> >>some in-line comments...
> >>
> >>
> >>>>Some extensions:
> >>>>Purpose: ?Reads up to ?size? bytes of data from the stream into an 
> >>>>array of bytes. An attempt is made to read as many as ?size? bytes, but 
> >>>>a smaller number but may be read. The number of bytes actually read is 
> >>>>returned as ?nbytes?. If ?size? is zero, then no bytes are read and 
> >>>>nbytes is 0; otherwise, there is an attempt to read at least one byte.?
> >>>That should already be the case - that is why 'nbytes' is
> >>>returned, indicating the actual number of bytes copied.
> >>>nbytes can be smaller than size w/o posing an error
> >>>condition (as in POSIX).  Is that what you propose?
> >>Well I think the whole description should be redone based on the 
> >>blocking or not behavior. Also the 'nbytes' returned should add, like in 
> >>POSIX, the error code (negative). IF a new read is made after an fatal 
> >>error, then the exception should kick in. See below.
> >
> >See below.
> >
> >>>>3. In this implementation we have no choice but to wait for an 
> >>>>exception to be thrown when the stream has dried up (use the read 
> >>>>method). It would be nice to have -1 set to nbytes to first time the 
> >>>>end of stream is reached (if applies), then the next call to read will 
> >>>>throw an exception.
> >>>What is the error condition the exception is supposed to
> >>>catch:
> >>>
> >>> - stream died
> >>> - no data available
> >>>
> >>>The first one is available via the state of the stream.
> >>>
> >>>The second one is what you refer to, right?  But I think
> >>>that this is not an error condition which should raise an
> >>>exception: data might be available again a second later.
> >>>
> >>>If data availability poses an error condition or not should
> >>>be decided on application level IMHO.  
> >>>
> >>>Am I missing something?
> >>Basically the exception should be thrown if the last read did return 
> >>with an error. If the error was rectified in the mean time, the read 
> >>should go one as a new call. Therefore the state machine is important to 
> >>have, because a change of state will flush the last error.
> >
> >I am not sure if I would agree with this one.
> >
> >Assume the following pseudo code:
> >
> >
> > char chache[10] = "";
> > int  nbytes;
> >
> > while ( nbytes = socket.read (cache, 10))
> > {
> >   if ( 0 > nbytes )
> >   { 
> >     print (cache);
> >   }
> >   else
> >   { 
> >     sleep (1);
> >   }  
> > }
> >
> >
> >Adding an exception on the second failed read breaks that
> >schema, and makes code more complex.  Why?  Because the
> >application has to explicitely keep track of the stream
> >state.
> >
> >However, POSIX and most other stream APIs I can think of
> >would fully support the schema above.
> >
> >Also, what is the application supposed to do after the first
> >non-read?  It is bound to eventually try again, and raise an
> >exception.
> >
> >However, that is related to the next point you raise...
> >
> >(PS.: new data would not change the state, really, in the
> >sense of state diagram...)
> >
> >
> >>>>4. There is no way to check the size of the data waiting for read. It 
> >>>>would be nice if we can query the stream on how many bytes in the pipe 
> >>>>are waiting. This is important if the client must allocate a buffer to 
> >>>>read this data. So an additional method like ?available? should be 
> >>>>added. Note that this will slightly overlap with the ?wait? method for 
> >>>>?read? or ?any? mask, but will return a more accurate figure. Another 
> >>>>solution is to extend the wait method.
> >>>I would have no idea how to implement that on a stream:
> >>>without an additional protocol, you have no chance to know
> >>>how many bytes will arrive.
> >>>
> >>>However, that is one of the reasons why the message API was
> >>>proposed: that implies an additional protocol on the wire,
> >>>but will allow the application to query the message size
> >>>before performing the message read.
> >>I don't say how many will arrive, but how many 'has' arrived. That mean 
> >>how many are currently stored somewhere within the stream and that are 
> >>ensured to be returned when the next call the read is done. In that case 
> >>we can test for a possible block (if blocking) and size of the buffer to 
> >>allocate (if necessary) as well as to avoid a catch block.
> >
> >Even that would be difficult to implement.  Assume you
> >implement that on a BSD socket.  If you do that strainght
> >forward, so just map the read to a socket read etc, you
> >don't have that test call.
> >
> >If you want to provide the test call, you have to
> >continously READ the data from the socket, to push it into a
> >cache, and to return the current size of the cache on the
> >test call.
> >
> >  - How much do you cache?  1kB?  1MB? 1GB?
> >  - How do you know that the app actually wants all the
> >    data?
> >  - you imply the existence of a thread in the SAGA
> >    implementation which continously reads on the stream
> >  - you loose zero copy, so degrade performance
> >
> >These are again exactly the problems the message API is
> >supposed to solve - there the cache size is given by the
> >message size, and it is assumed that the application is
> >interested in one complete message, at least.  And zero copy
> >should be preserved as the message size will be known in
> >advance.
> >
> >Cheers, Andre.
-- 
"So much time, so little to do..."  -- Garfield