[saga-rg] SAGA strawman API: comments to Stream functions

[Pascal Kleijer]

Hi all,

Just a remark:

Thomas Eickermann wrote:
> Hi all,
> 
> here is some more feedback to the API regarding the steering use-cases
> "Collaborative Visualization of Atmospherical Data" and
> "Computational steering of a ground water pollution simulation"
> Both use our VISIT toolkit for the communication and i looked
> at the Stream API to check, whether VISIT could be build on top of it.
> 
> The overall answer is 'yes, but ...' there are mainly two things missing:
> 
> * resource discovery (as Andrei also mentioned in his comment),
>   we would need a functionality like:
>      register(service_name, url);
>      unregister(service_name, url);
>      url = query(service_name);
> 
> * in both use-cases there is an interactive visualisation/steering
>   application sitting at one end of the stream. Such applications
>   typically use some kind of GUI-toolkit with operates in an
>   event-callback mode or similar, where callbacks can be registered
>   for events occuring at socket/pipe/...-descriptors. On the other hand, in
>   many implementations, a Stream will somehow use a socket, pipe or
>   something else with a descriptor. Therefore we need a function that 
> returns
>   the descriptor associated to a stream:
>      fd = descriptor(Stream);
> 
>   However, users have to be aware, that several Streams may share the 
> same decriptor
>   (e.g. in VISIT, if several Streams are tunneled through a single 
> connection
>   of a different protocol like ssh).

A descriptor might not be available. Take the case in Java, you 
basically get a stream back but unless you query for the class and can 
know what you are looking for, there is no such information available. 
Also Java enables to add as many filters between the original 
source/drain and the actual end point (user), you can thus completely 
mask what is happening to the user. Also in Java the descriptor is not 
related to the stream but to the factory that creates the stream.

What about the 80/20 rule for the SAGA? Is this not too specific?




> 
> * a general remark on Streams: i like the idea of keeping the Stream-API 
> close
>   to the BSD socket API. However, as Andrei mentioned, we also often have
>   to deal with more message- or block-oriented communication patterns.
>   In the internal API the we use in the VISIT toolkit, we have therefore
>   added a timeout parameter to all functions and slightly changed the 
> semantics.
>   For example
> 
>   nread = read(stream, buffer, size, timeout);
> 
>   works as: try to read size bytes from stream, return if either:
>   - size bytes have been read, or
>   - timeout seconds have passed (wait forever, if timeout < 0), or
>   - an error has occured
>   the function returns the number of bytes read or -1 in case of an error.
> 
>   This differs from the normal BSD-read, which blocks until *some* data 
> is available
>   and return that data (but nor more than size bytes).
> 
>   We found that more convenient to implement synchronous 
> message-oriented applications
>   and 'normal' Stream communication with a single and straight-forward API.
> 
> Best regards,
> Thomas
> 
> 
> Andrei Hutanu wrote:
>> Hi all,
>>
>> Here is my feedback regarding the Viz-LSU use case. In some cases
>> these requirements might be outside the scope of SAGA, I would like to 
>> know if that is the case though ..
>>
>> *Block-based data transmission is not covered by the current API
>> *Resource discovery is not covered by the current API
>> *Job submission to multiple resources (co-scheduling) is not covered 
>> by the current API
>> *Simple job submission seems to be covered by the API. Here is a list 
>> of things
>> that are not covered (because outside the 80-20 rule?)  and
>> there doesn't seem to be a "generic" attribute in the JobDefinition 
>> class where these attributes
>> could be specified if the underlying scheduler happens to support them.
>> ** Logical file requirement (the jobs needs to run on a machine where 
>> an instance of this particular logical file exists)
>> ** Graphics requirements
>> ** Networking requirements (network interface, bandwidth to ..)
>> ** Performance-based descriptions : GFlops, memory bandwidth ..
>>
>> Andrei
>>
>> Thilo Kielmann wrote:
>>
>>> Dear all,
>>>
>>> the SAGA-RG has its use case document up for public comment, ending 
>>> oct 30.
>>>
>>> So far, there are 0 comments! :-(
>>>
>>> I hereby urge everybody to have a look and comment, even if the 
>>> comment is just
>>> trivially in favour...
>>>
>>>
>>> Thanks for your help,
>>>
>>>
>>> Thilo
>>>  
> 
> ------------------------------------------------------------
> Dr. Thomas Eickermann
> Zentralinstitut fuer Angewandte Mathematik
> Forschungszentrum Juelich GmbH
> D-52425 Juelich
>                                      Phone: +49 2461 61-6596
> Email: Th.Eickermann at fz-juelich.de   Fax:   +49 2461 61-6656
> 


-- 

Best regards,
Pascal Kleijer

----------------------------------------------------------------
   HPC Marketing Promotion Division, NEC Corporation
   1-10, Nisshin-cho, Fuchu, Tokyo, 183-8501, Japan.
   Tel: +81-(0)42/333.6389       Fax: +81-(0)42/333.6382