[saga-rg] SAGA strawman API: comments to Stream functions

[Thomas Eickermann]

Hi all,

here is some more feedback to the API regarding the steering use-cases
"Collaborative Visualization of Atmospherical Data" and
"Computational steering of a ground water pollution simulation"
Both use our VISIT toolkit for the communication and i looked
at the Stream API to check, whether VISIT could be build on top of it.

The overall answer is 'yes, but ...' there are mainly two things missing:

* resource discovery (as Andrei also mentioned in his comment),
   we would need a functionality like:
      register(service_name, url);
      unregister(service_name, url);
      url = query(service_name);

* in both use-cases there is an interactive visualisation/steering
   application sitting at one end of the stream. Such applications
   typically use some kind of GUI-toolkit with operates in an
   event-callback mode or similar, where callbacks can be registered
   for events occuring at socket/pipe/...-descriptors. On the other hand, in
   many implementations, a Stream will somehow use a socket, pipe or
   something else with a descriptor. Therefore we need a function that returns
   the descriptor associated to a stream:
      fd = descriptor(Stream);

   However, users have to be aware, that several Streams may share the same decriptor
   (e.g. in VISIT, if several Streams are tunneled through a single connection
   of a different protocol like ssh).

* a general remark on Streams: i like the idea of keeping the Stream-API close
   to the BSD socket API. However, as Andrei mentioned, we also often have
   to deal with more message- or block-oriented communication patterns.
   In the internal API the we use in the VISIT toolkit, we have therefore
   added a timeout parameter to all functions and slightly changed the semantics.
   For example

   nread = read(stream, buffer, size, timeout);

   works as: try to read size bytes from stream, return if either:
   - size bytes have been read, or
   - timeout seconds have passed (wait forever, if timeout < 0), or
   - an error has occured
   the function returns the number of bytes read or -1 in case of an error.

   This differs from the normal BSD-read, which blocks until *some* data is available
   and return that data (but nor more than size bytes).

   We found that more convenient to implement synchronous message-oriented applications
   and 'normal' Stream communication with a single and straight-forward API.

Best regards,
Thomas


Andrei Hutanu wrote:
> Hi all,
> 
> Here is my feedback regarding the Viz-LSU use case. In some cases
> these requirements might be outside the scope of SAGA, I would like to 
> know if that is the case though ..
> 
> *Block-based data transmission is not covered by the current API
> *Resource discovery is not covered by the current API
> *Job submission to multiple resources (co-scheduling) is not covered by 
> the current API
> *Simple job submission seems to be covered by the API. Here is a list of 
> things
> that are not covered (because outside the 80-20 rule?)  and
> there doesn't seem to be a "generic" attribute in the JobDefinition 
> class where these attributes
> could be specified if the underlying scheduler happens to support them.
> ** Logical file requirement (the jobs needs to run on a machine where an 
> instance of this particular logical file exists)
> ** Graphics requirements
> ** Networking requirements (network interface, bandwidth to ..)
> ** Performance-based descriptions : GFlops, memory bandwidth ..
> 
> Andrei
> 
> Thilo Kielmann wrote:
> 
>> Dear all,
>>
>> the SAGA-RG has its use case document up for public comment, ending 
>> oct 30.
>>
>> So far, there are 0 comments! :-(
>>
>> I hereby urge everybody to have a look and comment, even if the 
>> comment is just
>> trivially in favour...
>>
>>
>> Thanks for your help,
>>
>>
>> Thilo
>>  

------------------------------------------------------------
Dr. Thomas Eickermann
Zentralinstitut fuer Angewandte Mathematik
Forschungszentrum Juelich GmbH
D-52425 Juelich
                                      Phone: +49 2461 61-6596
Email: Th.Eickermann at fz-juelich.de   Fax:   +49 2461 61-6656

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3760 bytes
Desc: S/MIME Cryptographic Signature
Url : http://www.ogf.org/pipermail/saga-rg/attachments/20051110/c8f87e55/attachment-0003.bin