[saga-rg] SAGA strawman API: comments to Stream functions
Thomas Eickermann
th.eickermann at fz-juelich.de
Fri Dec 2 10:05:42 CST 2005
Hi Andre,
thanks for your answer. I agree with your comments that access
to raw descriptors is somewhat problematic.
Nevertheless i regard them as quite important for implementing
real-life interactive applications. Two examples:
1. AVS/Express allows to trigger a module if an event occurs at a
descriptor. I do not know of any other way to implement stream
communication within AVS/Express and i imagine, that the situation
is quite similar for other tools like OpenExplorer.
2. If you use some GUI toolkit (e.g. Qt), you can also register callbacks
for events on descriptors. If such a descriptor is not available, the
only thing you can probably do is to use timers and polling. There are
obviously good reasons to avoid that, if possible.
See further comments inline.
Best regards,
Thomas
Andre Merzky wrote:
> Hi Thomas,
>
> sorry for the late answer... *sigh*
>
>
> Quoting [Thomas Eickermann] (Nov 10 2005):
>
>>Hi all,
>>
>>here is some more feedback to the API regarding the steering use-cases
>>"Collaborative Visualization of Atmospherical Data" and
>>"Computational steering of a ground water pollution simulation"
>>Both use our VISIT toolkit for the communication and i looked
>>at the Stream API to check, whether VISIT could be build on top of it.
>>
>>The overall answer is 'yes, but ...' there are mainly two things missing:
>>
>>* resource discovery (as Andrei also mentioned in his comment),
>> we would need a functionality like:
>> register(service_name, url);
>> unregister(service_name, url);
>> url = query(service_name);
>
>
> Yes, that is indeed completely missing. I put a more
> explicit TODO item in our issue list.
>
>
>
>>* in both use-cases there is an interactive visualisation/steering
>> application sitting at one end of the stream. Such applications
>> typically use some kind of GUI-toolkit with operates in an
>> event-callback mode or similar, where callbacks can be registered
>> for events occuring at socket/pipe/...-descriptors. On the other hand, in
>> many implementations, a Stream will somehow use a socket, pipe or
>> something else with a descriptor. Therefore we need a function that
>> returns
>> the descriptor associated to a stream:
>> fd = descriptor(Stream);
>
>
> This is complicated, as we do not know _if_ there will be a
> native descriptor available, and if operations on that will
> interfere with the implementation.
>
availability:
right, i was somewhat unprecise. What i meant was the native
respresentation of a socket in the given language (a descriptor
in C, an IO::Socket obejct in Perl and probably there is something
like that in most languages.
> E.g. the implementation might be based on Globus-XIO or on
> Reliable UDP. For the first I do not know if a descriptor
> is availbale via the API, for the second there are two
> descriptors (one tcp, one udp) the application would need to
> handle.
To handle the second case, 'descriptor' would have to return
an array of descriptors. I don't see that in the first case
an implementation could return something better than a NULL-array.
>
> Also, the implementation might transparently add a protocol
> (say checksumming), and direct tempering with the desfriptor
> might very well break that protocol.
>
I agree, the only thing the user is allowed to do with the
descriptor, is to call 'select'. Read, write, seek, close
would most likely break the protocol.
However, this cannot be enforced, and i am aware that this
conflicts the principle of hiding the implementation from
the user (to minimize the risk that the user breaks something).
> So, we might be able to add the API function, but it is
> difficult to define what it should return in the cases
> outlined above...
>
>
>
>>However, users have to be aware, that several Streams
>>may share the same decriptor (e.g. in VISIT, if several
>>Streams are tunneled through a single connection of a
>>different protocol like ssh).
>
>
>
>>* a general remark on Streams: i like the idea of keeping the Stream-API
>>close
>> to the BSD socket API. However, as Andrei mentioned, we also often have
>> to deal with more message- or block-oriented communication patterns.
>> In the internal API the we use in the VISIT toolkit, we have therefore
>> added a timeout parameter to all functions and slightly changed the
>> semantics.
>> For example
>>
>> nread = read(stream, buffer, size, timeout);
>>
>> works as: try to read size bytes from stream, return if either:
>> - size bytes have been read, or
>> - timeout seconds have passed (wait forever, if timeout < 0), or
>> - an error has occured
>> the function returns the number of bytes read or -1 in case of an error.
>>
>> This differs from the normal BSD-read, which blocks until *some* data is
>> available
>> and return that data (but nor more than size bytes).
>>
>> We found that more convenient to implement synchronous message-oriented
>> applications
>> and 'normal' Stream communication with a single and straight-forward API.
>
>
> Thanks for this feedback! You may have seen the mail
> exchange to that topic. If you are interested, could you
> give us your feedback to the proposal written down at:
>
> http://wiki.cct.lsu.edu/saga/space/SAGA+API/Messages
>
> Best regards,
>
> Andre.
>
>
>
>>Best regards,
>>Thomas
>>
>>
>>Andrei Hutanu wrote:
>>
>>>Hi all,
>>>
>>>Here is my feedback regarding the Viz-LSU use case. In some cases
>>>these requirements might be outside the scope of SAGA, I would like to
>>>know if that is the case though ..
>>>
>>>*Block-based data transmission is not covered by the current API
>>>*Resource discovery is not covered by the current API
>>>*Job submission to multiple resources (co-scheduling) is not covered by
>>>the current API
>>>*Simple job submission seems to be covered by the API. Here is a list of
>>>things
>>>that are not covered (because outside the 80-20 rule?) and
>>>there doesn't seem to be a "generic" attribute in the JobDefinition
>>>class where these attributes
>>>could be specified if the underlying scheduler happens to support them.
>>>** Logical file requirement (the jobs needs to run on a machine where an
>>>instance of this particular logical file exists)
>>>** Graphics requirements
>>>** Networking requirements (network interface, bandwidth to ..)
>>>** Performance-based descriptions : GFlops, memory bandwidth ..
>>>
>>>Andrei
>>>
>>>Thilo Kielmann wrote:
>>>
>>>
>>>>Dear all,
>>>>
>>>>the SAGA-RG has its use case document up for public comment, ending
>>>>oct 30.
>>>>
>>>>So far, there are 0 comments! :-(
>>>>
>>>>I hereby urge everybody to have a look and comment, even if the
>>>>comment is just
>>>>trivially in favour...
>>>>
>>>>
>>>>Thanks for your help,
>>>>
>>>>
>>>>Thilo
>>>>
>>
>>------------------------------------------------------------
>>Dr. Thomas Eickermann
>>Zentralinstitut fuer Angewandte Mathematik
>>Forschungszentrum Juelich GmbH
>>D-52425 Juelich
>> Phone: +49 2461 61-6596
>>Email: Th.Eickermann at fz-juelich.de Fax: +49 2461 61-6656
>>
------------------------------------------------------------
Dr. Thomas Eickermann
Zentralinstitut fuer Angewandte Mathematik
Forschungszentrum Juelich GmbH
D-52425 Juelich
Phone: +49 2461 61-6596
Email: Th.Eickermann at fz-juelich.de Fax: +49 2461 61-6656
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3760 bytes
Desc: S/MIME Cryptographic Signature
Url : http://www.ogf.org/pipermail/saga-rg/attachments/20051202/dedd6a8b/attachment-0003.bin
More information about the saga-rg
mailing list