[SAGA-RG] Service Discovery

[Andre Merzky]

Content-Description: comments.txt
> Andre's email
> =============
>  
>   - it would be good to tie the API package closer to the
>     other SAGA packages.  In particular, it would be very
>     good if the url returned by service.get_url() would
>     explicitely be formed so that it can be used for the
>     construction of saga::job_service, saga::directory,
>     saga::rpc instance etc.  
>  
> This depends upon what is published - if the URL is published
> correctly we will return it.

Ah, I see: there is no prescription on what the URL is
usable for, as you have, naturally, no control over what
URLs are added to the information repository in the first
place.  Understand.

However, that leaves some questions on how to use the
returned URLs in the application.  Assume an application
queries for 

  list <string> urls = d.list_services ("*data*", "*", "*");

(query probably ill formatted...) Can the application use
the returned URL to create a saga::directory, or a
saga::file?

I assume that this question cannot be answered within the SD
API, and I am not sure what mechanism could be used to
answer it at all.  Any thoughts?

However, if that use of the returned URLs (to create
saga::xyz class instances) is the intended use of the SD
API, adding a statement in that respect would clarify the
intendet use of the extension.


>   - service should be an interface, not a class.  A language
>     binding (e.g. for Java) might render that as interface,
>     but in the langauge independend specs we agreed to model
>     the functional API packages as classes.  Unless I am
>     missing some point here?
> 
> That was a mistake in the text - in the SIDL we have the service as an
> interface but not in the accompanying text. However if the service is
> to implement an attribute interface then it can't itself be an
> interface but must become a class - we could of course extend rather
> than implement.

I have to apologize, my mistake: I meant it should be a
class, not an interface.  The following still holds: "in the
langauge independend specs we agreed to model the functional
API packages as classes."


> The discoverer should be a class returning services.
> 
> Incidentally it may makes sense to rename "service" to
> "service_description".

Yes, it does.


>   - we do have an attribute interface (doh! ;-) in the SAGA
>     core API spec.  I think it would be best to use that
>     interface for the service interface/class.
> 
> Yes - there seems to be no problem with that in principle for the
> service attributes.
> 
>   - why is get_attributes on discoverer, but get_values on
>     service?  That seems inconsistent?
> 
> Discoverer has get_attribute_names() method which will return an array
> of all possible standard names deliverable by the existing information
> system adapters.  All services have the same set so associating this
> data with the discoverer rather then the service seems to make sense.

If the attribute set is fixed, then using the
saga::attribute interface in the service (or
service_description) instances would be easy.  The document
should specify what attributes are available IMHO.  Again,
technically this looks more and more like the
saga::job_description class, and IMHO it would be good to
design the service(_description) class the same way.


>   - is it possible to tighten the definition of possible
>     query strings?  For example, the current spec would
>     allow to query with the service filter for
> 
>       - type = "ftp"
>       - type = "gsiftp"
>       - type = "gridftp"
>       - type = "remote data"
> 
>     How does the end user know what to use?  Does Glue help
>     here (please excuse my ignorance about Glue...)
> 
> In the case of the type it is the responsibility of the service
> provider, publishing the service to publish a well known service
> type. I prefer the service name to take the form of a DNS registered
> name - registered by the service provider. e.g. org.globus.gridftp.

Ok, so thats out of scope.  SHould be mentioned in the text
I think.


>   - I am still worried that SAGA uses regular expressions in
>     one place, wildcards in other places, and now SQL, too.
>     Not that I know a better approach, or even solution, but
>     we should keep that problem in mind.  I think your
>     statement "uses SQL 92 syntax as if it were part of a
>     WHERE clause" is good: that is something which has a
>     good chance to be reusable by other SQL related
>     packages, should they emerge.
> 
> We really need the power of SQL!

ok


>   - I am somewhat confused about the difference between keys
>     and attributes in the service interface/class...  What
>     is the difference, really?  Do you have an example?
> 
> The attributes are based on the GLUE attributes. The key/value pairs
> are defined by the service publisher. The service publisher is free to
> define his own key names.
> 
> The key names may clash with the glue attribute names so we cannot use
> the attribute interface for both. We favout using the attribute
> interface for the glue attributes and retain our current solution for
> the key value pairs.

Ah, that clarifies a lot.  The first paragraph above would
help to understand the document, and should be added to the
text I think.

Interesting problem, clashing attribute name spaces...
I have no good idea at the moment, will think about it...


>   - Is it actually possible to tighten the format of the VO
>     string, e.g. is there some commonly agreed upon format?
> 
> In EGEE/gLite/WLCG we are moving towards DNS names again to avoid
> clashes. However again the actual names are out of scope. GLUE makes
> recommendations but insists upon nothing.

ok


>   - I think service.get_vo_names should be changed into an
>     attribute, too:  service.get_attribute ("vo_names").
>     Does that make sense?
> 
> We can do that. We will keep the get_url call however as it is a very
> common one.

Its not a problem to have the URL as an attrubute, _and_ to
have a specific getter method.  We have the same for job
state for example, which is an attribute (which makes sense
semantically), and has a getter method (get_state), because
it's used so often.


> Pascal's email
> ==============
> 
>     1) The attributes should be handled by the attribute
>     interface. This will allow better SAGA L&F integration. Also this
>     might remove the need for specific get_vo, get_key & get_values
>     methods. These can be treated as attributes of the class. In SAGA,
>     the getters and setters are in general covered by the attributes
>     interface and the implementation allows specific support for them
>     if needed. The SAGA API should, for the time being, go for the
>     minimum common set of functions.
> 
> Yes - see above
> 
>     2) I totally agree with Andre that the service should be an
>     interface and not a class. Well in a Java binding it might be an
>     all interface anyway.
> 
> Yes - see above
> 
>     3) The use of SQL is what made me read the section twice. I think
>     we had already a long discussion on the other parts of the API on
>     the usage of REGX, wildcards, SQL, etc. We might try to sort this
>     out either by F2F or telecon.
> 
> I don't think it is feasible to get the desired behaviour without SQL
> or something similar. We do want the code to be useful in gLite.

It would be good to check what other service discovery
backends are using, and if the used SQL is translatable to
that.  Do you have any experiences with other backends?


>     4) The name space of this API should be better then "sd". Why not
>     "discovery" or something more expressive?
> 
> As we are not allowed to make the namespace visible in the
> implementation I don't really care too much what goes in the SIDL as a
> name space. What is the justification for not having sub-namespaces
> within SAGA as it says in section 2.2.1 of the spec?

1. it would lead to clashes between namespace and class
   names (file.file, stream.stream, rpc.rpc, job.job). 

2. languages w/o name space support would need to flatten
   the namespace, resulting in method names like:

     saga_logical_file_logical_directory_list ("*");

3. given the two (possibly minor) disadvantages above, and
   no percieved need for additional name spaces, we decided
   against their use.


>     The API specification document and the language bindings L&F are
>     different things. In the document we have a more C style
>     coding. When the implementation is done we should use language
>     specifics:
> 
>     - list_services() -> in C: list_services()
>     - list_services() -> in C++: ListServices()
>     - list_services() -> in Java: list ()
> 
> Sometime rather soon we should define these styles. 

Oh yes.

Cheers, Andre.


> For both C++ and
> Java I would favour listServices(). Howebver it may be best to just
> follow the majority of the existing implementation code.
> 
> Steve and Paventhan
> 
> 




-- 
"XML is like violence: if it does not help, use more."