[SAGA-RG] Service Discovery

[Fisher, SM (Steve)]

Hi,

I have a new copy attached - we can discuss it this week

Steve

> > Andre's email
> > =============
> >  
> >   - it would be good to tie the API package closer to the
> >     other SAGA packages.  In particular, it would be very
> >     good if the url returned by service.get_url() would
> >     explicitely be formed so that it can be used for the
> >     construction of saga::job_service, saga::directory,
> >     saga::rpc instance etc.  
> >  
> > This depends upon what is published - if the URL is published
> > correctly we will return it.
> 
> Ah, I see: there is no prescription on what the URL is
> usable for, as you have, naturally, no control over what
> URLs are added to the information repository in the first
> place.  Understand.
> 
> However, that leaves some questions on how to use the
> returned URLs in the application.  Assume an application
> queries for 
> 
>   list <string> urls = d.list_services ("*data*", "*", "*");
> 
> (query probably ill formatted...) Can the application use
> the returned URL to create a saga::directory, or a
> saga::file?
> 
> I assume that this question cannot be answered within the SD
> API, and I am not sure what mechanism could be used to
> answer it at all.  Any thoughts?
> 
> However, if that use of the returned URLs (to create
> saga::xyz class instances) is the intended use of the SD
> API, adding a statement in that respect would clarify the
> intendet use of the extension.

The point of service discovery is to locate a service.  Any SAGA call
which needs to connect to a remote service and that makes this explict
can make use of SD. For example the RPC of 3.13 requires the URL of the
service provider. Also the job_service requires as input the contact
string for the resouce manager. Servide discover provides these URLs. I
haved added a sentence or two explaining this.

 
> I have to apologize, my mistake: I meant it should be a
> class, not an interface.  The following still holds: "in the
> langauge independend specs we agreed to model the functional
> API packages as classes."

Fixed


> > The discoverer should be a class returning services.
> > 
> > Incidentally it may makes sense to rename "service" to
> > "service_description".
> 
> Yes, it does.

Fixed


> >   - we do have an attribute interface (doh! ;-) in the SAGA
> >     core API spec.  I think it would be best to use that
> >     interface for the service interface/class.
> > 
> > Yes - there seems to be no problem with that in principle for the
> > service attributes.
> > 
> >   - why is get_attributes on discoverer, but get_values on
> >     service?  That seems inconsistent?
> > 
> > Discoverer has get_attribute_names() method which will 
> return an array
> > of all possible standard names deliverable by the existing 
> information
> > system adapters.  All services have the same set so associating this
> > data with the discoverer rather then the service seems to 
> make sense.
> 
> If the attribute set is fixed, then using the
> saga::attribute interface in the service (or
> service_description) instances would be easy.  The document
> should specify what attributes are available IMHO.  Again,
> technically this looks more and more like the
> saga::job_description class, and IMHO it would be good to
> design the service(_description) class the same way.

If we want to use the attribute interface, and I agree that we should,
perhaps the best way is to have two extra classes both of which
implement the Attribute interface. Then you can say, where s is a
service_description:

  s.get_data().get_attribute("some key value in the key value pair")

or 

  s.get_glue().get_attribute("some predefined 'glue' attribute")

This avoids the name clash problem - though we need a better name than
get_glue()

I would like people's agreement that this is the right way to go before
doing this. It makes things simepler but adds 2 trivial classes. I think
I like it!
 
> >   - is it possible to tighten the definition of possible
> >     query strings?  For example, the current spec would
> >     allow to query with the service filter for
> > 
> >       - type = "ftp"
> >       - type = "gsiftp"
> >       - type = "gridftp"
> >       - type = "remote data"
> > 
> >     How does the end user know what to use?  Does Glue help
> >     here (please excuse my ignorance about Glue...)
> > 
> > In the case of the type it is the responsibility of the service
> > provider, publishing the service to publish a well known service
> > type. I prefer the service name to take the form of a DNS registered
> > name - registered by the service provider. e.g. org.globus.gridftp.
> 
> Ok, so thats out of scope.  SHould be mentioned in the text
> I think.

Done
 
> >   - I am still worried that SAGA uses regular expressions in
> >     one place, wildcards in other places, and now SQL, too.
> >     Not that I know a better approach, or even solution, but
> >     we should keep that problem in mind.  I think your
> >     statement "uses SQL 92 syntax as if it were part of a
> >     WHERE clause" is good: that is something which has a
> >     good chance to be reusable by other SQL related
> >     packages, should they emerge.
> > 
> > We really need the power of SQL!
> 
> ok

I have tried to justify the choice in the text

 
> >   - I am somewhat confused about the difference between keys
> >     and attributes in the service interface/class...  What
> >     is the difference, really?  Do you have an example?
> > 
> > The attributes are based on the GLUE attributes. The key/value pairs
> > are defined by the service publisher. The service publisher 
> is free to
> > define his own key names.
> > 
> > The key names may clash with the glue attribute names so we 
> cannot use
> > the attribute interface for both. We favout using the attribute
> > interface for the glue attributes and retain our current 
> solution for
> > the key value pairs.
> 
> Ah, that clarifies a lot.  The first paragraph above would
> help to understand the document, and should be added to the
> text I think.

Done

> Interesting problem, clashing attribute name spaces...
> I have no good idea at the moment, will think about it...
> 
> 
> >   - Is it actually possible to tighten the format of the VO
> >     string, e.g. is there some commonly agreed upon format?
> > 
> > In EGEE/gLite/WLCG we are moving towards DNS names again to avoid
> > clashes. However again the actual names are out of scope. GLUE makes
> > recommendations but insists upon nothing.
> 
> ok
> 
> 
> >   - I think service.get_vo_names should be changed into an
> >     attribute, too:  service.get_attribute ("vo_names").
> >     Does that make sense?
> > 
> > We can do that. We will keep the get_url call however as it 
> is a very
> > common one.
> 
> Its not a problem to have the URL as an attrubute, _and_ to
> have a specific getter method.  We have the same for job
> state for example, which is an attribute (which makes sense
> semantically), and has a getter method (get_state), because
> it's used so often.
> 
> 
> > Pascal's email
> > ==============
> > 
> >     1) The attributes should be handled by the attribute
> >     interface. This will allow better SAGA L&F integration. 
> Also this
> >     might remove the need for specific get_vo, get_key & get_values
> >     methods. These can be treated as attributes of the 
> class. In SAGA,
> >     the getters and setters are in general covered by the attributes
> >     interface and the implementation allows specific 
> support for them
> >     if needed. The SAGA API should, for the time being, go for the
> >     minimum common set of functions.
> > 
> > Yes - see above
> > 
> >     2) I totally agree with Andre that the service should be an
> >     interface and not a class. Well in a Java binding it might be an
> >     all interface anyway.
> > 
> > Yes - see above
> > 
> >     3) The use of SQL is what made me read the section 
> twice. I think
> >     we had already a long discussion on the other parts of 
> the API on
> >     the usage of REGX, wildcards, SQL, etc. We might try to 
> sort this
> >     out either by F2F or telecon.
> > 
> > I don't think it is feasible to get the desired behaviour 
> without SQL
> > or something similar. We do want the code to be useful in gLite.
> 
> It would be good to check what other service discovery
> backends are using, and if the used SQL is translatable to
> that.  Do you have any experiences with other backends?
> 
> 
> >     4) The name space of this API should be better then 
> "sd". Why not
> >     "discovery" or something more expressive?
> > 
> > As we are not allowed to make the namespace visible in the
> > implementation I don't really care too much what goes in 
> the SIDL as a
> > name space. What is the justification for not having sub-namespaces
> > within SAGA as it says in section 2.2.1 of the spec?
> 
> 1. it would lead to clashes between namespace and class
>    names (file.file, stream.stream, rpc.rpc, job.job). 
> 
> 2. languages w/o name space support would need to flatten
>    the namespace, resulting in method names like:
> 
>      saga_logical_file_logical_directory_list ("*");
> 
> 3. given the two (possibly minor) disadvantages above, and
>    no percieved need for additional name spaces, we decided
>    against their use.

I think it was a bad choice - but I can live with these rules

> >     The API specification document and the language bindings L&F are
> >     different things. In the document we have a more C style
> >     coding. When the implementation is done we should use language
> >     specifics:
> > 
> >     - list_services() -> in C: list_services()
> >     - list_services() -> in C++: ListServices()
> >     - list_services() -> in Java: list ()
> > 
> > Sometime rather soon we should define these styles. 

So are we free to add namespaces - like all other large packages ;-)

Steve
-------------- next part --------------
A non-text attachment was scrubbed...
Name: saga_sd.dvi
Type: application/octet-stream
Size: 44428 bytes
Desc: saga_sd.dvi
Url : http://www.ogf.org/pipermail/saga-rg/attachments/20070821/5c9d73fb/attachment.obj