[Nmc-wg] Circuit monitoring - next draft of MDM proposal
Jason Zurawski
zurawski at internet2.edu
Wed Mar 30 08:26:18 CDT 2011
Hi Roman/All;
Comments inline:
On 3/30/11 8:04 AM, Roman Łapacz wrote:
> W dniu 2011-03-29 17:57, Aaron Brown pisze:
>> Hey Roman,
>>
>> Comments inline.
>>
>> Cheers,
>> Aaron
>>
>> On Mar 29, 2011, at 8:31 AM, Roman Łapacz wrote:
>>
>>> W dniu 2011-03-28 14:23, Aaron Brown pisze:
>>>>
>>>> On Mar 24, 2011, at 9:24 AM, Roman Łapacz wrote:
>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm sending updated document of MDM circuit monitoring.
>>>>>
>>>>> Aaron, Jason, all, you can find a simplified version of client
>>>>> access algorithm (section 5.3.2) but it still supports that one
>>>>> that you formulated some time ago and we all accepted. It is
>>>>> simplified because some assumptions have been made in the use case
>>>>> with AutoBAHN (see the picture in 5.3.2). Please, take a look at
>>>>> the examples of messages for this algorithm but also for the
>>>>> workflow presented in 5.2.1. Check if massages from and to the
>>>>> MA(at)/MA(t) are compatible with those for the pS-PS TS. Also take
>>>>> a look at the example of massage that is sent from the hLS to the
>>>>> gLS (section 9.2.3).
>>>>>
>>>>> Some time ago we were talking about capabilities and dropping the
>>>>> service type field in the lookup information. I found it very
>>>>> useful while working on the examples of messages. Although the MA
>>>>> services (MA(at) and MA(t)) store the topology information they can
>>>>> be treated as the pS-PS TS if they support topology storage
>>>>> functionality and register that with the hLS. Take a look at the
>>>>> examples of LSRegisterRequest type (I've used the parameter
>>>>> eventType but probably normal eventType element would be more
>>>>> suitable; but first I'd like to see your comments). So far I've
>>>>> proposed 6 capabilities:
>>>>>
>>>>> http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 (for MA)
>>>>> http://ggf.org/ns/nmwg/ops/storage/measurement/aggregated/autobahn/2.0
>>>>> (for MA with data aggregated by AutoBAHN)
>>>>> http://ggf.org/ns/nmwg/ops/creation/measurement/2.0 (for MP)
>>>>> http://ggf.org/ns/nmwg/ops/integration/autobahn/2.0 (for SIP which
>>>>> connects pS with AutoBAHN)
>>>>> http://ggf.org/ns/nmwg/ops/storage/topology/2.0 (for TS or MA(t)
>>>>> which store topology information)
>>>>> http://ggf.org/ns/nmwg/ops/storage/topology/aggregated/autobahn/2.0
>>>>> (for TS or MA (at) which store the topology information aggregated
>>>>> by AutoBAHN)
>>>>
>>>>
>>>> So the proposal uses eventTypes for operations (e.g.
>>>> http://ggf.org/ns/nmwg/ops/storage/measurement/2.0 means "this
>>>> services can be used to store measurements"). My recollection is
>>>> that the goal was to move away from using eventTypes as operations;
>>>> they'd just be used to describe functions. To do this, we'd need
>>>> some way of including the supported messages in the elements
>>>> registered up by the services. The clients could then look for
>>>> things like "the services that support MetadataStoreRequest and
>>>> utilization data". I'm not positive the best way forward on this.
>>>
>>> I would not use message types because they also might be misleading
>>> like service types. I like the idea of registering the operations
>>> that are supported by a service. Changing a bit your example: "the
>>> services that support store operation of utilization data".
>>
>> I guess the open question here is what meaning service type (or
>> supported operation) is meant to convey. Is there an instance where
>> two different services would support the same data types and the same
>> messages, but have different semantics?
>
> I don't think there's such an instance but it's possible to have it in
> future. Jason, what do you think about 'operation' parameters or use of
> message types?
> (For the time being I'll replace eventType parameters with operation
> parameters and later update it when we have something better. In fact I
> have to do this because topology services of two pS implantation have
> different service types - "TS" and "MA").
I have to admit I am not following this proposal very well, why do you
feel it is necessary to do this? Topology data is topology data, no
matter who is storing it. I am not really sure why it's important to be
able to identify the type of service directly via the Lookup Service in
this case. If you can go into more detail about why this is necessary
at the information services level perhaps that would be helpful.
If you need to identify what type the service really is you can always
ask it directly, after the Lookup Service tells you that it contains
topology. This is just one extra step after all, and avoids trying to
impart yet another meaning on the eventType. I would rather not
overload the eventType to mean both "type of data" and "service
capability" unless there is a very compelling reason to do so.
You mention also using a "parameter", how would that work? Would this
be similar to how we currently use the "keyword" construct? Note that
if this were the case the summarization algorithm and discovery APIs
would need to change, this is not impossible of course, but I would like
to know the entire reason before forcing a global flag day to implement
this.
>>>> As to the parameter-based eventTypes, I think we need to move away
>>>> from those if, for no other reason, than to ensure that clients can
>>>> uniformly query the LS without having to worry about what form the
>>>> eventType will take. Having multiple ways, especially if different
>>>> software packages do things differently, is more likely to result in
>>>> clients that can only work with one type or the other.
>>>
>>> I've used eventType parameters and I knew this wasn't a good solution
>>> but I'd like to see your comments on this. The parameter 'operation'
>>> would be more suitable. What do you think?
>>>
>>>> A few other comments and questions on the document after a first pass.
>>>>
>>>> 1. We'll need to think about the specifics of how to generate the
>>>> segment
>>>> identifiers when there's a OSCARS/AutoBAHN bridging. When this
>>>> happens, there are
>>>> 2 very different identifier schemes, and it's not clear how an
>>>> agent would
>>>> go about generating the circuit descriptor since they need to
>>>> know how other
>>>> domains are going to generate their segment identifiers.
>>>
>>> But the GLIF format and its domain part isn't enough? An agent does
>>> not have to know what the last part of the GLIF urn means.
>>> (I'll forward this comment to the AutoBAHN team)
>>
>> In the model, the starting domain creates the circuit descriptor.
>> However, to do that, that domain needs to know how the other domains
>> are going to set the identifiers for their segment descriptors. There
>> are two ways to do that:
>>
>> 1) have a programmatic way that each domain uses to create the segment
>> id using information that is available to all domains
>> 2) have the starting domain contact each of the other domains and
>> request the segment identifiers
>>
>> I'd prefer #1 if possible since it makes it substantially easier to
>> deploy. However, for #1, there's not a good way, if OSCARS is the
>> starting domain, to figure out how an AutoBAHN domain is going to name
>> its segments since all OSCARS will have is a GRI (not compatible with
>> AutoBAHN's id structure), and the domain names.
>
> I'm afraid that #2 will have to be used in such case. I talked to the
> AutoBAHN guy and he promised to comment this soon. My first impression
> is that #1 will require some extended communication between those two
> systems and thus some major changes in the existing implementations.
>
>>
>>>> 2. In the segment and circuit descriptors, there are two different
>>>> relationship
>>>> names used: 'connects' and 'serialcompound'. In both cases, the
>>>> relationship
>>>> is describing the constituent elements that make up the link
>>>> (links in the
>>>> case of the circuit descriptor, and ports in the case of the segment
>>>> descriptor). Since they're describing the same conceptual
>>>> relationship,
>>>> having the same name would be good.
>>>
>>> Agree. I was thinking about this while writing descriptor examples in
>>> the doc but which one to choose? I would take "connects" but this
>>> must be agreed by all of us. So are you fine to take "connects"?
>>
>> I'd just been using "over", but would be fine with "connects".
>
> Let's stay with "connects" (I remember Freek's concern that "over"
> refers more to multi-layering).
>
>>
>>>> 3. It looks like the SIP agent is registering the segment ID. I'm
>>>> curious why
>>>> it's being registered there.
>>>
>>> The SIP contains info about running monitoring sessions and stores it
>>> after they are finished to keep the history (detailed info - segment
>>> descriptors - can be fetched from the MA(t) or the MA(at)).
>>>
>>>
>>>>
>>>> 4. All of the XQuery requests to the hLS are of the form "what
>>>> services contain
>>>> eventType X?". This type of request can be handled using the summary
>>>> requests which has the side benefit of not embedding more XQuery into
>>>> protocol.
>>>>
>>>> 5. Relatedly, I'd like to specify that XPath queries be the only
>>>> ones used the
>>>> TS. This way there's a hope of moving away from the XQuery-based
>>>> protocols,
>>>> and their dependency on an XML Database. Since XPath can be
>>>> converted to SQL
>>>> requests, that'd be much easier to transition to something else. It'd
>>>> probably also be good to specify exactly the subset of XPath to
>>>> be supported
>>>> so that we don't hit the XQuery issue where you are required to
>>>> have a
>>>> full-blown XPath implementation to implement the protocol. To
>>>> keep the
>>>> XPath syntax simplified, it'd probably be good to eschew the use of
>>>> functions (at least in the beginning). e.g.: using
>>>> /*:link[@id="XYZ"] or we
>>>> could strip off the new for the namespace and have
>>>> /link[@id="XYZ"] .
>>>
>>> To keep compatibility I'll replace xquery statements with xpaths (we
>>> have started doing this).
>>>
>>>>
>>>> 6. In the TSQueryResponse example, it looks like far more is
>>>> returned than I'd
>>>> expect given the query. Was this just a copy-and-paste instance?
>>>
>>> Yes, you're right (SIP->MA(t)). There's a short comment in the
>>> request why.
>>>
>>>>
>>>> 7. The TSAddRequest wraps the individual network elements in an
>>>> nml:topology
>>>> element. In the TSQueryResponse response, the individual elements are
>>>> wrapped in a 'datum'. It'd probably make sense to have the
>>>> TSAddRequest wrap
>>>> the individual elements in a 'datum' as well.
>>>
>>> Fine for me (I didn't do it this way because I followed the example
>>> http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-TopologyService/doc/requests/TSAddRequest.xml
>>> where there's no datum element)
>>
>> Yeah, I figured as such. I'd just like to change the semantics to be
>> more similar to other services.
>>
>>>> 8. What kinds of additional info are you thinking about in 9.2.1.9?
>>>> Is this
>>>> stuff that is expected to be retrievable by the client, or is it
>>>> stuff that
>>>> the MA is going to strip out?
>>>
>>> Additional info in parameters may contain some general info about the
>>> reservation and its monitoring (for example: duration) but now I
>>> don't have anything specific in my mind.
>>>
>>>> 9. I'm curious what kinds of subjects are envisioned for the SIP to MP
>>>> messages.
>>>
>>> I don't think the MDM release has an MP service (collecting the link
>>> status) ready to be used so it will be developed sooner or later.
>>> It's difficult now to say how a possible request will look like (not
>>> sure I'll be involved in the work on messages for it).
>>>
>>>> 10. In 9.2.2.1, the client does a request for every service
>>>> containing anything
>>>> on pionier.net <http://pionier.net/>. That could easily become
>>>> non-scalable. It might make sense
>>>> to qualify the request a bit (e.g. by including eventTypes of
>>>> interest):
>>>
>>> This request goes to the gLS so only address(es) of hLS(es) of
>>> pionier domain will be provided (I don't expect a domain may have
>>> many hLS(es)). But of course additional filtering info may be used
>>> (depends on what is summarized and sent by the hLS to the gLS).
>>
>> Well, the result would be any hLS that has any data on the pionier
>> domain. So if there are other domains out there that running PingER,
>> HADES, owamp or bwctl tests against pionier hosts, they'd be returned
>> as well. On the flipside, I'm not sure adding the eventType would help
>> this situation since many of those domains could also have utilization
>> data (which may or may not be applicable to the pionier domain), so a
>> search for utilization data and pionier could grab the same set of
>> domains.
>
> I'm thinking about this differently, the gLS would response with the
> hLS(es) that are deployed in the pionier domain. The hLS would register
> information that contains (meta)data from specific domain(s). So, for
> example: "give me the hLSes deployed in/representing pionier.net that
> know services with utilization data".
The queries don't really work that way, you can't ask a question such as
"tell me who the authoritative hLS is for domain x". Remember this is a
federated model, anyone can set up an hLS at any time in a given domain
(without the knowledge of someone else who as already set one up) and
claim it is authoritative. You could conceivably also do this in an
unrelated domain.
The queries are designed to revolve around the data itself instead of
some notion of administrative knowledge. You search for data that is
related to domain x, and you receive many answers (e.g. data that
originated there and has an end in another domain, or the reverse).
Aaron's (and my) concern is that doing a query for just a domain
qualifier is the same thing as dong a DNS zone transfer - it dumps lots
and lots of information and is very resource consuming. There is
nothing that stops you from doing this of course, its just not something
a kind user should do. I believe it is possible for a more specific
query (e.g. give me topology data for domain x, or give me topology data
for domain x and domain y) to be used instead. Is this fair?
Thanks;
-jason
> Cheers,
> Roman
>
>> I'm wondering if the LS summarization should account for the
>> relationships between the data and the domains. The semantics of the
>> summary query to the gLS would be "give me the hLSes that know about
>> services with utilization data about internet2.edu
>> <http://internet2.edu>", as opposed to, "give me the hLSes that know
>> about services with data about internet2.edu <http://internet2.edu>,
>> and services with utilization data).
>>
>>>> <nmwg:metadata metadataIdRef="meta1"
>>>> id="metadata.8532851"><summary:subject>
>>>> <summary:subject>
>>>> <nml:domain id="pionier.net <http://pionier.net/>"/>
>>>> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType>
>>>> </summary:subject>
>>>> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType>
>>>> </nmwg:metadata>
>>>>
>>>> 11. Also, Jason noticed that the LS response has a datum element
>>>> that it shouldn't:
>>>>
>>>> <nmwg:message>
>>>> <nmwg:metadata metadataIdRef="meta1" id="metadata.8532851">
>>>> <summary:subject>
>>>> <nml:domain id="psu.edu <http://psu.edu/>" />
>>>> <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType>
>>>> </summary:subject>
>>>> <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/summary/2.0</nmwg:eventType>
>>>> </nmwg:metadata>
>>>> <nmwg:data metadataIdRef="metadata.8532851" id="data.14027668">
>>>> <nmwg:metadata xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/"
>>>> id="3b010db87b462c778bd017447fcc762c">
>>>> <perfsonar:subject
>>>> xmlns:perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/">
>>>> <psservice:service
>>>> xmlns:psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/">
>>>> <psservice:serviceName>Lookup Service</psservice:serviceName>
>>>> <psservice:accessPoint>http://128.118.46.245:9995/perfSONAR_PS/services/hLS</psservice:accessPoint>
>>>> <psservice:serviceType>hLS</psservice:serviceType>
>>>> <psservice:serviceDescription>perfSONAR_PS Lookup Service at Penn
>>>> State in University Park, PA USA</psservice:serviceDescription>
>>>> </psservice:service>
>>>> </perfsonar:subject>
>>>> </nmwg:metadata>
>>>> </nmwg:data>
>>>> </nmwg:message>
>>>
>>> I'll update it.
>>>
>>> thanks,
>>> Roman
>>>
>>>>
>>>>
>>>> Cheers,
>>>> Aaron
More information about the Nmc-wg
mailing list