[RUS-WG] Extending the WS-Enumeration enumerate method for RUS queries

[Gilbert Netzer]

Hi everybody,

after studying the WS-Enumeration sepcification
(http://www.w3.org/Submission/WS-Enumeration/) more closely, I have a few
thoughts on how to implement a form of querying the RUS that can handle
large result sets more memory and resource efficient.

To sum up I want to suggest the following:

1. Replace the extractUsageRecords operation with the <wsen:Enumerate>
   method.
2. State the servers SHOULD implement the <wsman:OptimizeEnumeration>
   extension.
3. State the client MUST understand the result of a
   <wsman:OptimizeEnumeration> extension if they used this extension in
   their request.


First of all I think we should use WS-Enumeration for accessing a "large"
result set in handy pieces. The advantage would be that existing
enumeration implementations (on both server and client side) can be used to
do the actual data transfer. Such implementations will probably also show
up in middleware toolkits. For instance the next version of the Globus
Toolkit will contain a WS-Enumeration implementation. Besides the advantage
for implementors, this would also mean that we can save us the trouble of
specifying our own flavor of enumeration that essentially duplicates the
functionality of an existing specification. The downside is that the
WS-Enumeration is not part of WS-RF and therefore duplicates some of the
lifecycle management functionality of the WS-RF (via the Renew, GetStatus,
Release and EnumerationEnd operations). I do however not think that this is
a big issue.

In WS-Enumeration the optional Enumerate operation is used to request a new
enumeration from the data source, in our case this would be the RUS. In
this request a filter condition can be specified using the <wsen:Filter>
element. This element can be of any given dialect, however currently only
the XPath dialect is specified. This basically is equivalent to the
specification of the searchTerm currently used by the extraction methods
(extract*) so it could actually replace the extraction methods of the RUS.

One problem with the WS-Enumeration specification is that a client always
has to use at least two requests (one Enumerate and one Pull operation) to
get the requested data even if the result contains only a small number of
data items. This problem was already addressed by the WS-Management
specification
(http://www.dmtf.org/standards/published_documents/DSP0226.pdf) which
provides a extension <wsman:OptimizeEnumeration> using the WS-Enumeration
extensiability points to accomplish this in a backward compatible manner.
The RUS specification could leverage this to actually accomplish a method
to extract UsageRecord data in a very similar way that to what is currently
possible plus add good support for large result sets.

The problem with this approach could be that it makes the RUS specification
dependent on two other specifications (WS-Enumeration and parts of
WS-Management), but overall I would still consider this a worthwile approach.

Please post comments to this on the rus-wg at ogf.org mailing list and add
them to the tracker (https://forge.gridforum.org/sf/go/artf5887) if possible.

Best Regards
Gilbert Netzer
PDC/KTH