[glue-wg] flat xml rendering

Thu Oct 4 07:37:49 EDT 2012

Hi Warren, all

On 10/02/2012 03:16 PM, Warren Smith wrote:
>
> I don't think I was arguing against the substitution groups, I think
> I was arguing against the reasoning that since we may add new GLUE2
> entities in the future, we should have the -ies elements (e.g.
> Activities) at the top level to group them so that a user knows what
> elements are Activities without having to look at the schema.
>
> So yeah, it seems like we haven't convinced each other, so maybe what
> we should do is describe each of the choices, get everyone who is
> interested to vote (or whatever), make a choice and move forward.
>

What about providing several sample documents? Cannot do it for today 
meeting
but I can craft something to show ARC and EMI-ES use case, and how would 
it change
using the flat thing.

Currently we have GLUE2 XML documents based on the hierarchical schema.

> -----Original Message----- From: david.meredith at stfc.ac.uk
> [mailto:david.meredith at stfc.ac.uk] Sent: Tuesday, October 02, 2012
> 5:45 AM To: Warren Smith; stephen.burke at stfc.ac.uk;
> florido.paganelli at hep.lu.se; glue-wg at ogf.org Subject: RE: [glue-wg]
> flat xml rendering
>
> Hi Warren, all, Comments interleaved below. While I disagree with
> some of your comments below, I still don't think any of these points
> are significant blockers.
>
> Cheers, David
>
>> -----Original Message----- From: Warren Smith
>> [mailto:wsmith at tacc.utexas.edu] Sent: 20 September 2012 13:51 To:
>> Meredith, David (STFC,DL,SC); Burke, Stephen (STFC,RAL,PPD);
>> florido.paganelli at hep.lu.se; glue-wg at ogf.org Subject: RE: [glue-wg]
>> flat xml rendering
>>
>>
>> I still disagree for a few reasons.
>>
>> The grouping elements do add a tiny bit of complexity, particularly
>> if you feel (like I do) that they are unnecessary. A "flat"
>> argument along these lines is that for FutureGrid (and perhaps for
>> XSEDE in the near future), I produce GLUE 2 documents that only
>> contain a single ComputingActivity. These documents describe
>> updates to jobs that I'm pulling from scheduler log files. Using an
>> Activities element in this document is really unnecessary.
>
> I agree that it adds a 'tiny bit of extra complexity'. I guess we
> leave this to the WG to decide.
>
>
>> Regarding new elements, I don't think speculation about adding new
>> elements in the future is a strong argument for having the XML
>> schema a certain way. I'm generally wary of "just in case"
>> designing and we already seem to have a naming convention that
>> includes the base type in the element name in almost all cases. In
>> the very few current cases where an element name doesn't indicate
>> what substitution group it is in, I also don't mind consulting the
>> schema to determine this - that's one of the reasons why we have a
>> schema.
>
> I disagree. Substitution groups follow the good coding principle of
> 'Open for extension, but closed for modification'. In addition,
> substitution groups realize the conceptual spec more closely which
> explicitly states that "Grid services requiring a richer set of
> attributes for the [Endpoint|Service], specific models MAY be derived
> by specializing from the [Endpoint|Service] class and adding new
> properties or relationships." Without substitution groups, the XSD
> doesn't support this.
>
>
>> Another thought on the grouping elements is which ones to use? If
>> we use them, I don't necessarily agree that using the substitution
>> groups or base complexTypes is the best approach. It may be logical
>> for us as the schema creators, but it may not be logical to users.
>> For example, as a user, in a single document, I'd like to have
>> compute-related information in one area and storage-related
>> information in another for easier viewing. A different way to put
>> it is that just because two objects are in the same substitution
>> group, that doesn't mean that that is the strongest relationship
>> and they should be grouped together.
>>
>> The above argument can also be applied to the ordering of the
>> elements in the schema - I just noticed that this schema orders
>> elements by substitution group. In the TeraGrid GLUE 2 XML schema,
>> I ordered them in the order they are presented in GFD.147:
>> base/abstract first, then compute, then storage. I think I
>> personally prefer this ordering (because it matches GFD.147 and
>> because it fits the way I look at things), but it doesn't seem
>> critical.
>
> I disagree. For me, the benefits provided using substitution groups
> far outweigh the disadvantages of less strict ordering when ordering
> by substitution group.
>
>> Heh, I honestly find the BaseType attribute more objectionable
>> than the grouping entities. It just seems redundant, like something
>> that would rarely, if ever, be used, and it just looks odd when
>> viewing the XML document.
>
> Regardless of BaseType attribute or grouping entity, I still think it
> is useful to explicitly indicate what is the base type of an element
> within the XML instance doc without having to explicitly refer to the
> XSD; both approaches enable simple XPath style queries on a doc e.g.
> 'select all activities' or 'select all endpoints'. I think this is
> valuable.
>
>
>> Warren
>>
>>
>> -----Original Message----- From: david.meredith at stfc.ac.uk
>> [mailto:david.meredith at stfc.ac.uk] Sent: Monday, September 17, 2012
>> 3:43 AM To: Warren Smith; stephen.burke at stfc.ac.uk;
>> florido.paganelli at hep.lu.se; glue- wg at ogf.org Subject: RE:
>> [glue-wg] flat xml rendering
>>
>> hi Warren, all, (in response to Warren's response, cc'd below):
>>
>> I think that having the ...s grouping elements does not add any
>> complexity and at the same time provides a slightly more
>> structured document; it provides a fixed location/bag for those
>> elements that implement a particular glue2 xsd:substitutionGroup
>> (this is why the grouping elements are not defined universally for
>> all the elements in <Entities> - these other elements are concrete
>> and cannot be substituted for different element implementations).
>>
>> Now, consider new elements that may come along in future glue
>> profiles that also define one of the glue2 xsd:substitutionGroups;
>> looking at an instance document, it is not obvious that a new
>> element (e.g. new activty, new service, new endpoint etc) is a
>> substitute for a particular substitutionGroup without consulting
>> the schema. I think this is also the original reason for the
>> BaseType="..." attribute (which I carried over from the original
>> nested schema); it also serves to clearly indicate the elements
>> base type without having to consult the schema or make any
>> inferences from the elements name (a per your modified XPath
>> below). It is also easy to build XPath to say "select all entities
>> that have BaseType=X" (which is similar to the xpath for selecting
>> all elements of a particular type, e.g. select all Activities
>> '/Entities/Activities/*').
>>
>> Now, I do admit that there is a simple elegance in having all the
>> glue entities as equal siblings within <Entities> (no ...s
>> grouping elements), and there may be a good technical reason to
>> keep all entities as siblings with the same level of nesting....
>>
>> Therefore, at minimum I think it necessary to keep the
>> 'BaseType=...' attributes, but its not really a big deal (at least
>> to me) to keep or drop the '...s' grouping elements, even though I
>> do find it v.useful to be able to collapse/expand the ...s grouping
>> elements when browsing through an instance doc (if all the elements
>> are equal siblings, i can't do this).
>>
>> I'll be giving a talk at the EGI TF on GLUE2 XML renderings and
>> can provide some examples, hopefully we will gain some more
>> insights.
>>
>> cheers, david
>>
>> --------------------------------------------------------------
>> David Meredith STFC eScience Centre Daresbury Laboratory (A32)
>> Warrington, Cheshire WA4 4AD
>>
>> email: david.meredith at stfc.ac.uk tel: +44 (0)1925 603762
>>
>> ________________________________________ From: Warren Smith
>> [wsmith at tacc.utexas.edu] Sent: 06 September 2012 23:50 To:
>> Meredith, David (STFC,DL,SC); Burke, Stephen (STFC,RAL,PPD);
>> florido.paganelli at hep.lu.se; glue-wg at ogf.org Subject: RE: [glue-wg]
>> flat xml rendering
>>
>> Ok, I see what you are saying - I'm just now looking at the schema
>> and I had some assumptions about it that weren't correct. However,
>> after a little reading about XPath, it looks like there are a few
>> ways that you can return all activities without requiring an
>> Activities element as a parent.
>>
>> /Entities/*[contains(local-name(),'Activity')] - returns all
>> elements under Entities that have 'Activity' in them
>>
>> /Entities/*[substring(name(), string-length(name()) - 7) =
>> 'Activity'] - returns all elements under Entities that end in
>> 'Activity'
>>
>> These XPath expressions aren't as nice as
>> '/Entities/Activities/*', but they aren't that bad and I don't know
>> how often this case will come up. In XPath 2.0, it looks like there
>> is an 'ends-with()' function that allows a much simpler version of
>> the 2nd expression above. I personally wouldn't use the additional
>> complexity of these XPath statements as a reason to add *s parent
>> elements.
>>
>>
>> Sorry that I had to miss the last telecon since I was on vacation,
>> so I apologize if some of my comments about the schema were
>> discussed:
>>
>> * My preference is to do away with all ...s grouping elements. I
>> still don't see them as being needed (or desired).
>>
>> * If the decision is to keep the ...s grouping elements (I hope
>> not! :-P ), the schema should probably use them consistently (e.g.
>> wherever the element has maxOccurs="unbounded"). They aren't being
>> used right now in quite a few spots: Location, Contact, the
>> elements at the end of ExtensibleEntities_t, and number of other
>> complexType definitions.
>>
>> * Why is there a BaseType="..." attribute on many of the elements?
>> It seems like redundant information and looking at the schema, I
>> can't figure out why they are needed...
>>
>> * The Extension representation is slightly different from how I
>> defined them in the TeraGrid schema. This schema has elements for
>> LocalID and Key (as well as Value). For the TeraGrid schema, I had
>> an Extension element with Key as an attribute (I forgot to include
>> a LocalID attribute), and the text of the Extension element was
>> the value. This schema mostly avoids attributes, so using elements
>> probably makes more sense...
>>
>>
>> That's all I notice at this point.
>>
>>
>> Warren
>>
>>
>> -----Original Message----- From: david.meredith at stfc.ac.uk
>> [mailto:david.meredith at stfc.ac.uk] Sent: Friday, August 31, 2012
>> 10:00 AM To: Warren Smith; stephen.burke at stfc.ac.uk;
>> florido.paganelli at hep.lu.se; glue- wg at ogf.org Subject: RE:
>> [glue-wg] flat xml rendering
>>
>> Hi Warren, In the glue2_2.xsd at
>> http://redmine.ogf.org/dmsf/glue-wg?folder_id=32, there is no
>> '<ComputingActivites>,' only '<Activities>' which serves to group
>> the elements that are substitutable for the 'AbstractActivity'
>> substitutionGroup (so currently this includes both 'Activity' and
>> 'ComputingActivity' - and of course any activities that may be
>> profiled in the future that may also define the same
>> substitutionGroup). The relevant XSD fragment is as follows (line
>> 139) - note that it simply serves as a wrapper around
>> AbstractActivity:
>>
>> <element name="Activities" minOccurs="0" maxOccurs="1">
>> <complexType> <sequence> <element ref="glue:AbstractActivity"
>> minOccurs="0" maxOccurs="unbounded"/> </sequence> </complexType>
>> </element>
>>
>> Therefore, given the following instance doc:
>>
>> <Entities... > ... <glue:Activities> <glue:Activity
>> BaseType="Activity"> <glue:ID>http://activity1.ac.uk/1</glue:ID>
>> <glue:Associations/> </glue:Activity>
>>
>> <glue:ComputingActivity BaseType="Activity">
>> <glue:ID>computingActivity1</glue:ID> <glue:State>some
>> state</glue:State> <glue:Owner>someone</glue:Owner>
>> <glue:Associations/> </glue:ComputingActivity>
>>
>> ...here nest more elements that can substitute for the
>> AbstractActivity substitutionGroup... </glue:Activities> ...
>> </Entities>
>>
>> To select all of the different flavours of Activity in a single
>> query, you could use the following XPath: '/Entities/Activities/*'.
>> Similarly, for all services; '/Entities/Services/*' and for all
>> Endpoints; '/Entities/Endpoints/*' etc.  I guess it depends if you
>> think this is useful. Without the 'Activities' element, to select
>> all the different types of activity, you would currently need to
>> issue two XPath queries and concat the two docs:
>> '/Entities/Activity' and '/Entities/ComputingActivity'.
>>
>> Personally, I think it is useful to be able to select/collapse all
>> of the Services, Endpoints, Activities etc with one button click
>> or query, especially when considering large documents that may have
>> 1000s of endpoints listed. Either way, it's only a small change
>> from the version that does not define these grouping elements
>> (http://redmine.ogf.org/dmsf/glue-wg?folder_id=31)
>>
>> David
>>
>>> -----Original Message----- From: Warren Smith
>>> [mailto:wsmith at tacc.utexas.edu] Sent: 31 August 2012 14:48 To:
>>> Burke, Stephen (STFC,RAL,PPD); Meredith, David (STFC,DL,SC);
>>> Meredith, David (STFC,DL,SC); florido.paganelli at hep.lu.se;
>>> glue-wg at ogf.org Subject: RE: [glue-wg] flat xml rendering
>>>
>>>
>>> My general opinion is not to have any -ies elements (e.g.
>>> ComputingActivities) to group elements. To me, this grouping of
>>> elements (e.g. ComputingActivity) isn't needed in XML. It
>>> doesn't seem to make it any easier to process the XML using SAX
>>> or DOM. It seems easy to find all -y elements in a document
>>> without a -ies element as a parent for visualizing. Similarly, it
>>> also seems easy to find -y elements in an XML document if you
>>> want to translate it to LDAP or SQL or
>> whatever.
>>>
>>> I haven't really done much xPath... Can someone explain how
>>> grouping elements would help? It seems like either way you'd
>>> include 'ComputingActivity' in the expression. That is the
>>> expression would start with
>>> "/Entities/ComputingActivities/ComputingActivity..." or
>>> "/Entities/ComputingActivity...".
>>>
>>>
>>> Warren
>>>
>>>
>>> -----Original Message----- From: glue-wg-bounces at ogf.org
>>> [mailto:glue-wg-bounces at ogf.org] On Behalf Of
>>> stephen.burke at stfc.ac.uk Sent: Friday, August 31, 2012 5:20 AM
>>> To: david.meredith at stfc.ac.uk; david.meredith at stfc.ac.uk;
>>> florido.paganelli at hep.lu.se; glue-wg at ogf.org Subject: Re:
>>> [glue-wg] flat xml rendering
>>>
>>> glue-wg-bounces at ogf.org [mailto:glue-wg-bounces at ogf.org] On
>>>> Behalf Of david.meredith at stfc.ac.uk said: I think these group
>>>> elements could be useful for simplifying XPath queries and for
>>>> visualizing large documents with tools that can collapse/expand
>>>> elements (see attached image, also uploaded at the following
>>>> link).
>>>
>>> Bear in mind that in the production Grid there are about 4000
>>> endpoints, so grouping may not help a lot with visualisation
>>> unless you do
>> it more finely.
>>>
>>> Stephen
>>>
>>> _______________________________________________ glue-wg mailing
>>> list glue-wg at ogf.org
>>> https://www.ogf.org/mailman/listinfo/glue-wg
>> -- Scanned by iCritical. -- Scanned by iCritical.
> -- Scanned by iCritical.
>

-- 
Florido Paganelli
Lund University - Particle Physics
ARC Middleware
EMI Project