[DFDL-WG] Fw: dfdl:sequence and other specialized annotations

Wed Mar 18 07:21:56 CDT 2009

Any thoughts on this? Note one correction below.

----------------------------------------------------

We discussed this on the call yesterday as minuted by Alan. 

I missed xsd:any from the table in the mail below, so that's added.

Specialized annotations may help alleviate something that has been noted 
by folk at IBM when creating DFDL schemas.  If using dfdl:format on 
complex types to set properties, it is usually to default values for the 
elements of the complex type. But the first thing following the complex 
type is the model group (sequence, choice) so any properties with the same 
name have to be explicitly set.  That's all working as designed but it is 
a common mistake to forget to do this. The cause is having properties with 
the same name on both sequence and element.  Having specialized 
annotations provides a way round this by allowing the introduction of 
'parallel' scoping. 

<xs:annotation><xs:appinfo source=”http://www.ogf.org/dfdl/”>
  <dfdl:defineFormat>
    <dfdl:sequence terminator="$" ... />
    <dfdl:element terminator="%" ... />
    <dfdl:simpleType ... />
    <dfdl:choice ... />
    <dfdl:any ... />
  </dfdl:defineFormat>
</xs:appinfo></xs:annotation>

<xs:complexType name="myType">
    <xs:annotation><xs:appinfo source=”http://www.ogf.org/dfdl/”>
      <dfdl:complexType>
        <dfdl:sequence lengthKind="implicit" ... />
        <dfdl:element lengthKind="delimited" ... />
      </dfdl:complexType>
    </xs:appinfo></xs:annotation>
  <xs:sequence>
    <xs:annotation><xs:appinfo source=”http://www.ogf.org/dfdl/”>
      <dfdl:sequence ... />
    </xs:appinfo></xs:annotation>
    <xs:element name="el1" type="xs:string">
    <xs:annotation><xs:appinfo source=”http://www.ogf.org/dfdl/”>
      <dfdl:element ... />
    </xs:appinfo></xs:annotation>
    </xs:element>
    <xs:element name="el2" type="xs:string">
    <xs:annotation><xs:appinfo source=”http://www.ogf.org/dfdl/”>
      <dfdl:element ... />
    </xs:appinfo></xs:annotation>
    </xs:element>
    <xs:element name="el3" type="xs:string">
    <xs:annotation><xs:appinfo source=”http://www.ogf.org/dfdl/”>
      <dfdl:element ... />
    </xs:appinfo></xs:annotation>
    </xs:element>
  </xs:sequence>
</xs:complexType>

This makes it very clear which properties are going to be used where and 
allows separate defaults to be set up for different kinds of xsd object. 
But there are a couple of problems with this. 

1) What does this mean?

<xs:complexType name="myType" dfdl:lengthKind="implicit" ... >

Possible answers:
a) We don't allow short form on xs:complexType.
b) Short form applies to all DFDL specializations
c) Short form applies to one specific DFDL specialization, eg, 
dfdl:element.

Personally I think a) would be a shame and a loss of convenience; c) is 
too arbitrary; b) looks the most sensible, but opens up another question, 
namely whether this equivalent is also allowed.

<xs:complexType name="myType">
    <xs:annotation><xs:appinfo source=”http://www.ogf.org/dfdl/”>
      <dfdl:complexType lengthKind="delimited">
        <dfdl:sequence lengthKind="implicit" ... />
        <dfdl:element lengthKind="delimited" ... />
      </dfdl:complexType>
    </xs:appinfo></xs:annotation>

2) I am forced to set common properties in multiple places. 

<xs:annotation><xs:appinfo source=”http://www.ogf.org/dfdl/”>
  <dfdl:defineFormat>
    <dfdl:sequence encoding="UTF-8" ... />
    <dfdl:element encoding="UTF-8" ... />
    <dfdl:simpleType encoding="UTF-8" ... />
    <dfdl:choice encoding="UTF-8" ... />
    <dfdl:any encoding="UTF-8" ... />
  </dfdl:defineFormat>
</xs:appinfo></xs:annotation>

But we could also allow this, and say that dfdl:format applied to all xsd 
objects.  Note that this is consistent with 1) b) above,

<xs:annotation><xs:appinfo source=”http://www.ogf.org/dfdl/”>
  <dfdl:defineFormat>
    <dfdl:format encoding="UTF-8" ... />
    <dfdl:sequence ... />
    <dfdl:element ... />
    <dfdl:simpleType ... />
    <dfdl:choice ... />
    <dfdl:any ... />
  </dfdl:defineFormat>
</xs:appinfo></xs:annotation>

We should also bear in mind when deciding all this that we do already 
provide one other facility which provides a form of parallelism.

<dfdl:format ref=”reusableDef"... />

So I can already create a set of annotations applicable to a specific kind 
of xsd object and reuse that via the dfdl:ref attribute.

Regards

Steve Hanson
Programming Model Architect
WebSphere Message Brokers
Hursley, UK
Internet: smh at uk.ibm.com
Phone (+44)/(0) 1962-815848
----- Forwarded by Steve Hanson/UK/IBM on 12/03/2009 10:00 -----

Steve Hanson/UK/IBM 
09/03/2009 16:10

To
DFDL Working Group
cc

Subject
Fw: [DFDL-WG] dfdl:sequence and other specialized annotations

Although it was minuted in last week's call that we decided to drop 
specialised annotations, I don't actually recall that was decision on the 
call.

The proposal below is in accordance with the suggestion by Martin, and 
echoed by Dave, that we have only specialised annotations for xsd objects.

Schema object
DFDL annotation allowed
Notes
Schema declaration 
 dfdl:format
Only within an enclosing dfd:defineFormat. 
Element declaration
 dfdl:element
Just those properties allowed on elements
Element reference
 dfdl:element
Just those properties allowed on elements.
Overrides dfdl:element properties on global element
Complex type definition
 dfdl:complexType
Full dfdl:format properties.
For scoping purposes only.
Simple type definition
 dfdl:simpleType
Just those properties allowed on simple types.
Sequence declaration
 dfdl:sequence
Just those properties allowed on sequences.
Choice declaration
 dfdl:choice
Just those properties allowed on choices.
Group reference
 dfdl:group
Just those properties allowed on sequence/choice.
Overrides dfdl:sequence/dfdl:choice properties on global group 
sequence/choice model group
Wildcard element
dfdl:any
Just those properties allowed on wildcard elementsv

An alternative as I previously wrote is to allow dfdl:sequence and 
dfdl:choice on a group reference. 

Group reference (to sequence group)
 dfdl:sequence
Just those properties allowed on sequence/choice.
Overrides dfdl:sequence properties on global group model sequence group
Group reference (to choice group)
dfdl:choice
Just those properties allowed on sequence/choice.
Overrides dfdl:choice properties on global group model choice group

Although this does not look as good visually, it is worth bearing in mind 
section 22.2 of the spec:

22.2 Group Reference
A copy of the definition referenced by a group reference is substituted 
inline for its declaration while combining any DFDL format annotation 
elements associated with it as given in Section ‎10.5. (TBD: xref to 
scoping?)  Processing then proceeds as for an anonymous group declaration.

Note:
The use of specialised annotations relies on the removal of something that 
we had talked about but not yet reflected in the spec - dfdl:format on 
global xsd:group definitions. The idea of allowing this came about fairly 
recently when we changed the rules for hidden objects.  I was unhappy that 
embedded hidden xsd objects within DFDL annotations would make them 
invisible to a schema validator, so dfdl:hidden was changed to be a 
reference to a global group containing the hidden objects. I suggested 
that we could also put scoping annotations on global groups to support 
scoping within the hidden construct, it seemed a good idea at the time, 
but it was not fully explored. Thinking it through, because the global 
group effectively disappears when the reference is resolved, then so does 
its annotations. I'm therefore happy to retract scoping annotations on 
global groups if it makes specialised annotations work. 

Regards

Steve Hanson
Programming Model Architect
WebSphere Message Brokers
Hursley, UK
Internet: smh at uk.ibm.com
Phone (+44)/(0) 1962-815848

From:
Steve Hanson/UK/IBM at IBMGB
To:
Suman Kalia <kalia at ca.ibm.com>
Cc:
dfdl-wg at ogf.org
Date:
02/03/2009 23:45
Subject:
Re: [DFDL-WG] Reducing the number of DFDL properties.

Suman 

A DFDL annotation on  a group reference overrides a DFDL annotation on the 
content of the model group (ie, sequence, choice), not a DFDL annotation 
on the global group.  It allows you to override the sequence/choice 
properties at point of use.  The DFDL annotation on a global group is like 
that on a complex type - it is a scoping construct only. 

Regards

Steve Hanson
Programming Model Architect
WebSphere Message Brokers
Hursley, UK
Internet: smh at uk.ibm.com
Phone (+44)/(0) 1962-815848 

Suman Kalia <kalia at ca.ibm.com> 
02/03/2009 22:02 

To
Steve Hanson/UK/IBM at IBMGB 
cc
dfdl-wg at ogf.org, "Dave Glick" <dglick at dracorp.com> 
Subject
Re: [DFDL-WG] Reducing the number of DFDL properties.

I can see where you are coming from and I agree with most of your proposal 
except putting dfdl:sequence, dfdl:choice , dfdl:all on group references 
as it does not give the right optics.. From consistency point of view, the 
references (group and element references) should have the equivalent/same 
set of annotations that can appear on the referenced artifacts.   

Suman Kalia
IBM Toronto Lab
WMB Toolkit Architect and Development Lead
WebSphere Business Integration Application Connectivity Tools 

http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.html

Tel : 905-413-3923  T/L  969-3923
Fax : 905-413-4850 T/L  969-4850
Internet ID : kalia at ca.ibm.com 

From: 
Steve Hanson <smh at uk.ibm.com> 
To: 
Suman Kalia/Toronto/IBM at IBMCA 
Cc: 
dfdl-wg at ogf.org, "Dave Glick" <dglick at dracorp.com> 
Date: 
03/02/2009 12:45 PM 
Subject: 
Re: [DFDL-WG] Reducing the number of DFDL properties.

If I recall, one reason for having the specialised elements was so that 
the content of the annotations could be more tightly validated by the 
schema-for-dfdl xsd. 

However, the problem with the specialised annotations is that we do not 
provide them for all circumstances. For example, the spec allows me to use 
dfdl:sequence on an xs:sequence but not an xs:group reference to a global 
xs:group with compostion xs:sequence.  Nor does it allow me to use a 
specialised element on xs:simpleType or xs:restriction.  So as it stands, 
validation would not be fool-proof anyway. 

I think we must either ditch specialisations, or mandate the use of 
specialisations on the corresponding schema objects. The latter is more 
workable now we have changed the scoping rules so that scoping of dfdl 
properties can only be done from complex type or schema level, as below. 
But it does mean that dfdl annotations are less flexible. 

dfdl;defineFormat  =>  dfdl:format 
xs:complexType => dfdl:format 
xs:group => dfdl:format 
xs:sequence or xs:group ref to a global group with xs:sequence => 
dfdl:sequence 
xs:choice or xs:group ref to a global group with xs:choice  => dfdl:choice 

xs:element or xs:element ref => dfdl:element 
xs:any => dfdl:any 
xs:simpleType, xs:restriction => dfdl:simpleType 

In other words, you use dfdl:format when you are scoping properties, you 
must use specialised annotation for specific objects. 

Regards

Steve Hanson
Programming Model Architect
WebSphere Message Brokers
Hursley, UK
Internet: smh at uk.ibm.com
Phone (+44)/(0) 1962-815848 
Suman Kalia <kalia at ca.ibm.com> 
Sent by: dfdl-wg-bounces at ogf.org 
02/03/2009 16:25 

To
"Dave Glick" <dglick at dracorp.com> 
cc
dfdl-wg at ogf.org, dfdl-wg-bounces at ogf.org 
Subject
Re: [DFDL-WG] Reducing the number of DFDL properties.

I am not sure what kind of confusion or redundancy is caused by these 
specialized annotations.  In the absence of these specialized annotations, 
you will have to go through plethora of  annotations and determine which 
ones are applicable for sequence , choice, all, elements etc.. 

The danger is we don't have these levels of abstractions, then number of 
folks (specifically implementers) would build them anyway and then we will 
have to contend with incompatible abstractions.. 

Suman Kalia
IBM Toronto Lab
WMB Toolkit Architect and Development Lead
WebSphere Business Integration Application Connectivity Tools 

http://www.ibm.com/developerworks/websphere/zones/businessintegration/wmb.html

Tel : 905-413-3923  T/L  969-3923
Fax : 905-413-4850 T/L  969-4850
Internet ID : kalia at ca.ibm.com 
From: 
"Dave Glick" <dglick at dracorp.com> 
To: 
"Alan Powell" <alan_powell at uk.ibm.com>, <mbeckerle.dfdl at gmail.com> 
Cc: 
dfdl-wg at ogf.org 
Date: 
03/02/2009 11:16 AM 
Subject: 
Re: [DFDL-WG] Reducing the number of DFDL properties.

Alan/Mike, 

I agree with this – one of my complaints with v033 was that these 
specialized annotation elements just added confusion and redundancy. 

Dave 

---
David Glick  |  dglick at dracorp.com  |  703.299.0700 x212
Data Research and Analysis Corp.  |  www.dracorp.com 

From: dfdl-wg-bounces at ogf.org [mailto:dfdl-wg-bounces at ogf.org] On Behalf 
Of Alan Powell
Sent: Monday, March 02, 2009 11:03 AM
To: mbeckerle.dfdl at gmail.com
Cc: dfdl-wg at ogf.org
Subject: Re: [DFDL-WG] Reducing the number of DFDL properties. 

Mike 

Thanks 

Can I also suggest dropping the dfdl:sequence, dfdl:choice, dfdl:element 
and dfdl:any specialized annotations and just have dfdl:format. 

Alan Powell

MP 211, IBM UK Labs, Hursley,  Winchester, SO21 2JN, England
Notes Id: Alan Powell/UK/IBM     email: alan_powell at uk.ibm.com 
Tel: +44 (0)1962 815073                  Fax: +44 (0)1962 816898 

From: 
"Mike Beckerle" <mbeckerle.dfdl at gmail.com> 
To: 
Alan Powell/UK/IBM at IBMGB 
Cc: 
<dfdl-wg at ogf.org> 
Date: 
28/02/2009 19:38 
Subject: 
RE: Reducing the number of DFDL properties.

If a property is redundant, I'm in favor of dropping it. 

If a property adds generality that we don't have a use case for, I'm in 
favor of dropping it. 

I am happy to drop type substitution from v1.0. It's a convenience that 
can be achieved a different way. 

E.g., if you really want "float" to mean "float in my particular binary 
representation", then just put a type definition with DFDL annotations in 
a different namespace, and when you write your DFDL schema, arrange for 
the default namespace to pick it up from your namespace, and not xs:float. 

<xs:element name="myElement" type="float"/> <!-- float here is 
myNamespace:float which can have DFDL annotations on it --> 
<xs:element name="anotherElem" type="xs:float"/> <!-- explicitly xs:float 
without further adornment --> 

If you want the XSD unadorned "float" type, be explicit and use 
"xs:float". Voila - no loss of flexibility, equal textual convenience. 

I think that would satisfy the community that really wanted very compact 
"slideware acceptable" schemas. This is the same group that wants 
short-form annotations as well. 
Mike Beckerle | OGF DFDL WG Co-Chair | CTO | Oco, Inc.
Tel:  781-810-2100  | 100 Fifth Ave., 4th Floor, Waltham MA 02451 | 
mbeckerle.dfdl at gmail.com 

From: Alan Powell [mailto:alan_powell at uk.ibm.com] 
Sent: Thursday, February 26, 2009 12:22 PM
To: mbeckerle.dfdl at gmail.com
Cc: dfdl-wg at ogf.org
Subject: Reducing the number of DFDL properties.

Mike 

A number of people at IBM have become concerned at the number of 
properties in DFDL and have identified a number of 'usability' properties 
that could be dropped. They feel that we should be simplifying the 
properties wherever possible and not introducing multiple ways of doing 
the same function without very good reason. 

The following are offered for consideration. 
1.        lengthKind='nullterminated'
This is just shorthand for lengthKind=delimited and terminator='%Null'. It 
was felt this this is not even the most common terminator so why have a 
special case? 
2.        trimKind
It is felt that there aren't any cases when you would want to pad but not 
trim and vice versa so make padKind control both. 
3.        typeSubstitution.
Is this needed in DFDL v1? 

Can you consider these before the call next week 

Alan Powell

MP 211, IBM UK Labs, Hursley,  Winchester, SO21 2JN, England
Notes Id: Alan Powell/UK/IBM     email: alan_powell at uk.ibm.com 
Tel: +44 (0)1962 815073                  Fax: +44 (0)1962 816898 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU 

--
dfdl-wg mailing list
dfdl-wg at ogf.org
http://www.ogf.org/mailman/listinfo/dfdl-wg --
dfdl-wg mailing list
dfdl-wg at ogf.org
http://www.ogf.org/mailman/listinfo/dfdl-wg 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU 

--
  dfdl-wg mailing list
  dfdl-wg at ogf.org
  http://www.ogf.org/mailman/listinfo/dfdl-wg

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ogf.org/pipermail/dfdl-wg/attachments/20090318/ed2c10fc/attachment-0001.html 