[DFDL-WG] Action 286 - Daffodil v IBM DFDL feature comparison

Tue Jul 19 15:12:30 EDT 2016

Table with Daffodil column added, and additional details below.

*Feature * *Detection * *IBM DFDL * Daffodil v2.0
Validation External switch Yes partial. Parsing only.
Named Formats dfdl:defineFormat or dfdl:ref Yes Yes
Choices xs:choice in xsd Yes Partial - choiceDispatchKey and
choiceBranchKey not supported.
Arrays where size not known in advance dfdl:occursCountKind 'implicit',
'parsed', 'stopValue' Partial - not 'stopValue' Partial – not ‘stopValue’
Expressions Use of a DFDL expression in any property value Yes Yes.
Limitation - query-style expressions not supported. These are expressions
which require iteration over children of an element rather than
constant-time access.  For example fn:count(/a/b) where element 'a' has
multiple element 'b' children separated by other element declarations.
Hence, gathering all the b children requires iterating.
End of parent dfdl:lengthKind "endOfParent" No No
Simple type restrictions xs:simpleType in xsd Partial - not unions Partial
– not unions
Text representation for types other than String dfdl:representation "text"
for Number, Calendar or Boolean types Yes Partial – type Boolean not
implemented
Delimiters dfdl:separator <> "" or dfdl:initiator <> "" or dfdl:terminator
<> "" or dfdl:lengthKind "delimited" Yes Yes
Nils XSDL nillable 'true' in xsd Yes Partial - nilKind 'logicalValue' not
supported
Defaults XSDL default or fixed in xsd Partial - unparsing only Partial
Simple types - ?? (Some support in parsing and in unparsing)
Complex types – No
Bi-Directional text. dfdl:textBiDi 'yes' No No
Lengths in Bits dfdl:alignmentUnits 'bits' or dfdl:lengthUnits 'bits' Yes
Yes
Delimited lengths and representation binary element dfdl:representation
'binary' (or implied binary) and dfdl:lengthKind 'delimited' Yes Partial –
delimiters must not use character class entities like %NL or %WSP*.
Regular expressions dfdl:lengthKind 'pattern',
dfdl:assert with dfdl:testkind 'pattern' ,
dfdl:discriminator with dfdl:testkind 'pattern' Partial - only for
dfdl:lengthKind Yes
Zoned numbers dfdl:textNumberRep 'zoned' Yes No
IBM 390 packed numbers dfdl:binaryNumberRep 'packed'   Yes No
IBM 390 packed calendars dfdl:binaryCalendarRep 'packed' No No
IBM 390 floats dfdl:binaryFloatRep 'ibm390Hex' Yes No
Unordered sequences dfdl:sequenceKind 'unordered' Yes No
Floating elements dfdl:floating 'yes' No No
dfdl functions in expression language dfdl:functions in expression No Yes
Hidden groups dfdl:hiddenGroupRef <> '' No Yes
Calculated values dfdl:inputValueCalc <> '' or dfdl:outputValueCalc <> '' No
Yes
Escape schemes dfd:defineEscapeScheme in xsd Yes Yes
Extended encodings Any dfdl:encoding value beyond the core list Yes Partial
- iso-8859-1 works. Others - TBD.
Asserts dfdl:assert in xsd Partial - not on global elements or simple types
Yes
Discriminators dfdl:discriminator in xsd Partial - not on global elements
or simple types Yes
Prefixed lengths dfdl:lengthKind 'prefixed' Partial - not nested prefixed
lengths No
Variables dfdl:defineVariable,
dfdl:newVariableInstances,
dfdl:setVariable
Variables in DFDL expression language
Note that variables as a feature is dependent on the Expressions
feature. Partial
- not newVariableinstance or external variables Partial – not
newVariableInstance
BCD calendars dfdl:binaryCalendarRep "bcd"   Yes No
BCD numbers dfdl:binaryNumberRep "bcd" Yes No
Multiple schemas xs:include or xs:import in xsd Yes Yes
IBM 4690 packed numbers dfdl:binaryNumberRep "ibm4690Packed" Yes No
IBM 4690 packed calendars dfdl:binaryCalendarRep "ibm4690Packed" Yes No
DFDL Byte Value Entities Use of %#r syntax in a DFDL String Literal other
than the dfdl:fillByte property Yes No
DFDL Standard Character Set Encodings dfdl:encoding name begins with
"X-DFDL-". No Partial: X-DFDL-US-ASCII-7-BIT-PACKED, but not the 6-bit
variant.
Bit Order - Least Significant Bit First dfdl:bitOrder with value
'leastSignificantBitFirst' No Yes
There is a list of implementation specific things not implemented for
Daffodil also. The list needs updating, but includes at least:

* xs:boolean - not supported
* dfdl:ignoreCase - not supported
* binary form for xs:date, xs:time, and xs:dateTime are not supported.
* dfdl:utf16Width='variable' not supported
* extended ICU symbols 'u' and 'I' in calendarPattern - not supported
* dfdl:textStandardBase - not supported (behaves as 10)

W.r.t, the additional you listed for IBM:

   - fn:count(), fn:exactlyOne(), fn:name(), fn:local-name(),
   fn:namespace-uri()

Daffodil has fn:count(), fn:exactlyOne(), but not the other 3.

   - BOMs

Daffodil does not yet support BOMs

   - Correct distinction between 'empty' and 'missing' when parsing (action
   140)

No known limitation here - I would be curious as to a test that illustrates
this limitation so we could try it on Daffodil

   - Use of namespaces in expression paths

Daffodil supports prefixed QNames in path steps. However, Daffodil's
interpretation of NCNames in path steps does NOT currently take any default
namespace into account. This makes Daffodil incompatible with IBM DFDL,
which does not allow QNames, and so depends on those names being made
meaningful by way of default namespace. Making Daffodil compatible with
IBM's behavior here is Daffodil JIRA ticket
https://opensource.ncsa.illinois.edu/jira/browse/DFDL-1446

   - dfdl:separatorSuppressionPolicy "trailingEmptyStrict"

Daffodil supports this property value.

Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF Intellectual Property Policy
<http://www.ogf.org/About/abt_policies.php>

On Mon, Jul 4, 2016 at 11:49 AM, Steve Hanson <smh at uk.ibm.com> wrote:

> *286*
> *Create table that compares Daffodil features to IBM DFDL features
> (Steve/Mike)*
> 16/2: Take table from section 22 of DFDL 1.0 spec and add columns showing
> implementations.
> 1/3: No progress
> 10/5: No progress. Work on this for next call.
> ...
> 7/6: No further progress
> 5/7:
>
> To kick this off here is the optional features table from spec section 21
> with a column added for IBM DFDL
>
> *Feature* *Detection* *IBM DFDL*
> Validation External switch Yes
> Named Formats dfdl:defineFormat or dfdl:ref Yes
> Choices xs:choice in xsd Yes
> Arrays where size not known in advance dfdl:occursCountKind 'implicit',
> 'parsed', 'stopValue' Partial - not 'stopValue'
> Expressions Use of a DFDL expression in any property value Yes
> End of parent dfdl:lengthKind "endOfParent" No
> Simple type restrictions xs:simpleType in xsd Partial - not unions
> Text representation for types other than String dfdl:representation
> "text" for Number, Calendar or Boolean types Yes
> Delimiters dfdl:separator <> "" or dfdl:initiator <> "" or
> dfdl:terminator <> "" or dfdl:lengthKind "delimited" Yes
> Nils XSDL nillable 'true' in xsd Yes
> Defaults XSDL default or fixed in xsd Partial - unparsing only
> Bi-Directional text. dfdl:textBiDi 'yes' No
> Lengths in Bits dfdl:alignmentUnits 'bits' or dfdl:lengthUnits 'bits' Yes
> Delimited lengths and representation binary element dfdl:representation
> 'binary' (or implied binary) and dfdl:lengthKind 'delimited' Yes
> Regular expressions dfdl:lengthKind 'pattern',
> dfdl:assert with dfdl:testkind 'pattern' ,
> dfdl:discriminator with dfdl:testkind 'pattern' Partial - only for
> dfdl:lengthKind
> Zoned numbers dfdl:textNumberRep 'zoned' Yes
> IBM 390 packed numbers dfdl:binaryNumberRep 'packed'   Yes
> IBM 390 packed calendars dfdl:binaryCalendarRep 'packed' No
> IBM 390 floats dfdl:binaryFloatRep 'ibm390Hex' Yes
> Unordered sequences dfdl:sequenceKind 'unordered' Yes
> Floating elements dfdl:floating 'yes' No
> dfdl functions in expression language dfdl:functions in expression No
> Hidden groups dfdl:hiddenGroupRef <> '' No
> Calculated values dfdl:inputValueCalc <> '' or dfdl:outputValueCalc <> ''
> No
> Escape schemes dfd:defineEscapeScheme in xsd Yes
> Extended encodings Any dfdl:encoding value beyond the core list Yes
> Asserts dfdl:assert in xsd Partial - not on global elements or simple
> types
> Discriminators dfdl:discriminator in xsd Partial - not on global elements
> or simple types
> Prefixed lengths dfdl:lengthKind 'prefixed' Partial - not nested prefixed
> lengths
> Variables dfdl:defineVariable,
> dfdl:newVariableInstances,
> dfdl:setVariable
> Variables in DFDL expression language
> Note that variables as a feature is dependent on the Expressions feature. Partial
> - not newVariableinstance or external variables
> BCD calendars dfdl:binaryCalendarRep "bcd"   Yes
> BCD numbers dfdl:binaryNumberRep "bcd" Yes
> Multiple schemas xs:include or xs:import in xsd Yes
> IBM 4690 packed numbers dfdl:binaryNumberRep "ibm4690Packed" Yes
> IBM 4690 packed calendars dfdl:binaryCalendarRep "ibm4690Packed" Yes
> DFDL Byte Value Entities Use of %#r syntax in a DFDL String Literal other
> than the dfdl:fillByte property Yes
> DFDL Standard Character Set Encodings dfdl:encoding name begins with
> "X-DFDL-". No
> Bit Order - Least Significant Bit First dfdl:bitOrder with value
> 'leastSignificantBitFirst' No
>
> Some IBM DFDL restrictions are not captured in the table, mainly due to
> non-implementation of many of the later errata. Full details here
> https://www.ibm.com/support/knowledgecenter/SSMKHH_10.0.0/com.ibm.etools.mft.doc/df00150_.htm
> but the key omissions are:
>
>    - fn:count(), fn:exactlyOne(), fn:name(), fn:local-name(),
>    fn:namespace-uri()
>    - BOMs
>    - Correct distinction between 'empty' and 'missing' when parsing
>    (action 140)
>    - Use of namespaces in expression paths
>    - dfdl:separatorSuppressionPolicy "trailingEmptyStrict"
>
>
>
> Regards
>
> Steve Hanson
> *IBM Integration Bus*
> <http://www-03.ibm.com/software/products/en/ibm-integration-bus>,
> Hursley, UK
> Architect, *IBM DFDL*
> <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
> Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/>
> *smh at uk.ibm.com* <smh at uk.ibm.com>
> tel:+44-1962-815848
> mob:+44-7717-378890
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>
>
> --
>   dfdl-wg mailing list
>   dfdl-wg at ogf.org
>   https://www.ogf.org/mailman/listinfo/dfdl-wg
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ogf.org/pipermail/dfdl-wg/attachments/20160719/c0717b78/attachment-0001.html>