[DFDL-WG] Action 286 - Daffodil v IBM DFDL feature comparison

Wed Aug 3 05:18:29 EDT 2016

Action 286 closed.

286
Create table that compares Daffodil features to IBM DFDL features 
(Steve/Mike)
16/2: Take table from section 22 of DFDL 1.0 spec and add columns showing 
implementations.
1/3: No progress
10/5: No progress. Work on this for next call.
...
7/6: No further progress
5/7: Steve has added column for IBM DFDL and emailed to WG. Mike will add 
column for Daffodil.
2/8: Closed. Mike has added column and discussion for Daffodil and emailed 
to WG. Steve to convert to an experience document and add to Redmine @ 
https://redmine.ogf.org/dmsf_files/13539?download=. Document can be 
updated as required.

Regards

Steve Hanson
IBM Integration Bus, Hursley, UK
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
smh at uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890

From:        Mike Beckerle <mbeckerle.dfdl at gmail.com> 
To:        Steve Hanson/UK/IBM at IBMGB 
Cc:        DFDL-WG <dfdl-wg at ogf.org> 
Date:        19/07/2016 20:13 
Subject:        Re: [DFDL-WG] Action 286 - Daffodil v IBM DFDL feature 
comparison 

Table with Daffodil column added, and additional details below. 
Feature 
Detection 
IBM DFDL 
Daffodil v2.0 
Validation 
External switch 
Yes 
partial. Parsing only. 
Named Formats 
dfdl:defineFormat or dfdl:ref 
Yes 
Yes 
Choices 
xs:choice in xsd 
Yes 
Partial - choiceDispatchKey and choiceBranchKey not supported. 
Arrays where size not known in advance 
dfdl:occursCountKind 'implicit', 'parsed', 'stopValue' 
Partial - not 'stopValue' 
Partial – not ‘stopValue’ 
Expressions 
Use of a DFDL expression in any property value 
Yes 
Yes. Limitation - query-style expressions not supported. These are 
expressions which require iteration over children of an element rather 
than constant-time access.  For example fn:count(/a/b) where element 'a' 
has multiple element 'b' children separated by other element declarations. 
Hence, gathering all the b children requires iterating. 
End of parent 
dfdl:lengthKind "endOfParent" 
No 
No 
Simple type restrictions 
xs:simpleType in xsd 
Partial - not unions 
Partial – not unions 
Text representation for types other than String 
dfdl:representation "text" for Number, Calendar or Boolean types 
Yes 
Partial – type Boolean not implemented 
Delimiters 
dfdl:separator <> "" or dfdl:initiator <> "" or dfdl:terminator <> "" or 
dfdl:lengthKind "delimited" 
Yes 
Yes 
Nils 
XSDL nillable 'true' in xsd 
Yes 
Partial - nilKind 'logicalValue' not supported 
Defaults 
XSDL default or fixed in xsd 
Partial - unparsing only 
Partial
Simple types - ?? (Some support in parsing and in unparsing)
Complex types – No 
Bi-Directional text. 
dfdl:textBiDi 'yes' 
No 
No 
Lengths in Bits 
dfdl:alignmentUnits 'bits' or dfdl:lengthUnits 'bits' 
Yes 
Yes 
Delimited lengths and representation binary element 
dfdl:representation 'binary' (or implied binary) and dfdl:lengthKind 
'delimited' 
Yes 
Partial – delimiters must not use character class entities like %NL or 
%WSP*. 
Regular expressions 
dfdl:lengthKind 'pattern', 
dfdl:assert with dfdl:testkind 'pattern' , 
dfdl:discriminator with dfdl:testkind 'pattern' 
Partial - only for dfdl:lengthKind 
Yes 
Zoned numbers 
dfdl:textNumberRep 'zoned' 
Yes 
No 
IBM 390 packed numbers 
dfdl:binaryNumberRep 'packed'   
Yes 
No 
IBM 390 packed calendars 
dfdl:binaryCalendarRep 'packed' 
No 
No 
IBM 390 floats 
dfdl:binaryFloatRep 'ibm390Hex' 
Yes 
No 
Unordered sequences 
dfdl:sequenceKind 'unordered' 
Yes 
No 
Floating elements 
dfdl:floating 'yes' 
No 
No 
dfdl functions in expression language 
dfdl:functions in expression 
No 
Yes 
Hidden groups 
dfdl:hiddenGroupRef <> '' 
No 
Yes 
Calculated values 
dfdl:inputValueCalc <> '' or dfdl:outputValueCalc <> '' 
No 
Yes 
Escape schemes 
dfd:defineEscapeScheme in xsd 
Yes 
Yes 
Extended encodings 
Any dfdl:encoding value beyond the core list 
Yes 
Partial - iso-8859-1 works. Others - TBD. 
Asserts 
dfdl:assert in xsd 
Partial - not on global elements or simple types 
Yes 
Discriminators 
dfdl:discriminator in xsd 
Partial - not on global elements or simple types 
Yes 
Prefixed lengths 
dfdl:lengthKind 'prefixed' 
Partial - not nested prefixed lengths 
No 
Variables 
dfdl:defineVariable, 
dfdl:newVariableInstances, 
dfdl:setVariable 
Variables in DFDL expression language 
Note that variables as a feature is dependent on the Expressions feature. 
Partial - not newVariableinstance or external variables 
Partial – not newVariableInstance 
BCD calendars 
dfdl:binaryCalendarRep "bcd"   
Yes 
No 
BCD numbers 
dfdl:binaryNumberRep "bcd" 
Yes 
No 
Multiple schemas 
xs:include or xs:import in xsd 
Yes 
Yes 
IBM 4690 packed numbers 
dfdl:binaryNumberRep "ibm4690Packed" 
Yes 
No 
IBM 4690 packed calendars 
dfdl:binaryCalendarRep "ibm4690Packed" 
Yes 
No 
DFDL Byte Value Entities 
Use of %#r syntax in a DFDL String Literal other than the dfdl:fillByte 
property 
Yes 
No 
DFDL Standard Character Set Encodings 
dfdl:encoding name begins with "X-DFDL-". 
No 
Partial: X-DFDL-US-ASCII-7-BIT-PACKED, but not the 6-bit variant. 
Bit Order - Least Significant Bit First 
dfdl:bitOrder with value 'leastSignificantBitFirst' 
No 
Yes

There is a list of implementation specific things not implemented for 
Daffodil also. The list needs updating, but includes at least:

* xs:boolean - not supported 
* dfdl:ignoreCase - not supported 
* binary form for xs:date, xs:time, and xs:dateTime are not supported. 
* dfdl:utf16Width='variable' not supported
* extended ICU symbols 'u' and 'I' in calendarPattern - not supported 
* dfdl:textStandardBase - not supported (behaves as 10) 

W.r.t, the additional you listed for IBM: 
fn:count(), fn:exactlyOne(), fn:name(), fn:local-name(), 
fn:namespace-uri()
Daffodil has fn:count(), fn:exactlyOne(), but not the other 3. 
BOMs
Daffodil does not yet support BOMs 
Correct distinction between 'empty' and 'missing' when parsing (action 
140)
No known limitation here - I would be curious as to a test that 
illustrates this limitation so we could try it on Daffodil 
Use of namespaces in expression paths
Daffodil supports prefixed QNames in path steps. However, Daffodil's 
interpretation of NCNames in path steps does NOT currently take any 
default namespace into account. This makes Daffodil incompatible with IBM 
DFDL, which does not allow QNames, and so depends on those names being 
made meaningful by way of default namespace. Making Daffodil compatible 
with IBM's behavior here is Daffodil JIRA ticket 
https://opensource.ncsa.illinois.edu/jira/browse/DFDL-1446 
dfdl:separatorSuppressionPolicy "trailingEmptyStrict" 
Daffodil supports this property value. 

Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | 
www.tresys.com 
Please note: Contributions to the DFDL Workgroup's email discussions are 
subject to the OGF Intellectual Property Policy 

On Mon, Jul 4, 2016 at 11:49 AM, Steve Hanson <smh at uk.ibm.com> wrote: 
286
Create table that compares Daffodil features to IBM DFDL features 
(Steve/Mike) 
16/2: Take table from section 22 of DFDL 1.0 spec and add columns showing 
implementations. 
1/3: No progress 
10/5: No progress. Work on this for next call. 
... 
7/6: No further progress 
5/7:
To kick this off here is the optional features table from spec section 21 
with a column added for IBM DFDL 

Feature 
Detection 
IBM DFDL 
Validation 
External switch 
Yes 
Named Formats 
dfdl:defineFormat or dfdl:ref 
Yes 
Choices 
xs:choice in xsd 
Yes 
Arrays where size not known in advance 
dfdl:occursCountKind 'implicit', 'parsed', 'stopValue' 
Partial - not 'stopValue' 
Expressions 
Use of a DFDL expression in any property value 
Yes 
End of parent 
dfdl:lengthKind "endOfParent" 
No 
Simple type restrictions 
xs:simpleType in xsd 
Partial - not unions 
Text representation for types other than String 
dfdl:representation "text" for Number, Calendar or Boolean types 
Yes 
Delimiters 
dfdl:separator <> "" or dfdl:initiator <> "" or dfdl:terminator <> "" or 
dfdl:lengthKind "delimited" 
Yes 
Nils 
XSDL nillable 'true' in xsd 
Yes 
Defaults 
XSDL default or fixed in xsd 
Partial - unparsing only 
Bi-Directional text. 
dfdl:textBiDi 'yes' 
No 
Lengths in Bits 
dfdl:alignmentUnits 'bits' or dfdl:lengthUnits 'bits' 
Yes 
Delimited lengths and representation binary element 
dfdl:representation 'binary' (or implied binary) and dfdl:lengthKind 
'delimited' 
Yes 
Regular expressions 
dfdl:lengthKind 'pattern', 
dfdl:assert with dfdl:testkind 'pattern' , 
dfdl:discriminator with dfdl:testkind 'pattern' 
Partial - only for dfdl:lengthKind 
Zoned numbers 
dfdl:textNumberRep 'zoned' 
Yes 
IBM 390 packed numbers 
dfdl:binaryNumberRep 'packed'   
Yes 
IBM 390 packed calendars 
dfdl:binaryCalendarRep 'packed' 
No 
IBM 390 floats 
dfdl:binaryFloatRep 'ibm390Hex' 
Yes 
Unordered sequences 
dfdl:sequenceKind 'unordered' 
Yes 
Floating elements 
dfdl:floating 'yes' 
No 
dfdl functions in expression language 
dfdl:functions in expression 
No 
Hidden groups 
dfdl:hiddenGroupRef <> '' 
No 
Calculated values 
dfdl:inputValueCalc <> '' or dfdl:outputValueCalc <> '' 
No 
Escape schemes 
dfd:defineEscapeScheme in xsd 
Yes 
Extended encodings 
Any dfdl:encoding value beyond the core list 
Yes 
Asserts 
dfdl:assert in xsd 
Partial - not on global elements or simple types 
Discriminators 
dfdl:discriminator in xsd 
Partial - not on global elements or simple types 
Prefixed lengths 
dfdl:lengthKind 'prefixed' 
Partial - not nested prefixed lengths 
Variables 
dfdl:defineVariable, 
dfdl:newVariableInstances, 
dfdl:setVariable 
Variables in DFDL expression language 
Note that variables as a feature is dependent on the Expressions feature. 
Partial - not newVariableinstance or external variables 
BCD calendars 
dfdl:binaryCalendarRep "bcd"   
Yes 
BCD numbers 
dfdl:binaryNumberRep "bcd" 
Yes 
Multiple schemas 
xs:include or xs:import in xsd 
Yes 
IBM 4690 packed numbers 
dfdl:binaryNumberRep "ibm4690Packed" 
Yes 
IBM 4690 packed calendars 
dfdl:binaryCalendarRep "ibm4690Packed" 
Yes 
DFDL Byte Value Entities 
Use of %#r syntax in a DFDL String Literal other than the dfdl:fillByte 
property 
Yes 
DFDL Standard Character Set Encodings 
dfdl:encoding name begins with "X-DFDL-". 
No 
Bit Order - Least Significant Bit First 
dfdl:bitOrder with value 'leastSignificantBitFirst' 
No

Some IBM DFDL restrictions are not captured in the table, mainly due to 
non-implementation of many of the later errata. Full details here 
https://www.ibm.com/support/knowledgecenter/SSMKHH_10.0.0/com.ibm.etools.mft.doc/df00150_.htm 
but the key omissions are: 
fn:count(), fn:exactlyOne(), fn:name(), fn:local-name(), 
fn:namespace-uri() 
BOMs 
Correct distinction between 'empty' and 'missing' when parsing (action 
140) 
Use of namespaces in expression paths 
dfdl:separatorSuppressionPolicy "trailingEmptyStrict" 

Regards

Steve Hanson
IBM Integration Bus, Hursley, UK
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
smh at uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU 

--
  dfdl-wg mailing list
  dfdl-wg at ogf.org
  https://www.ogf.org/mailman/listinfo/dfdl-wg 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ogf.org/pipermail/dfdl-wg/attachments/20160803/15f3b1f3/attachment-0001.html>