[DFDL-WG] Action 233 (deferred) - "byte order not sufficient..." - draft document on experience with binary format MIL-STD-2045

[Steve Hanson]

Mike

Please find attached IBM's initial comments to your experience document, 
as Word comments.  We only got as far as the 3 x required extensions, not 
looked at the optional usability stuff in detail yet.

We think we have our collective heads around the least significant bit 
ordering concept, but we think the explanation could be clearer and show 
the bits on-the-wire. Some debate as to whether this could be considered 
some variation of byteOrder but you've obviously thought this through and 
concluded a separate property is best. Also should bit order apply to text 
reps, given that byteOrder is binary rep only and any byte ordering 
variations in encodings are handled as separate encodings (eg, UTF-16LE 
and UTF-16BE).

Regarding the US-ASCII-7-Bit-Packed encoding enum, this was added via 
erratum previously using the idea of DFDL-specific named encoding. But we 
are thinking that this could have been handled as an x- encoding, rather 
than specifically adding it to the spec.  And thinking further on that 
same thread, should byteOrder be made to work like encoding and allow x- 
enums, then the new byteOrder would become a x- enum.  The Wikipedia 
article you cite on Endianness mentions other byte orders (eg, 
Middle-Endian, PDP-Endian).



Regards
 
Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh at uk.ibm.com
tel:+44-1962-815848



From:   Mike Beckerle <mbeckerle.dfdl at gmail.com>
To:     "dfdl-wg at ogf.org" <dfdl-wg at ogf.org>, 
Date:   24/06/2014 20:27
Subject:        [DFDL-WG] Action 233 (deferred) - "byte order not 
sufficient..." - draft document on experience with binary format 
MIL-STD-2045
Sent by:        dfdl-wg-bounces at ogf.org



I have created an experience document about the "bit order" issue, which 
was a deferred action 233, and the subject of a public comment.

The document is here: http://redmine.ogf.org/dmsf_files/13268. The public 
comment item is http://redmine.ogf.org/boards/15/topics/43.

It recommends a new dfdl:bitOrder property, and a new dfdl:byteOrder enum 
value, without which it is impossible to model these data formats. It also 
recommends  several other improvements to DFDL to facilitate handling 
these data formats. 

The formats in question are a variety of MIL-STD formats which are all 
densely packed binary data. These formats are in broad use. MIL-STD-2045 
is one part of this family and this particular format specification is 
generally available without any restrictions from a US DoD web site (
http://assistdocs.com) so I made this specific format the subject of the 
document as it illustrates all the problematic issues.

We have implemented the dfdl:bitOrder property in Daffodil, and it works 
with some useful tests now passing. 

We have also enhanced our TDML implementation to enable creation of tests 
for this feature (and in the process actually found two bugs in the 
MIL-STD-2045 spec!). 

Both the property and this TDML enhancement are described in the document.

The sponsors of the Daffodil project are extremely keen to get this needed 
binary support into the DFDL v1.0 standard so as to have multiple DFDL 
implementations support it. 

...mikeb

Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | 
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are 
subject to the OGF Intellectual Property Policy
--
  dfdl-wg mailing list
  dfdl-wg at ogf.org
  https://www.ogf.org/mailman/listinfo/dfdl-wg

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ogf.org/pipermail/dfdl-wg/attachments/20140708/49279c1a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: draft-gwdi-mil-std-2045-v1-IBM.docx
Type: application/octet-stream
Size: 206196 bytes
Desc: not available
URL: <http://www.ogf.org/pipermail/dfdl-wg/attachments/20140708/49279c1a/attachment-0001.obj>