[DFDL-WG] Action 304: Data Streaming (was) Re: Review/Feedback wanted on proposal: Data Streaming for base64 and other Layered Transformations

[Mike Beckerle]

The data layering feature previously proposed has been implemented in
Daffodil.

There were minor changes from prior Wiki description. The wiki page is
updated to reflect what the implementation actually does. The examples on
the wiki page actually run and have been incorporated into Daffodil
regression tests.

https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+Data+Layering+for+base64

This feature is also being successfully used to create iCalendar and IMF
format DFDL schemas. Those are not yet released for public consumption, but
the feature is being successfully used for "real" formats and is thusfar,
working as designed.

Of the changes since prior draft of the design note, We changed a property
value name. Property dfdl:layerLengthKind='boundaryMark' was done to
replace the 'terminator' property value to avoid confusion with ordinary
dfdl:terminator property.

Thusfar we've not run into a need for a more general mechanism for passing
parameters from the schema to the layering. The boundary mark, and the
length have been sufficient, but this feels like an area where something
more general may be needed in the future. For example, there are really 3
different kinds of layerings all known as "base64". Rather than having one
general "base64" with a parameter, we're currently requiring that each
variant have its own layering transform name, e.g., "base64_MIME" is one of
them (the only one implemented thusfar).

...mikeb

Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF Intellectual Property Policy
<http://www.ogf.org/About/abt_policies.php>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ogf.org/pipermail/dfdl-wg/attachments/20180515/b576da3d/attachment.html>