[DFDL-WG] Action 174: Making DFDL implementations easier
Steve Hanson
smh at uk.ibm.com
Fri May 25 12:01:50 EDT 2012
Interesting, but there's a problem - variables are currently an optional
feature! You could swap variables for paths, which is what I think Mike
proposed in an earlier mail?
exprSubset = / | / exprList
atom = . | .. | identifier
exprList = atom | atom / exprList
That's essentially equivalent to what MRM provides for its 'repeat
reference' and 'length reference' properties.
It doesn't handle things like an array count being 0 based, where you need
the ability to add 1 to the result. And you can't do a simple comparison,
which would make it easy to add choices and discriminators to an
implementation. The trouble is that once you add in operators and
literals you've pulled in a lot of the language.
Regards
Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh at uk.ibm.com
tel:+44-1962-815848
From: Tim Kimber/UK/IBM
To: Mike Beckerle <mbeckerle.dfdl at gmail.com>
Cc: dfdl-wg at ogf.org, dfdl-wg-bounces at ogf.org, Steve
Hanson/UK/IBM at IBMGB
Date: 25/05/2012 14:29
Subject: Re: [DFDL-WG] Action 174: Making DFDL implementations
easier
My current thinking is:
- Minimum expression language is a simple variable reference. e.g.
{$myLength} or {$myParentStructureLength}.
- That would require the ability to declare DFDL variables, set them, read
them and put them into and out of scope. And restore their value after
backtracking if backtracking is supported in the implementation.
This would make all the XPath functions optional, and would also make it
unnecessary to use an XPath query engine.
regards,
Tim Kimber, Common Transformation Team,
Hursley, UK
Internet: kimbert at uk.ibm.com
Tel. 01962-816742
Internal tel. 246742
From: Mike Beckerle <mbeckerle.dfdl at gmail.com>
To: Steve Hanson/UK/IBM at IBMGB
Cc: Tim Kimber/UK/IBM at IBMGB, dfdl-wg at ogf.org, dfdl-wg-bounces at ogf.org
Date: 25/05/2012 13:58
Subject: Re: [DFDL-WG] Action 174: Making DFDL implementations
easier
But the expression language is highly restricted in the subset.
On Fri, May 25, 2012 at 7:59 AM, Steve Hanson <smh at uk.ibm.com> wrote:
Tim
I tend to agree with endOfParent as optional.
Expression language was kept in the core to handle occursCountKind
'expression' which is also in the core. The examples of binary data we
have seen from NSA and ESA both have occurs counts in the data, in the
same way as COBOL does. When you have untagged binary data, it's typically
the way to provide an array size. It was felt that dropping this from the
core did left us with too little capability.
Regards
Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh at uk.ibm.com
tel:+44-1962-815848
From: Tim Kimber/UK/IBM
To: Steve Hanson/UK/IBM at IBMGB
Cc: dfdl-wg at ogf.org, dfdl-wg-bounces at ogf.org
Date: 25/05/2012 10:07
Subject: Re: [DFDL-WG] Action 174: Making DFDL implementations
easier
1) I would make endOfParent an optional feature - there are not many
formats that require it.
3) There are many formats that do not require the expression language - it
is only required when a property value or an assert/discriminator needs to
query already-parsed data. On that basis, I think the entire expression
language feature should be optional.
regards,
Tim Kimber, Common Transformation Team,
Hursley, UK
Internet: kimbert at uk.ibm.com
Tel. 01962-816742
Internal tel. 246742
From: Steve Hanson/UK/IBM at IBMGB
To: dfdl-wg at ogf.org
Date: 25/05/2012 00:08
Subject: [DFDL-WG] Action 174: Making DFDL implementations easier
Sent by: dfdl-wg-bounces at ogf.org
Agreed on list, just need to answer questions 1) and 3) below.
Regards
Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh at uk.ibm.com
tel:+44-1962-815848
----- Forwarded by Steve Hanson/UK/IBM on 23/05/2012 18:21 -----
From: Steve Hanson/UK/IBM
To: dfdl-wg at ogf.org
Date: 15/05/2012 09:55
Subject: Making DFDL implementations easier
Please see below for a proposal to make an additional set of DFDL features
optional. The goal is to make it considerably easier to create a minimal
conforming DFDL processor for binary data.
Regards
Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh at uk.ibm.com
tel:+44-1962-815848
----- Forwarded by Steve Hanson/UK/IBM on 11/05/2012 12:50 -----
Feature
Detection
Text representation for types other than String
dfdl:representation="text" for Number, Calendar or Boolean types
Delimiters
dfdl:separator <> "" or dfdl:initiator <> "" or dfdl:terminator <> "" or
dfdl:lengthKind="delimited"
BCD calendars
dfdl:binaryCalendarRep="bcd"
Multiple schemas
xs:include or xs:import in xsd
Named Formats
dfdl:defineFormat or dfdl:ref
Choices
xs:choice in xsd **
Arrays where size not known in advance
dfdl:occursCountKind 'implicit', 'parsed', 'stopValue' **
Advanced expressions
Advanced features of the DFDL expression language (tbd)
** Including one of these features mean that speculative parsing is
needed.
Remaining questions:
1) What about lengthKind 'endOfParent' ?
2) Is leaving out choices too restrictive?
3) Expression language subset
The result is that a minimal conformant DFDL implementation just needs to
support the following annotations and properties, and does not need
speculative parsing.
dfdl:element
dfdl:sequence
dfdl:format
byteOrder
encoding
utf16width
alignment
alignmentUnits (bytes)
fillByte
leadingSkip
trailingSkip
lengthKind (explicit, implicit)
length
lengthUnits (bytes, characters)
representation (binary)
textPadKind
textTrimKind
textStringJustification
textStringPadCharacter
truncateSpecifiedLengthString
decimalSigned
binaryNumberRep
binaryVirtualDecimalPoint
binaryFloatRep (ieee)
binaryBooleanTrueRep
binaryBooleanFalseRep
binaryCalendarRep (binarySeconds, binaryMilliseconds)
binaryCalendarEpoch
sequenceKind (ordered)
occursCountKind (fixed, expression)
occursCount
Regards
Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh at uk.ibm.com
tel:+44-1962-815848
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
--
dfdl-wg mailing list
dfdl-wg at ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
--
dfdl-wg mailing list
dfdl-wg at ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
--
Mike Beckerle | OGF DFDL WG Co-Chair
Tel: 781-330-0412
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ogf.org/pipermail/dfdl-wg/attachments/20120525/039424b6/attachment-0001.html>
More information about the dfdl-wg
mailing list