[DFDL-WG] Backtracing behavior for optional elements
Steve Hanson
smh at uk.ibm.com
Tue Feb 12 05:20:41 EST 2013
Glad that we are all in agreement.
The particular use case that motivated me to write down the rules is the
IBM 4690 TLOG format. This is a binary, separated, positional format so it
is modeled with dfdl:separatorPolicy 'trailingEmptyLax'. However I
encountered two records that each contained a pair of unbounded arrays.
The end of the first is indicated by the next field having value x95 -
x99, the end of the second is encountering the end of record delimiter.
The separator does not change throughout. I was struggling to model this
correctly until Tim pointed out that although the rest of the record was
positional, the arrays were non-positional. Wrapping the arrays in a
sequence with dfdl:separatorPolicy 'anyEmpty' solved the problem.
I think this would be a good subject for a DFDL tutorial lesson.
Regards
Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh at uk.ibm.com
tel:+44-1962-815848
From: Tim Kimber/UK/IBM at IBMGB
To: dfdl-wg at ogf.org,
Date: 11/02/2013 19:21
Subject: Re: [DFDL-WG] Backtracing behavior for optional elements
Sent by: dfdl-wg-bounces at ogf.org
I agree with all of Steve's description, and all of Mike's response. And I
still think that in an ideal world we would include in the specification a
set of grammars that describe the various 'styles' of group, including
groups with no separator, positional separators and non-positional
separators.
regards,
Tim Kimber, DFDL Team,
Hursley, UK
Internet: kimbert at uk.ibm.com
Tel. 01962-816742
Internal tel. 37246742
From: Mike Beckerle <mbeckerle.dfdl at gmail.com>
To: Steve Hanson/UK/IBM at IBMGB,
Cc: dfdl-wg at ogf.org
Date: 11/02/2013 18:16
Subject: Re: [DFDL-WG] Backtracing behavior for optional elements
Sent by: dfdl-wg-bounces at ogf.org
This sounds right.
Let me run an array scenario past you. Tell me if you think I am
interpreting consistently with your rules.
What you've said here is that we distinguish positional and non-positional
separators. They are very different.
Positional separators are greedy and drive the parser decision. Once
matched, they no longer tolerate failure to parse. So, if I have an array
with occursCountKind='parsed', then finding a positional separator means I
am NOT at the end of the physical array. I will have syntax for one more
element to be parsed successfully, though I may suppress its value being
added to the infoset if it is optional and I get the appropriate empty
representation after the separator. Failure means the array is broken.
Success means I will look for yet another element (because this is ock
parsed).
The above makes sense to me. This is what 'separators' means to me for the
most part, that they are a driving part of the syntax/format.
The non-positional separators case is 100% different.
In that case, the decision that a separator was found is revisited on
failure. An ock='parsed' array/optional will be ended. The thing after it
in the sequence will be attempted next.
This makes sense, I almost wish we didn't have to call it 'separator', but
I think it is a useful behavior certainly, and the right interpretation of
the properties we have in the spec and 140 stuff today.
On Mon, Feb 11, 2013 at 9:56 AM, Steve Hanson <smh at uk.ibm.com> wrote:
If a processing error occurs for an optional element in a sequence, the
speculative behaviour of the DFDL parser says that the optional element is
assumed not to be present, and the next alternative in the sequence is
tried. That is fine when there are no separators involved, but we need to
clear on what happens when there are separators.
1) Positional separators (separatorSuppressionPolicy is 'never',
'trailingEmpty' and 'trailingEmptyStrict').
The key point about positional separators is that they are expected in the
data, so if an error occurs while parsing the optional element, it does
not make sense to backtrack to the start offset the element and try to
match the next element. Yes there's a point of uncertainty in the sense
that the element is either there or it has empty representation, but if an
error occurs I think it must be treated as a hard error, and not cause
backtracking.
2) Non-positional separators (separatorSuppressionPolicy is 'anyEmpty').
This behaves like the non-separator case and the next alternative in the
sequence is tried from the start offset. However, because 'anyEmpty'
behavior is lax, it is possible that the next thing in the data is a
separator, so the parser must cater for that when the element is found to
have empty representation. But if an error occurs establishing
representation, I think the parser should just backtrack and try to match
the next element.
Does that sound correct?
Regards
Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh at uk.ibm.com
tel:+44-1962-815848
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
--
dfdl-wg mailing list
dfdl-wg at ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
--
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
www.tresys.com
--
dfdl-wg mailing list
dfdl-wg at ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
--
dfdl-wg mailing list
dfdl-wg at ogf.org
https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ogf.org/pipermail/dfdl-wg/attachments/20130212/3cd47f6d/attachment-0001.html>
More information about the dfdl-wg
mailing list