[DFDL-WG] OCK expression and count of 0 for a choice member....

Alex Wood1 WOODA at uk.ibm.com
Fri Apr 24 09:07:28 EDT 2015 

Hi All,

Please see below for a history of the issue.
This arose from fuzz testing of the IBM DFDL parser which produced a test 
with a coutn of 0 for  an OCK expression array which was a choice member. 
And subsequent reference to the specification.

It was not clear what the correct outcome should be in a choice where the 
first member is an array using OCK expression where the count resolves to 
0.
a.) resolve the choice to the zero length array
b.) move to the next choice branch
c.) throw an error 


Kind Regards,

- Alex

Alex Wood - 
Software Engineer - 
WebSphere Message Broker Development
DFDL Development

MP 211, IBM UK Labs, Hursley Park, Winchester, Hants. SO21 2JN.
Tel: Internal 246272, External 01962 816272
Notes: Alex Wood1/UK/IBM at IBMGB
e-mail: wooda at uk.ibm.com




From:   Steve Hanson/UK/IBM
To:     Alex Wood1/UK/IBM at IBMGB
Cc:     Andrew Edwards/UK/IBM at IBMGB, Mark Frost/UK/IBM
Date:   24/04/2015 09:19
Subject:        Re: OCK expression and count of 0 for a choice member....


When I wrote the paragraph below, the one thing that troubled me was that 
the spec defines known-to-exist and known-not-to-exist in terms of 
occurrences. In the choice branch example, it is the element as a whole we 
are looking at. That's fine for scalar as element == occurrence but for an 
array it's not the same.  I think the spec is missing a definition of what 
'missing' means for an array element. I would say that an array element is 
missing if all occurrences are missing. And an array element is not 
missing if any occurrence has a representation (empty, nil, normal).  With 
that in place, my paragraph makes sense, I think.

I believe we have the same issue with 'parsed' and 'stopValue'.

Regards
 
Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh at uk.ibm.com
tel:+44-1962-815848




From:   Steve Hanson/UK/IBM
To:     Alex Wood1/UK/IBM at IBMGB
Cc:     Andrew Edwards/UK/IBM at IBMGB, Mark Frost/UK/IBM at IBMGB
Date:   23/04/2015 18:52
Subject:        Re: OCK expression and count of 0 for a choice member....


Here is one interpretation...

A choice is resolved by parsing the branches until one is known-to-exist 
as described in section 9.3.3.  Section 9.3.1.2 defines known-to-exist (in 
the absence of a discriminator, initiator or direct dispatch) as an 
occurrence having empty, nil or normal representation. Section 9.3.1.3 
defines known-not-to-exist (again in the absence of a discriminator, 
initiator or direct dispatchm or an assert) as an occurrence being missing 
or causing a processing error. If occursCount is zero no occurrences are 
looked for in the data (erratum 5.9) so the element has no representation 
and must be missing. Therefore a choice branch containing such an element 
is known-not-to-exist. 

So in your example, the first choice branch containing myInt is 
known-not-to-exist and the parser tries the next branch. 

This appears to contradict section 15.1.1 though. I suspect that 15.1.1 
was not updated to match section 9.3 when the latter was added.

If you want to make the first choice branch known-to-exist when the count 
is zero then I think wrapping myInt in a sequence would work. Or wrapping 
myInt in a complex element. 

Definitely one to take to the WG though, if only to correct section 15.1.1 
to match section 9.

Regards
 
Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh at uk.ibm.com
tel:+44-1962-815848




From:   Alex Wood1/UK/IBM
To:     Steve Hanson/UK/IBM at IBMGB
Cc:     Andrew Edwards/UK/IBM at IBMGB, Mark Frost/UK/IBM at IBMGB
Date:   23/04/2015 16:33
Subject:        OCK expression and count of 0 for a choice member....


Hi Steve

Just been discussing this with Andy and Mark.
I think the spec 

<xs:element name="Choice_Expression" dfdl:ref="config" 
dfdl:lengthKind="implicit">
   <xs:complexType>
      <xs:sequence dfdl:ref="config">
             <xs:element ref="myCount"></xs:element>
                <xs:choice dfdl:choiceLengthKind="implicit" 
dfdl:ref="config">  
                    <xs:element ref="myInt" minOccurs="1" 
maxOccurs="3"></xs:element>
                        <xs:element ref="myTxt"></xs:element>
                </xs:choice>
      </xs:sequence>
   </xs:complexType>
</xs:element>

Where myInt  has occursCountKind="expression" occursCount="{../myCount}"

A given instance of this message could have myCount==0

Is this valid?
Should it resolve to 0 occurrences of myInt or move on to myTxt ?

Section15 of the spec says:

The Root of the Branch MUST NOT be optional. That is XSDL minOccurs MUST 
BE greater than 0. 

But in this case minOccurs is >0.

Assuming this is not an error then in terms of resolving the choice 
section 15.1.1 says.. 

15.1.1 Resolving Choices via Speculation Speculative resolution works as 
follows: 
1) Attempt to parse the first branch of the choice. 
2) If this fails with a processing error 
a) If a dfdl:discriminator evaluated to true earlier on this branch then 
the parser is 'bound' to this branch and parsing of the entire choice 
construct fails with a processing error. 
b) If the branch has a dfdl:initiator and the choice has 
dfdl:initiatedContent ‘yes’ then the parser is 'bound' to this branch and 
parsing of the entire choice construct fails with a processing error. c) 
Otherwise we repeat from step 1 for the next branch of the choice. 
3) It is a processing error if the branches of the choice are exhausted. 
4) If a branch is successfully parsed without error, then that branch's 
infoset becomes the infoset for the parse of the choice construct. 

So seems like this is 4.) we did not fail to parse myInt... 

However talking with mark about real scenarios that this might apply to, a 
choice two repeating fields with counts earlier in the data only one of 
which must appear. you'd expect 0 of the first means >0 of the second and 
visa versa... So you'd probably want 0 myInt allowed the choice to resolve 
to myTxt.

Thoughts ?

If you agree we need more clarity in he spec will forward to WG. 


Kind Regards,

- Alex

Alex Wood - 
Software Engineer - 
WebSphere Message Broker Development
DFDL Development

MP 211, IBM UK Labs, Hursley Park, Winchester, Hants. SO21 2JN.
Tel: Internal 246272, External 01962 816272
Notes: Alex Wood1/UK/IBM at IBMGB
e-mail: wooda at uk.ibm.com


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ogf.org/pipermail/dfdl-wg/attachments/20150424/006e2d16/attachment-0001.html>

More information about the dfdl-wg mailing list