[DFDL-WG] Agenda for OGF DFDL WG call 20 January 2010- 13:00 UK (8:00 ET) - TLOG

Wed Jan 20 05:11:46 CST 2010

Finally had some time to look at the TLOG stuff again (the format emitted 
by IBM 4680 & 4690 POS controllers). This time I've looked at the MRM 
code, & spoken with domain expert David Bennett, to establish the full 
behaviour. Unfortunately there are some issues which mean DFDL 1.0 is not 
capable of handling it.

1. The individual fields are a mixture of ASCII strings, proprietary 
packed strings (rare), proprietary packed decimals, binary integers 
(rare). All fields are delimited by a separator. 

2. The fields are all defined with a length in bytes, but most of the 
string and decimal fields are actually variable length. If the data 
exceeds the length in bytes when parsing or unparsing the MRM throws an 
error. However, talking to David B, the length is really intended to be 
used if there was ever a fixed length equivalent format, so is really for 
validation only. To parse the current TLOG formats it is sufficient to use 
the delimiter. 
Note to WG: Validation using the specified length only works for strings. 
Should we allow dfdl:length to be specified when dfdl:lengthKind=delimited 
or pattern, and use it as an extra constraint when parsing/unparsing?

3. Scanning for the separator, or maybe use of a data pattern, is needed 
to extract the variable length data, including packed decimal data (which 
we would consider a binary type). 
Note to WG: Should we allow binary scanning when the users says it is safe 
to do so?

4. Packed strings. This is a packed data type used when the range of 
possible chars in the string is limited to 0-9, A-F. You can view this as 
a BCD that can also carry A-F. MRM does not try to turn this into an 
integer, instead it treats it as character data. That is x12x34 would 
result in string '1234'. Odd numbers of digits are padded with a x0 
nibble. On unparsing the MRM throws an error if a character other than 
0-9, A-F is encountered.  In practice, these packed strings are rare and 
invariably only carry 0-9 anyway. I'm still trying to establish why this 
data type is needed, as it could be treated as a logical integer and 
modelled as BCD.

5. Packed decimals. Like a packed decimal in the IBM sense. These can 
carry negative numbers but use a leading xD sign nibble. No sign nibble if 
positive. Odd number of digits (including sign if present) are padded with 
xF nibble. This is best illustrated using examples. 
1234     =>     x12x34
123      =>     xF1x23
-1234    =>     xFDx12x34
-123     =>     xD1x23
Note to WG: Should we support this data type natively?

6. Some of the packed decimals are interpreted as an array of bit flags. I 
assume that DFDL would model these using dfdl:hidden.

Regards

Steve Hanson
Programming Model Architect, WebSphere Message  Brokers,
OGF DFDL WG Co-Chair,
Hursley, UK,
Internet: smh at uk.ibm.com,
Phone (+44)/(0) 1962-815848

From:
Alan Powell/UK/IBM at IBMGB
To:
dfdl-wg at ogf.org
Date:
19/01/2010 18:40
Subject:
[DFDL-WG] Agenda for OGF DFDL WG call 20 January        2010-   13:00   UK 
(8:00 ET)
Sent by:
dfdl-wg-bounces at ogf.org

1.   045  - Disciminators 
Mike was writing up proposal 

2  Unparsing lengthKind = 'pattern' 
Review email from Alan 

3. TLOG 
SteveH has been investigating 

4, Empty Sequences 
Is the current section correct 

1.1        Empty Sequences 
A sequence having no children is syntactically legal in DFDL; however, a 
sequence having no children must have content length zero. It can still 
have non-zero length Prefix and Suffix regions, but the SequenceContent 
region in between must be of length zero. It is a schema definition error 
if the SequenceContent region of an empty sequence is not length zero. 

5. Mike review issues 
Attached document gives replies to comments/issues. Ones without replies 
need to be discussed 

6. Go through remaining actions 
7 Review Schedule 
Activity

Schedule
Who
Complete Action items 

             - 18 Dec 2009 
 WG 
Complete Spec 
Write up work items 
            – 23 Dec 2009 
AP 
Restructure and complete specification 
              - 23 Dec 2009 
AP 
Issue Draft 038 
23 Dec 2009

WG review 
WG review 
7 Dec – 08 Jan 2010 
WG 
Incorporate review comments 
4 Jan - 29 Jan 2010 
AP + 
Issue Draft 039 
15 Jan 2010

Incorporate review comments 
4 Jan - 29 Jan 2010 
AP + 
Issue Draft 040 
29 Jan 2010

Initial OGF Editor Review 
Initial Editor review 
1 Feb - 1 Mar 2010 
OGF 
Initial GFSG review 
1 Feb - 1 Mar 2010

Issue Draft 041 
1 Mar 2010

OGF Public Comment period (60 days) 

1 Mar - 30 Apr 2010 
OGF 
OGF 28 Munich 

15-19 March 2010 

Incorporate comments 
Incorporate comments 
28 May 2010

Issue Draft 042 
28 May 2010

Final OGF Editor Review 
Final  Editor review 
June  2010 
OGF 
final GFSG review 
June  2010

Issue Final specification 
30 June 2010

Publish proposed recommendation 

1 July 2010

Grid recommendation process 

1 Jan - 1 April 2011

Current Actions: 
No
Action 

045
20/05 AP: Speculative Parsing 
27/05: Psuedo code has been circulated. Review for next call 
03/06: Comments received and will be incorporated 
09/06: Progress but not discussed 
17/06: Discussed briefly 
24/06: No Progress 
01/07: No Progress 
15/07: No progress. MB not happy with the way the algorithm is documented, 
need to find a better way. 
29/07: No Progress 
05/08: No Progress. Will document behaviour as a set of rules. 
12/08: No Progress 
... 
16/09: no progress 
30/09: AP distributed proposal and others commented. Brief discussion AP 
to incorporate update and reissue 
07/10: Updated proposal was discussed.Comments will be incorporated into 
the next version. 
14/10: Alan to update proposal to include array scenario where minOccurs > 
0 
21/10: Updated proposal reviewed 
28/10: Updated proposal reviewed see minutes 
04/11: Discussed semantics of disciminators on arrays. MB to produce 
examples 
11/11: Absorbing action 033 into 045.  Maybe decorated discrminator kinds 
are needed after all. MB and SF to continue with examples.   
18/11: Went through WTX implementation of example. SF to gather more 
documentation about WTX discriminator rules. 
25/11: Further discussion. Will get more WTX documentation. Need to 
confirm that no changes need to Resolving Uncertainty doc. 
04/11: Further discussion about arrays. 
09/12: Reviewed proposed discriminator semantic. 
16/12: Reviewed discriminator examples and WTX semantic. 
23/12: SF to provide better description of WTX behaviour and invite B 
Connolley to next call 
06/01:B Connolly not available. SF to provide more complete description. 
13/01: Stephaine took us through a description of WTX identifiers. Mike 
agreed to write up in DFDL terms. 
049
20/05 AP Built-in specification description and schemas 
03/06: not discussed 
24/06: No Progress 
24/06: No Progress (hope to get these from test cases) 
15/07: No progress. Once available, the examples in the spec should use 
the dfdl:defineFormat annotations they provide. 
... 
14/10: no progress 
21/10: Discussed the real need for this being in the specification. It 
seemed that the main value is it define a schema location for downloading 
'known' defaults from the web. 
28/10: no progress 
04/11: no progress 
11/11: no update 
18/11: no update 
25/11: Agreed to try to produce for CSV and fixed formats 
04/12: no update 
09/12: no update 
16/12: no update 
23/12: no update 
06/01: no progress. If there is no resource to complete this action it can 
be deferred 
13/01:no progess 
064
MB/SH Request WG presentation at OGF 28 
25/11: Session requested 
04/12: no update 
09/12: no update 
16/12: SH has changed request to a general session rather tha WG in the 
hope of attracting more people. 
23/12: no update 
06/01: not heard anything yet 
13/01: no update 
066
Investigate format for defining test cases 
25/11:IBM to see if it is possible to publish its test case format. 
04/12: no update 
09/12: no update 
16/12: reminded dent to project manager 
23/12: SH will send another reminder. 
06/01: Another reminder will be sent 
13/01: no update 
068
Should the roots of messages be designated.? 
09/12: Yes. New dfdl:documentRoot property 
Closed 
16/12: reopened and decided to drop property subject to agreement from SKK 
and SF 
23/12: SKK review decision to drop  dfdl:documentRoot 
13/01: closed 
071
Semantics of length=0, nil handling and defaults. 
23/12:SH no update 
06/01: SH has started 
13/01: SH proposal review. Minor updates to be made 
073
SH: Control of overpunching zoned positive sign 
13/01: no update 

Alan Powell

MP 211, IBM UK Labs, Hursley,  Winchester, SO21 2JN, England
Notes Id: Alan Powell/UK/IBM     email: alan_powell at uk.ibm.com 
Tel: +44 (0)1962 815073                  Fax: +44 (0)1962 816898

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU 

--
  dfdl-wg mailing list
  dfdl-wg at ogf.org
  http://www.ogf.org/mailman/listinfo/dfdl-wg

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ogf.org/pipermail/dfdl-wg/attachments/20100120/ff82ec94/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ogf-dfdl-v1.0-Core-037.final.MB.Comments.doc
Type: application/octet-stream
Size: 66048 bytes
Desc: not available
Url : http://www.ogf.org/pipermail/dfdl-wg/attachments/20100120/ff82ec94/attachment-0001.obj 