[DFDL-WG] Fw: Transaction log data types from IBM 4690 point-of-sale terminals - add to DFDL spec?

[Steve Hanson]

DFDL WG agreed to add the new enum to binaryNumberRep and 
binaryCalendarRep. Errata will be taken.

Regards

Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh at uk.ibm.com
tel:+44-1962-815848
----- Forwarded by Steve Hanson/UK/IBM on 18/09/2012 16:43 -----

From:   Steve Hanson/UK/IBM
To:     dfdl-wg at ogf.org
Date:   17/09/2012 15:23
Subject:        Transaction log data types from IBM 4690 point-of-sale 
terminals - add to DFDL spec?


IBM 4690 data format 

IBM 4690 is a widely used retail Point-of-Sale system.  The transactions 
logs emitted by 4690 are in a compact, ASCII delimited binary format, 
known as TLOG. Each TLOG consists of a number of records, called 
'strings'. Each 'string' consists of fields delimited by a colon (x2C), 
although some strings use double quote/colon/double quote instead 
(x22x2Cx22).  The fields themselves can be fixed length or variable length 
but the delimiter is always present. The fields can have a number of 
physical reps but the most frequent rep is a packed number rep that is not 
BCD or S/390 packed decimal, and so DFDL can't model it. It is used to 
represent the vast majority of decimal and integer numbers in TLOG data.
This packed decimal format is best described by the operations that act on 
the data:
Pack: Take an ASCII string of characters, drop the first nibble of each 
byte, shuffle the remaining nibbles to the right, and if there is an odd 
number of nibbles pad with an xF nibble on the left.
Unpack: Discard any leading xF nibble, and expand each nibble into a byte 
by adding a x3 nibble in front, giving an ASCII string.
It can be seen that while the pack operation may be applied to any ASCII 
string, the unpack operation will only round trip if the original 
characters are in the range x30 to x3E, hence the operations were only 
intended to be used for packing and unpacking unsigned numbers. A 
convention has arisen however whereby the minus sign x2D may be used to 
indicate negative values, even though xD unpacks as x3D which is the 
equals sign. In summary:
Nibbles represent digits 0 - 9 in the usual BCD manner
A positive value is simply indicated by digits
A negative number is indicated by digits with the leading nibble being xD
If a positive or negative value packs to an odd number of nibbles, pad on 
the left with xF
Some examples help:
12                        x12
123                      xF1x23
-12                       xFDx12
-123                     xD1x23
Note that x00xD1x23 is not valid as this would unpack to 0=123 (ie, 
0-123). The only nibble that can appear to the left of an xD nibble is an 
xF pad nibble. Or put another way, you can't blindly byte pad with x00 to 
make up a fixed length. If such byte padding is required, replace any xF 
and xD nibbles with x0 nibble, then byte pad with x00, then reinstate any 
xD nibble in the new leftmost nibble.
Proposal to add support to DFDL:
Existing dfdl:binaryNumberRep property gets a new enum 'ibm4690Packed'. It 
applies to all signed and unsigned simple types derived from xsd:decimal 
(both built-in and user-defined). The following DFDL properties are used 
in conjunction:
dfdl:byteOrder (although in practice IBM 4690 is always big-endian)
dfdl:binaryDecimalVirtualPoint
Note that dfdl:binaryPackedSignCodes does not apply - the only negative 
nibble is xD - and dfdl:binaryNumberCheckPolicy has no effect.
Where the DFDL specification provides for general behaviours for 'packed' 
and 'bcd', those behaviours apply also to 'ibm4690Packed'. Specifically:
The same dfdl:lengthKind enums and rules apply.
No rounding when serializing, a value that can't be accommodated is a 
processing error.
If the logical type is unsigned and a negative value is received, it is a 
processing error.
The same should be done for dfdl:binaryCalendarRep to allow TLOG packed 
decimal representation for calendars.


Regards

Steve Hanson
Architect, Data Format Description Language (DFDL)
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh at uk.ibm.com
tel:+44-1962-815848
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ogf.org/pipermail/dfdl-wg/attachments/20120918/e6d31181/attachment-0001.html>