[DFDL-WG] Fw: DFDL functions testOn/Off and setOn/Off - action 107 - updated
Steve Hanson
smh at uk.ibm.com
Mon Sep 20 11:48:01 CDT 2010
Two corrections:
a) Use correct prefix for XPath integer() function
b) Can't use dfdl:outputValueCalc (or any other property) with
dfdl:inputValueCalc
Regards
Steve Hanson
Strategy, Common Transformation & DFDL
Co-Chair, OGF DFDL WG
IBM SWG, Hursley, UK,
smh at uk.ibm.com,
tel +44-(0)1962-815848
----- Forwarded by Steve Hanson/UK/IBM on 20/09/2010 17:29 -----
From:
Alan Powell/UK/IBM
To:
Steve Hanson/UK/IBM at IBMGB
Date:
20/09/2010 15:57
Subject:
Re: Fw: DFDL functions testOn/Off and setOn/Off - action 107
Steve
The spec doesn't allow inputValueCalc and outputValueCalc on the same
element.
Regards
Alan Powell
Development - MQSeries, Message Broker, ESB
IBM Software Group, Application and Integration Middleware Software
-------------------------------------------------------------------------------------------------------------------------------------------
IBM
MP211, Hursley Park
Hursley, SO21 2JN
United Kingdom
Phone: +44-1962-815073
e-mail: alan_powell at uk.ibm.com
----- Forwarded by Steve Hanson/UK/IBM on 20/09/2010 11:19 -----
From:
Steve Hanson/UK/IBM
To:
dfdl-wg at ogf.org
Date:
20/09/2010 11:18
Subject:
DFDL functions testOn/Off and setOn/Off - action 107
The expression language section of the spec (21.5.3). defines some
functions that make it easier to test/set individual bits in a byte.
Action 107 was raised to question whether they were needed and if so to
complete the definitions. In particular the spec for setOn/setOff seems
odd.
Here's an example of where they are useful, although upon investigation
the functions need changing a bit.
A financial standard exists called ISO8583. It is widely used with
debit/credit card transactions. An ISO8583 data stream consists of 128
optional fixed length fields with no tags or delimiters, preceded by a
mandatory bitmap that indicates the presence of all the other fields.
In DFDL a nice approach is to model each optional field as having
dfdl:occursCountKind="expression" with dfdl:occursCount referencing the
corresponding bit in the bitmap and casting its value to an integer to
give 0 or 1. Each bit in the bitmap would have its output value set using
dfdl:outputValueCalc, true if the corresponding field existed in the
infoset.
The representation of the bitmap is the interesting part. It can either be
packed or unpacked.
Packed
In packed format the bitmap is a real array of bits (8 bytes worth) which
must be modelled as individual booleans (ie, not an array) because each
needs to carry a distinct dfdl:outputValueCalc expression, as shown below.
For example:
<complexType name="ISO8583_Packed">
<sequence>
<element name="BitMap">
<complexType>
<sequence>
<!-- each bit in the bitmap modelled individually -->
<element name="Bit001" type="boolean"
dfdl:representation="binary"
dfdl:binaryBooleanTrueRep="1" dfdl:binaryBooleanFalseRep="0"
dfdl:lengthKind="explicit" dfdl:length="1"
dfdl:lengthUnits="bits"
dfdl:outputValueCalc="{if fn:exists(../Field001) then
true else false}" />
...
<element name="Bit064" type="boolean"
dfdl:representation="binary"
dfdl:binaryBooleanTrueRep="1" dfdl:binaryBooleanFalseRep="0"
dfdl:lengthKind="explicit" dfdl:length="1"
dfdl:lengthUnits="bits"
dfdl:outputValueCalc="{if fn:exists(../Field064) then
true else false}" />
</sequence>
</complexType>
</element>
<!-- each field below determines its optional occurrence based on a bit
in the bitmap field -->
<element name="Field001" type="string" minOccurs="0"
dfdl:occursKind="expression"
dfdl:occurs="{xs:integer(../BitMap/Bit001)}" />
...
<element name="Field064" type="string" minOccurs="0"
dfdl:occursKind="expression"
dfdl:occurs="{xs:integer(../BitMap/Bit064)}" />
</sequence>
</complexType>
Unpacked
However, in the unpacked format the value of each nibble of the bitmap is
taken and unpacked as text (ASCII or EBCDIC). Eg, bitmap
'0000000100100011010001010110011110001001101010111100110111101111' (64
bits) = x01x23x45x67x89xABxCDxEF (8 bytes) =
x30x31x32x33x34x35x36x37x38x39x41x42x43x44x45x46 (16 bytes) in ASCII. This
can be modelled as an unsigned long with representation as a base 16
standard text number.
The problem is then how to test and set the individual bits in the
unsigned long. That's where the DFDL functions come in useful, because
there are no XPath functions for bitwise operations.
If the functions are defined as follows:
xs:boolean dfdl:testBit(xs:unsignedLong, xs:unsignedShort)
Returns Boolean true if the bit number given by arg #2 is set on
in the integer given by arg #1, otherwise returns Boolean false.
xs:unsignedLong dfdl:setBits(xs:boolean+)
Returns an integer being the value of the bit positions provided
by the Boolean arguments, where true=1, false=0. The # of args must be a
multiple of 8.
Note that the bit numbering goes from left to right, in accordance with
section 12.3.7.2 of the spec.
Applying this to the example above gives:
<complexType name="ISO8583_Unpacked">
<sequence>
<element name="BitMapInt" type="unsignedLong"
dfdl:representation="text" dfdl:textNumberRep="standard"
dfdl:textNumberBase="16" dfdl:encoding="ascii"
dfdl:lengthKind="explicit" dfdl:length="16"
dfdl:lengthUnits="characters"
dfdl:outputValueCalc="{dfdl:setBits(fn:exists(../Field001),
...
fn:exists(../Field064)}" />
<!-- each field below determines its optional occurrence based on a bit
in the bitmap field -->
<element name="Field001" type="string" minOccurs="0"
dfdl:occursKind="expression"
dfdl:occurs="{xs:integer(dfdl:testBit(../BitMapInt, 1)}" />
...
<element name="Field064" type="string" minOccurs="0"
dfdl:occursKind="expression"
dfdl:occurs="{xs:integer(dfdl:testBit(../BitMapInt, 64)}" />
</sequence>
</complexType>
To keep the unpacked model as close to the packed model as possible, we
can incorporate the logical BitMap from the packed model and hide
BitMapInt. This means the Fieldxxx elements are the same for each.
<complexType name="ISO8583_Unpacked">
<sequence>
<sequence dfdl:hiddenGroupRef="HiddenBitMap" />
<element name="BitMap">
<complexType>
<sequence>
<!-- each bit in the bitmap modelled individually but derived
-->
<element name="Bit001" type="boolean"
dfdl:inputValueCalc="{dfdl:testBit(../BitMapInt, 1)}" />
...
<element name="Bit064" type="boolean"
dfdl:inputValueCalc="{dfdl:testBit(../BitMapInt, 64)}" />
</sequence>
</complexType>
</element>
<!-- each field below determines its optional occurrence based on a bit
in the bitmap field -->
<element name="Field001" type="string" minOccurs="0"
dfdl:occursKind="expression"
dfdl:occurs="{xs:integer(../BitMap/Bit001)}" />
...
<element name="Field064" type="string" minOccurs="0"
dfdl:occursKind="expression"
dfdl:occurs="{xs:integer(../BitMap/Bit064)}" />
</sequence>
</complexType>
<group name="HiddenBitMap">
<sequence>
<element name="BitMapInt" type="unsignedLong"
dfdl:representation="text" dfdl:textNumberRep="standard"
dfdl:textNumberBase="16" dfdl:encoding="ascii"
dfdl:lengthKind="explicit" dfdl:length="16"
dfdl:lengthUnits="characters"
dfdl:outputValueCalc="{dfdl:setBits(fn:exists(../Field001),
...
fn:exists(../Field064)}" />
</sequence>
</group>
For both packed and unpacked, the BitMap element could be hidden as well,
as there is no need for it in the infoset.
Regards
Steve Hanson
Strategy, Common Transformation & DFDL
Co-Chair, OGF DFDL WG
IBM SWG, Hursley, UK,
smh at uk.ibm.com,
tel +44-(0)1962-815848
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ogf.org/pipermail/dfdl-wg/attachments/20100920/a12462f8/attachment-0001.html
More information about the dfdl-wg
mailing list