[DFDL-WG] 7-bit ascii packed together

[Mike Beckerle]

On our call today we discussed this and agreed we should just pick a name
for this encoding which uses the US-ASCII 7-bit codepoints, but instead of
using a full octet per, uses exactly 7 bits per codepoint.

I suggest this name should be US-ASCII-7-bit-packed.

US-ASCII is 'preferred' by IANA over just "ASCII" on the Internet, because
it avoids the "ascii family" connotation.

Note that according to the IANA encoding names are not case sensitive.

On Mon, Oct 22, 2012 at 6:32 AM, Steve Hanson <smh at uk.ibm.com> wrote:

> Hi Mike
>
> I thought this would come up at some point, and my assumption has always
> been that we would handle it using special enums of dfdl:encoding, and for
> fixed length use lengthUnits 'characters'. That means we can continue with
> our existing rules for when you use lengthUnits 'bits' and not have to
> extend them to xs:string. We would disallow lengthUnits 'bytes'.
>
> I would suggest that a DFDL parser takes the 7-bits and pads to 8-bits
> before calling ICU, and the reverse after calling ICU when unparsing. That
> way we don't need to get ICU to handle this. (I'm assuming they don't, Tim
> is going to find out).
>
> Regards
>
> Steve Hanson
> Architect, Data Format Description Language (DFDL)
> Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/>
> IBM SWG, Hursley, UK*
> **smh at uk.ibm.com* <smh at uk.ibm.com>
> tel:+44-1962-815848
>
>
>
> From:        Mike Beckerle <mbeckerle.dfdl at gmail.com>
> To:        dfdl-wg at ogf.org,
> Date:        19/10/2012 20:44
> Subject:        [DFDL-WG] 7-bit ascii packed together
> Sent by:        dfdl-wg-bounces at ogf.org
> ------------------------------
>
>
>
>
> I have a data format in front of me that has 64 7-bit ASCII characters,
> but the format has them bit-packed, i.e., 448 = 7 * 64 bits, so ....the
> character codes aren't octet/byte aligned.
>
> Furthermore, the 'string' either uses up the entire 64 character maximum
> length OR it has a terminating character which is a 0x7F character code.
>
> I believe I was the advocate for a position that character codes should
> always be 8-bit aligned. That would be because I had never seen anything
> like this.
>
> I am told there are also 6-bit ascii-variations, similarly packed together
> to save space.
>
> BTW: This occurs in a specific US MIL STD message header format, so it's
> not like it's some obscure unused corner case.
>
> Right now, the best I think I can do is to model this data not as a string
> at all, but as an array of integers, each one having 7-bit length, and not
> aligned (that is, aligned to 1-bit). Doing that I can use
> occursCountKind='parsed', and an assertion to deal with the optional
> termination by 0x7F value.
>
> To handle this as a string, we'd need to be able to specify that the
> character codes are not aligned, and the width of the bit-fields making up
> each character code. Or I suppose we could just say this is a special kind
> of character set encoding "ASCII-7-bit-packed" or something.
>
> Having that, I could deal with the termination via a choice of either the
> terminated flavor, or the fixed length flavor (which excludes the
> terminator) by way of a choice of two strings each having a
> lengthKind="pattern".
>
> Comments?
>
>
>
>
>
> --
> Mike Beckerle | OGF DFDL WG Co-Chair
> Tel:  781-330-0412
> --
>  dfdl-wg mailing list
>  dfdl-wg at ogf.org
>  https://www.ogf.org/mailman/listinfo/dfdl-wg
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>



-- 
Mike Beckerle | OGF DFDL WG Co-Chair
Tel:  781-330-0412
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ogf.org/pipermail/dfdl-wg/attachments/20121031/979ee519/attachment.html>