[DFDL-WG] example/use-case wanted - separatorSuppressionPolicy 'trailingEmptyStrict' and 'never'

[Steve Hanson]

IBM has DFDL schemas for X12, various releases including the most common 
4010, obtainable via support ticket (same as for EDIFACT).
IBM has DFDL schemas for X12 HIPAA as part of App Connect for Healthcare 
v5.
I'm sure there are formats out there where all delimiters must be present. 
Old teletype formats like MATIP maybe. 

Regards
 
Steve Hanson
IBM Hybrid Integration, Hursley, UK
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
smh at uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890
Note: I work Tuesday to Friday 



From:   Mike Beckerle <mbeckerle.dfdl at gmail.com>
To:     Steve Hanson <smh at uk.ibm.com>
Cc:     DFDL-WG <dfdl-wg at ogf.org>
Date:   20/04/2021 20:03
Subject:        [EXTERNAL] Re: [DFDL-WG] example/use-case wanted - 
separatorSuppressionPolicy 'trailingEmptyStrict' and 'never'



I see that X12 is the USA-used EDI message standard - analogous to the 
UN/EDIFACT used outside USA. I'll pass along that this was motivated by 
X12, though to my knowledge there is no DFDL schema generally available 
for any part of X12. ZjQcmQRYFpfptBannerStart 
This Message Is From an External Sender 
This message came from outside your organization. 
ZjQcmQRYFpfptBannerEnd
I see that X12 is the USA-used EDI message standard - analogous to the 
UN/EDIFACT used outside USA. 
I'll pass along that this was motivated by X12, though to my knowledge 
there is no DFDL schema generally available for any part of X12. 

I think for SSP 'never' I can gen-up an example pretty easily. 

Given what I've got below, can you think of where such formats would 
appear in anything real-world?

The only thing I can think of is somebody taking a COBOL structure with 
two "occurs N times depending on ..." arrays in it, and modernizing the 
format into something variable-length with separators for use in a more 
modern application. 

My example for separatorSuppressionPolicy='never' is below.

Assuming dfdl:occursCountKind='implicit' everywhere

<sequence dfdl:separator="," dfdl:separatorSuppressionPolicy="never">
  <element name="givenName" type="xs:string" minOccurs="0" 
maxOccurs="3"/>  
  <element name="surname" type="xs:string" minOccurs="0"/>
  <element name="ph" type="xs:string" minOccurs="0" maxOccurs="6"/>  
</sequence>

This format means there are always 10 locations separated by 9 commas. 

Whether something is a givenName, surname, or phone number is just 
determined positionally by counting the separators as the parse passes 
them. 

Well-formed instance:

"michael, james,,rogers,888-888-8888,777-777-7777,,,,"

XML flavored DFDL infoset: 

<givenName>michael</givenName>
<givenName>james</givenName>
<surname>rogers</surname>
<ph>888-888-8888</ph>
<ph>777-777-7777</ph>

Well-formed instance:

"madonna,,,,,,,,," 

XML flavored equivalent infoset:

<givenName>madonna</givenName>

Both the above examples would unparse to exactly the input data. 

Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense | 
www.owlcyberdefense.com
Please note: Contributions to the DFDL Workgroup's email discussions are 
subject to the OGF Intellectual Property Policy



On Tue, Apr 20, 2021 at 1:08 PM Steve Hanson <smh at uk.ibm.com> wrote:
'trailingEmptyStrict' was for X12 if I recall. It is not valid to have 
trailing separators. 
'never' I don't have an example for. 

Regards
 
Steve Hanson 
IBM Hybrid Integration, Hursley, UK
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
smh at uk.ibm.com
tel:+44-1962-815848
mob:+44-7717-378890
Note: I work Tuesday to Friday 



From:        Mike Beckerle <mbeckerle.dfdl at gmail.com> 
To:        DFDL-WG <dfdl-wg at ogf.org> 
Date:        20/04/2021 15:12 
Subject:        [EXTERNAL] [DFDL-WG] example/use-case wanted - 
separatorSuppressionPolicy 'trailingEmptyStrict' and 'never' 
Sent by:        "dfdl-wg" <dfdl-wg-bounces at ogf.org> 



Users of Apache Daffodil have asked what these property values are for, 
i.e., what are idiomatic uses of this property for actual formats, or in 
other words, what kind of data are these properties needed for? 

I'd like to provide them examples of this if anyone can recall the kinds 
of specific use cases for these property values. 

I've been searching through the example DFDL schemas I can find, and none 
use trailingEmptyStrict or never for this property. 

Mike Beckerle | OGF DFDL Workgroup Co-Chair | Owl Cyber Defense | 
www.owlcyberdefense.com 
Please note: Contributions to the DFDL Workgroup's email discussions are 
subject to the OGF Intellectual Property Policy 
--
 dfdl-wg mailing list
 dfdl-wg at ogf.org
 https://www.ogf.org/mailman/listinfo/dfdl-wg 


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ogf.org/pipermail/dfdl-wg/attachments/20210420/ff437650/attachment-0001.html>