[DFDL-WG] Clarification: attributes in DFDL schema that are used not for data, but for other non-DFDL annotation elements.

Mon Mar 5 09:00:55 EST 2018

Of note. We've implemented this in Daffodil. So far it seems to work well.
A surprising number of the schemas we test did in fact have files without
any namespace declaration for DFDL. In one large schema, there is a master
schema file that includes all the (over 100) others, and that master schema
had no dfdl namespace definition as it didn't need one. This is a likely
case that will be seen in a number of other large schemas.

We issue a schema-definition warning whenever we ignore a schema file, so
that coupled with the error that inevitably one gets if an important schema
file is omitted make this easy to diagnose and fix.

...mike beckerle

Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF Intellectual Property Policy
<http://www.ogf.org/About/abt_policies.php>

On Fri, Mar 2, 2018 at 2:40 PM, Mike Beckerle <mbeckerle.dfdl at gmail.com>
wrote:

> (sent 2nd time due to mail problem.)
>
> I like your suggested solution.
>
> I'd modify point (a) to say - examine each imported schema. If the schema
> does not have any xmlns namespace binding for the DFDL namespace URI, then
> it is explicitly not a DFDL schema, and so would not be treated as one.
> (E.g., what if somebody really wants to use say "dfd" as a prefix, and so
> they want to bind the dfdl namespace to "f:" instead of the usual "dfdl:".)
>
> The import is effectively skipped by the DFDL processor, though if the
> DFDL schema was being validated separately (by say, Xerces), then the
> import would be processed and so the validation of the added annotation
> language would occur.
>
> Rather than points (c) and (d) I agree that requiring such schemas to be
> imported, but not included works well. References to the namespace would
> fail (since the DFDL processor would be skipping the import.)
>
> The only "hardship" if you can even call it that, is if you wanted to have
> a DFDL schema that just contains say, reusable type definitions, but
> doesn't actually have any DFDL annotations in it, then you would have to
> artificially add a dfdl namespace prefix definition to it. That seems very
> reasonable to me.
>
> Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
> www.tresys.com
> Please note: Contributions to the DFDL Workgroup's email discussions are
> subject to the OGF Intellectual Property Policy
> <http://www.ogf.org/About/abt_policies.php>
>
>
> On Fri, Feb 23, 2018 at 12:58 PM, Steve Hanson <smh at uk.ibm.com> wrote:
>
>> Mike
>>
>> Had to pull in the DFDLGeneralFormat.dfdl.xsd to get everything resolved,
>> but when I did, this is what IBM DFDL gave:
>>
>> *CTDS1086E : Included or imported schema not a DFDL schema.
>> 'platform:/resource/Daffodil tests/otherAnnotationLanguage.xsd'.*
>>
>> I think this is an IBM specific check - I could not find this restriction
>> in a quick review of the DFDL 1.0 spec. I believe it was done because the
>> IBM DFDL validator can't assume how schemas are to be used, and so
>> validates each xsd file in a library independently - which means each one
>> must validate cleanly on its own. We don't use '.dfdl' in the name so can't
>> tell the intent from the name.
>>
>> Adding the DFDL namespace to the target schema removes that error but as
>> you might expect then gives:
>>
>> *CTDS1000E : Attribute declarations (local and global) are not allowed in
>> DFDL schemas.*
>> *CTDS1000E : Attribute declarations (local and global) are not allowed in
>> DFDL schemas.*
>> *CTDS1001E : Attribute references are not allowed in DFDL schemas.*
>> *CTDS1008E : Complex types with empty content are not allowed in DFDL
>> schemas. Complex type 'urn:otherAnnotationLanguage#otherAnnotation_._type'.*
>>
>> Commenting out the xs:include removes all errors. In fact we rely on this
>> behaviour - when IBM DFDL editor imports COBOL, the COBOL compiler adds its
>> own set of annotations.
>>
>> I think the way to allow your scenario would be to:
>>
>> a) insist that the target schema did *not* contain an xmlns:dfdl
>> declaration, so is explicitly *not* a DFDL schema
>> b) do not validate such a schema independently
>> c) when processing a DFDL schema that uses it, keep a list of all global
>> objects that the target schema offers
>> c) issue a SDE if an object in the list was referenced by an object in
>> the DFDL schema (outside of an annotation, obviously)
>>
>> If you insisted that the target schema *must* be in a different
>> namespace to any of the DFDL schemas in the set, you could do the check on
>> namespace and not need to keep a list of objects. Any reference to the
>> namespace by an object (outside of an annotation) is immediately a SDE. So
>> allow such schemas to be imported but do not allow them to be included.
>>
>> Thoughts?
>>
>> Regards
>>
>> Steve Hanson
>>
>> IBM Hybrid Integration, Hursley, UK
>> Architect, *IBM DFDL*
>> <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
>> Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/>
>> *smh at uk.ibm.com* <smh at uk.ibm.com>
>> tel:+44-1962-815848 <+44%201962%20815848>
>> mob:+44-7717-378890 <+44%207717%20378890>
>> Note: I work Tuesday to Friday
>>
>>
>>
>> From:        Mike Beckerle <mbeckerle.dfdl at gmail.com>
>> To:        Steve Hanson <smh at uk.ibm.com>
>> Cc:        dfdl-wg at ogf.org
>> Date:        20/02/2018 23:18
>> Subject:        Re: [DFDL-WG] Clarification: attributes in DFDL schema
>> that are used not for data, but for other non-DFDL annotation elements.
>> ------------------------------
>>
>>
>>
>> Yes, files are attached.
>>
>> Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
>> *www.tresys.com*
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.tresys.com&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=AJa9ThEymJXYnOqu84mJuw&m=rJgPyinFaSd9n2H_ZqeiRRId3AD2ehRelanacvNIQYc&s=osdwwVMBgbE2pMiuU7vHGdPGAkprML1I0AxGtqVyrLA&e=>
>> Please note: Contributions to the DFDL Workgroup's email discussions are
>> subject to the *OGF Intellectual Property Policy*
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ogf.org_About_abt-5Fpolicies.php&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=AJa9ThEymJXYnOqu84mJuw&m=rJgPyinFaSd9n2H_ZqeiRRId3AD2ehRelanacvNIQYc&s=aOae6-HU7VDdWzcvwekuay2VfJ7tCwybDhy_QSL3Gmc&e=>
>>
>>
>> On Tue, Feb 20, 2018 at 2:20 PM, Steve Hanson <*smh at uk.ibm.com*
>> <smh at uk.ibm.com>> wrote:
>> Mike, is it possible to post a cut-down example that shows enough to give
>> the error in Daffodil, and I'll see if IBM DFDL gives a similar error.
>>
>> Regards
>>
>> Steve Hanson
>> IBM Hybrid Integration, Hursley, UK
>> Architect, *IBM DFDL*
>> <http://www.ibm.com/developerworks/library/se-dfdl/index.html>
>> Co-Chair, *OGF DFDL Working Group*
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ogf.org_dfdl_&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=AJa9ThEymJXYnOqu84mJuw&m=rJgPyinFaSd9n2H_ZqeiRRId3AD2ehRelanacvNIQYc&s=C_3TxayHMwdjplAn6GOpYTYvEvJ9vy1x7YVviVT0HGU&e=>
>> *smh at uk.ibm.com* <smh at uk.ibm.com>
>> tel:*+44-1962-815848* <+44%201962%20815848>
>> mob:*+44-7717-378890* <+44%207717%20378890>
>> Note: I work Tuesday to Friday
>>
>>
>>
>> From:        Mike Beckerle <*mbeckerle.dfdl at gmail.com*
>> <mbeckerle.dfdl at gmail.com>>
>> To:        *dfdl-wg at ogf.org* <dfdl-wg at ogf.org>
>> Date:        20/02/2018 16:44
>> Subject:        [DFDL-WG] Clarification: attributes in DFDL schema that
>> are used not for data, but for other non-DFDL annotation elements.
>> Sent by:        "dfdl-wg" <*dfdl-wg-bounces at ogf.org*
>> <dfdl-wg-bounces at ogf.org>>
>> ------------------------------
>>
>>
>>
>>
>> Interesting issue.
>>
>> DFDL doesn't allow use of attributes.
>>
>> But we have a user that has a DFDL schema to which they also want to add
>> other non-DFDL annotations.
>>
>> Like DFDL itself, this other annotation language has a variety of
>> annotation elements that appear inside xs:appinfo blocks with specific
>> source attribute specified. A DFDL processor should be ignoring these.
>>
>> Those other annotation elements have attributes (as do DFDL's annotation
>> elements.)
>>
>> But Daffodil issues a schema definition error on the attribute
>> declarations of this other annotation-language schema. This because it has
>> no way of telling if these attribute declarations are for use *only* in
>> annotations that it will ignore, or if they are used for modeling data.
>>
>> Question is: Is it acceptable for a DFDL annotation to just say "Nowhere
>> in the extended schema - all files - can there be any attribute
>> declarations.", or do we need to accommodate attributes and in fact every
>> XSD construct that DFDL disallows, so long as they're not being used to
>> model data?
>>
>> Really there's two questions here. First, does the DFDL specification
>> take (or need to take) a position on this. Second, what's a practical
>> choice for a DFDL implementation like Daffodil to take here.
>>
>> ...mikeb
>>
>> Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
>> *www.tresys.com*
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.tresys.com&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=AJa9ThEymJXYnOqu84mJuw&m=sc82YpTJe2zaMkOmRGSHfOxkxJcq0n_c9FjQ8aQefxE&s=Rxsta0Lr-rXWM16oYenXGrP7xH4Ti_yQTGKV06lYxp4&e=>
>> Please note: Contributions to the DFDL Workgroup's email discussions are
>> subject to the *OGF Intellectual Property Policy*
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ogf.org_About_abt-5Fpolicies.php&d=DwMFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=AJa9ThEymJXYnOqu84mJuw&m=sc82YpTJe2zaMkOmRGSHfOxkxJcq0n_c9FjQ8aQefxE&s=uNKBF8PdzGH9PItZ16o0PqxPLYFR4naem91zZaseC1w&e=>
>> --
>>  dfdl-wg mailing list
>>  *dfdl-wg at ogf.org* <dfdl-wg at ogf.org>
>>
>> *https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ogf.org_mailman_listinfo_dfdl-2Dwg&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=AJa9ThEymJXYnOqu84mJuw&m=sc82YpTJe2zaMkOmRGSHfOxkxJcq0n_c9FjQ8aQefxE&s=JkCAzi0EGJmPp-0RyMaiKEjpR4gkvCzyc60Q-qrVoaw&e=*
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ogf.org_mailman_listinfo_dfdl-2Dwg&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=AJa9ThEymJXYnOqu84mJuw&m=sc82YpTJe2zaMkOmRGSHfOxkxJcq0n_c9FjQ8aQefxE&s=JkCAzi0EGJmPp-0RyMaiKEjpR4gkvCzyc60Q-qrVoaw&e=>
>>
>> Unless stated otherwise above:
>> IBM United Kingdom Limited - Registered in England and Wales with number
>> 741598.
>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>>
>>
>>
>> Unless stated otherwise above:
>> IBM United Kingdom Limited - Registered in England and Wales with number
>> 741598.
>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.ogf.org/pipermail/dfdl-wg/attachments/20180305/b6b2ab25/attachment-0001.html>