[cddlm] Meeting tomorrow, 11/3, 6am PST

[Steve Loughran]

Jun Tatemura wrote:
> Hi,
> Please find the attached file that shows my current plan
> on CDL component test cases. In order to test CDL resolution,
> I think we need to have a special API. I will develop unit test code
> with such API if we agree on that.
> 
> Thanks,
> Jun
> 

this is really interesting, and aligned with a lot of my thoughts.

I think it ought to be possible to avoid having a specific junit 
tests/test runner. I'd expect most junit test methods to consist of the 
following

1. a test method that loads a document
2. then asserts that certain things have value


e.g

void testValid1() {
   document d=load(VALID_1);
   assertResolvesTo(d,"cdl:system/app:a1/@app:attr","value");
}

or asserts that the load failed in some interesting way

     public void testWrongDocNamespace() throws Exception {
         assertInvalidCDL(CDL_DOC_WRONG_NAMESPACE, WRONG_NAMESPACE_TEXT);
     }

Failures are just as interesting as successes, and right now we lack 
enough fault standardisation to begin to test those uniformly. It is not 
enough to assert that the doc failed to load, you want the doc to fail 
in the way you expected (this is important for regression testing, 
believe me). Yet unless you wrap every single XML parser fault in the

The point is, the java-side is pretty simple, primarily a declaration of 
a doc to load and then either some exception expected or some xpath 
expressions and what they resolve to.

All of this could be expressed in XML test documents, docs that declare 
both the test and either some

  <t:test id="uuid:787878-acb0-0bca">
    <t:metadata>  <!-- rdf metadata -->
      <dc:author>
    </t:metadata>
    <t:description>Load a CDL document with an invalid 
namespace</t:description>
    <t:source>
	<cdl:cdl xmlns:cdl="http://example.org/23"
	    xmlns:xs="http://www.w3.org/2001/XMLSchema"
	    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	    />
    <t:source>
    <t:exception>
      <t:text t:required="false">
        <t:description>this is the xerces message</t:description>
          <t:value>Cannot find the declaration of element 
'cdl:cdl'"</t:value>
      <t:text>
    </t:exception>
   </t>

For a succeeding doc, t:exception would be absent and we would have a 
list of assertions about path values down the list

  <t:assertions>
          <t:assertResolvesTo>
            <t:xpath>cdl:system/app:a1/@app:attr</t:xpath>
            <t:value>value</t:value>
          <t:assertResolvesTo>
          <t:assertNotResolved>
            <t:xpath>cdl:system/app:a2</t:xpath>
          <t:assertNotResolved>
          <t:assertResolvesTo>
            <t:xpath>cdl:system/app:a1/@app:attr</t:xpath>
            <t:lazyref>cdl:system/app:a3</t:lazyref>
          <t:assertResolvesTo>
  </t:assert>

The value of this is
  1. its platform neutral
  2. the miracle that is XSLT could theoretically generate documentation 
from the test data, the way these people do

http://www.w3.org/2001/sw/DataAccess/tests/

That w3c working group has a interesting policy: 
http://www.w3.org/2001/sw/DataAccess/tests/README.html
  -all issues must come with a test case
  -every test case is bound back to an issue, which is in the metadata
  -every test case also has a status. it is required that all 
implementations implement "approved" test cases, but the other ones 
(NotClassified, Rejected, Obsoleted and Withdrawn) can be run, but it is 
not mandatory for an implementation.

  I think this approach will work for CDL, though we will need to evolve 
the list assertions as we go on. I'd also add an <assertFalse> assert 
that asserts that something nested inside didnt pass to simplify the 
set. with that, I could replace the
         <t:assertNotResolved>
	    <t:xpath>cdl:system/app:a2</t:xpath>
          <t:assertNotResolved>

  with the far longer

<t:assertFailed>
         <t:assertResolvesTo>
            <t:xpath>cdl:system/app:a2</t:xpath>
          <t:assertResolvesTo>
</t:assertFailed>



-Steve