[jsdl-wg] (Param Sweep) - Stephen Pickles' public comment

[geoff]

Hi Stephen,

Thanks for providing further input, it's much appreciated.

I'm sure that there'd be no objection to improving the wording of the 
specification, as per your suggestions, to ensure that there can be no 
misinterpretation. We can make sure that the words "decimal" and 
"double" are clearly explained in context, e.g. that "xsd:double" 
represents the value range for JSDL sweep xml documents to be considered 
valid, and recommending the use of "decimal" arithmetic in 
implementations. Perhaps an additional section (or references to 
external resources) for implementation guidance for cardinality 
determination, precision, etc?

I don't think that we should change LoopDouble to LoopDecimal though 
because: 1) primarily that may cause uncertainly from people mistakenly 
associating "decimal" with xsd:decimal.... which we do not use because 
it doesn't permit use of scientific notation, and 2) perhaps, although 
unlikely, someone may eventually choose to loop over xsd:decimal values 
and won't be able to call it LoopDecimal!

To point to some earlier deliberations on name convention and tightening 
up of the specification you're welcome to look at :
http://forge.ogf.org/sf/wiki/do/viewPage/projects.jsdl-wg/wiki/ParameterSweepExtension
I hope to reassure you that quite a few of the issues and concerns you 
raise have been discussed during teleconfs when preparing the 
specification. From recollection though there is a concensus that we 
should avoid "exotic"! xsd, e.g. defining new xsd datatypes/constructs 
to restrict permissible values, so as to ensure JSDL documents can be 
handled safely by as many tools as possible.

gef



Stephen Pickles wrote:
> Dear All,
> 
> To cut a long story short, I think that Geoff' is talking about a specification
> of LoopDouble that is subtly different to the one that went to public comment.
> 
> However, I also think that the specification that Geoff is defending
> (and has implemented) is actually BETTER than the published draft.
> 
> I shall first re-iterate why I believe the current specification to be flawed.
> Then I shall try to explain the kind of changes that would be needed to fix it.
> 
> Bear with me.
> 
> Why the current specification is flawed:
> ----------------------------------------------------
> 
> In section 4.3, which is entitled, significantly, "LoopDouble", we read:
> 
> <blockquote>
> The LoopDouble function provides an ordered list of double datatype values
> (corresponding to the IEEE double-precision 64-bit floating point type)...
> </blockquote>
> 
> and later:
> 
> <blockquote>
> The permissible value range for xsd:double is defined as consisting of
> "... the values m  2^e,  where m is an integer whose absolute value is less
> than 2^53, and e is an integer between -1075 and 970, inclusive
> </blockquote>
> 
> What all this implies (at least to this reader) is that the LoopDouble
> function deals with _binary_ 64-bit floating point numbers.
> It would seem therefore, taking the specification at face value, that
> an implementor of LoopDouble is allowed, even encouraged, to perform
> the necessary calculations in IEEE binary floating point arithmetic.
> It follows that conversion between the decimal string representations
> (xsd:double) and the nearest exact binary floating point must (or at least,
> is allowed to) happen in both directions. Now, as it is rather exceptional
> for any floating point decimal to have an exact representation in floating
> point binary (and vice versa), you end up with the problems I mentioned
> in my public comments and at a previous JSDL conference call, i.e.
> (1) how to determine cardinality
> (2) how to decide whether an exception "hits" the list generated by LoopDouble
> not to mention problems with reproducibility (across different implementations)
> that the slightest departure from IEEE compliance can cause.
> 
> These problems have nothing to do with the accuracy of binary floating point
> arithmetic - only in infinite precision can you do exact conversions between
> base 2 and base 10 floating point numbers.
> 
> Geoff''s comments don't change my position on this. As long as the specification
> is dealing explicitly or implicitly with binary floating point numbers, you need
> to change something to recover certainty about cardinality and reproducibility
> of exception-matching. My suggestion, which Geoff' calls option (B), addresses
> the cardinality problem only.
> 
> Why Geoff's way is better
> ----------------------------------
> 
> All the aforesaid problems arise from conversion between decimal and
> binary floating point. They go away if the xsd:double values in LoopDouble
> (i.e. both the input parameters and the ordered list of values it returns) are
> interpreted as exact floating point decimal values, and the arithmetic is
> performed (with sufficient precision) also in floating point decimal.
> 
> This offers an alternative resolution to option (B), i.e. to prescribe
> the use of _decimal_ floating point arithmetic in LoopDouble.
> 
> Let me call this option (C).
> 
> Is the JSDL working group willing to prescribe the use of decimal arithmetic?
> 
> Clearly, Java implementors should have no problem. Implementors in other
> languages should be OK if they have an equivalent of Java's BigDecimal class.
> Even without such library support, it should not be too difficult to implement
> using only integer arithmetic - you don't need the full armoury of arithmetic
> operations; just extended precision addition and/or subtraction should be
> sufficient. I've done such things in the past.
> 
> How to fix the spec for option (C)
> --------------------------------------------
> 
> These are my suggestions for changes to the specification to
> bring it in line with what Geoff has presumably implemented.
> 
> 1. Use language that makes it explicit that the values involved
> are exact floating-point decimals, and that arithmetic MUST
> be performed in floating-point decimal. Get rid of all mentions
> of "double" (which conveys 64-bit base-2 floating point arithmetic
> to most readers).
> 
> Of course, a user of the parameter sweep extension still can and will
> convert (approximately) the decimal string into some base-2
> internal representation once it reaches their application code.
> What they do with it is up to them.
> 
> 2. Change the name of the LoopDouble function to LoopDecimal.
> (Sorry, Geoff, you would need to change SOME code, even if
> it's just a tag name.)
> 
> 3. Include an informational reference to IEEE 754-2008.
> 
> 4. You could choose to limit the number of significant decimal digits that
> an implementation must support.
> 
> 5. You can retain xsd:double to constrain the allowable parameters
> and return values. Although xsd:double was designed as an approximate
> decimal string representation of some other binary floating point number,
> see e.g.
>   http://books.xmlschemata.org/relaxng/ch19-77065.html
> It has a format that's (almost) exactly what you want.
> 
> (Although, to be pedantic, you it would be wise  to constrain it further to
> disallow NaN, INF, and -INF as loop start, end and increment values.)
> 
> 6. Tighten up the language on termination conditions. And negative
> increments. Disallow zero increments. See my public comments.
> 
> 7. Once it's clear that you're working in the realm of floating point decimals,
> you can determine the trip count of an option (A) style loop by inspection.
> I strongly recommend that you include in the specification a formula
> for calculating the loop cardinality. I leave this as an exercise for the
> reader (hint: see the Fortran specification).
> 
> I hope this is clear. I hope it helps.
> 
> Stephen
> 
> PS I noticed some other errors while re-reading the spec:
> (i) The mention of xsd:decimal in section 4.3 is spurious, misleading
> and actually incorrect.
> (ii) There's a lot of jumping over "lay dogs" in the examples, where
> "lazy dogs" is surely intended.
>