Arrays issue - Re: [dfdl-wg] Issues: additional data types

[Robert E. McGrath]

Yes, this is one way to do arrays.

This approach emphasizes the use case where it is important to
access individual elements via XML.

There are two obvious down sides:

   1. space:  this will be >10 times the storage of the actual numbers.
      A big problem for many cases.
   2. array algorithms (e.g., scatter-gather, transpose) do
      block operations which are totally ugly in this markup.

A variant of this might mark up parts of the array, e.g., each row.


Two other general approaches can be considered:

Array as blob:  markup says 'this is an array, laid out like so',
data is a big blob. (Probably this is what Jim is talking about)

Array as external blob:  same as above, except payload is a URL,
e.g., to OpenDAP server where the data is. (Ideal for "virtual datasets")


The memo I was working on tries to lay these options out with the
advantages and disadvantages.

---
Robert E. McGrath
National Center for Supercomputing Applications
University of Illinois, Urbana-Champaign
Champaign, Illinois 61820
(217)-333-6549

mcgrath at ncsa.uiuc.edu