[DFDL-WG] DFDL and Multi-dimensional arrays - just say no to them?

[Westhead, Martin (Martin)]

My first reaction is this: If we are confident that we have the
necessary hooks/extensibility mechanisms that multi dimensional arrays
can be defined in all their glory as a supplement. Then it is very easy
to say no.

 

Another tack on this is the following. I do not think that we need to
support arbitrary array representations in XML. However we do need to be
able to describe the data format including metadata such as array
dimensions etc. So I would push to go a little farther than your 1d
array and say we need to define a way to describe the
multidimensionality of your 1d array. Likewise it would be really good
to have metadata to capture the description of sparse representations...

 

Martin

 

  _____  

From: dfdl-wg-bounces at ogf.org [mailto:dfdl-wg-bounces at ogf.org] On Behalf
Of Mike Beckerle
Sent: Wednesday, September 20, 2006 10:17 AM
To: dfdl-wg at ogf.org
Subject: [DFDL-WG] DFDL and Multi-dimensional arrays - just say no to
them?

 


I'd like to solicit opinions on this suggestion: 

We have been assuming we needed more than just 1d array support, but
this is now unclear. 

We've pushed back on "top-down" use of XML Schemas, i.e., DFDL must be
used bottom up, and transformation into the "logical form you wanted" is
not our job. That is, the DFDL Schema's logical organization is
constrained (heavily) by the data format. 

We could take an exactly analogous position with respect to
multi-dimensional or other complex arrays. 

The DFDL describes the representation, and the sttructure of the DFDL
schema will end up matching the shape of the representation of the
array. Transforming that into something that looks and acts like a
ordinary dense multi-dimensional array is a transformation that is
out-of-scope for us. 

E.g., if the array is stored as a run-length encoded vector, then it is
DFDL's job to describe this run-length encoded vector, but not to
project/transform it so that it can be accessed in a manner that hides
the run-length encoding and makes it look like an ordinary dense array. 

(I actually believe hiding sparse array implementations behind a
dense-array facade is generally not advised. Algorithmically you must
operate on the sparce representation anyway for efficiency. ) 

Suggested Conclusion: we can just say that we don't do multi-dimensional
arrays because it is out-of-scope for us. 

Comments? 

...mikeb 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.ogf.org/pipermail/dfdl-wg/attachments/20060920/275c74f5/attachment.html