Re: HDF5 dimension scales proposal

Mike Folk wrote:



Here is our latest dimension scales proposal, which I mentioned in Friday's meeting

Paper: http://hdf.ncsa.uiuc.edu/RFC/H5DimScales/H5dimscale_prop_No3.v3.pdf
Slides: http://hdf.ncsa.uiuc.edu/RFC/H5DimScales/HDF5_dimscale_prop_3.pdf

I've hesitated to share these because they contains a lot of background verbiage that is probably not of interest to most of you. To avoid this:

 - start with section 2 ("Proposal") in the paper
 - start with slide #13 in the slides.

If you wish the original MS Word and power point versions of these, just let me know.

Mike

--
Mike Folk, Scientific Data Tech (HDF)   http://hdf.ncsa.uiuc.edu
NCSA/U of Illinois at Urbana-Champaign          217-244-0647 voice
605 E. Springfield Ave., Champaign IL 61820 217-244-1987 fax

Hi Mike:

I like your general design, keeping complexity out of the file format by implementing through an API on top of existing stuff. Some questions and comments:

1. A dimension scale will be a dataset, and wont be hidden, so wont it always appear in a group and have a name there?

2. I dont think we really get netcdf-style shared dimensions out of this. You can sort of figure it out if you are sharing a dimension scale (even there there may be some ambiguities), but it doesnt seem like you can associate two dimensions in different variables if there's no dimension scale.

3. The idea that the dimension scale could have a different length then the dimension is confusing. I can see how that general case may be useful for complex things like meshes etc, though.

4. I think the main place where your proposal may fail to cover the general case is that you seem to require that a dimension scale is associated with a single dimension. But the general case is that it can be associated with several dimensions, eg lat(x,y). For that case, it makes more sense to associate a dimension scale with a dataspace. But then you still have to associate the dimensions of the data dataset with the dimensions of the dimension scale dataset. Giving the dimensions names and requiring their lengths to be the same would work, and would be an implementation of shared dimensions for the case of shared dimension scales.

5. Im unclear why, at the end of 4.5.2, you would have the object reference to the dimension scale be optional. It appears that you want to allow naming of dimensions. That I agree is a good idea, but perhaps should be seperate from whether there is a dimension scale, which should require an object reference?

Anyway, seems like were making progress!