Roy,
Okay guys, here is the "example" we are working on. A long time ago
we put the World Ocean Database and the GTSPP data, all subsurface
obs that we have or get, into the old HDF-EOS point structure. The
way we had it structured was that each file was a 10-degree square
with a name that contained the box number as in COADS. The HDF-EOS
point structure had levels that made what was a series of flat tables
look like a hierarchy by having a link field. So level 0 data would
be all the box 2's in the box 10, and the level 1 data would be the
data with the box2 number as the link field. This allowed you to
search for all the data in a box2 efficiently
A downside was that the number of rows in the field had to be equal
to the sum total of depths at which there were observations. This
meant that there is a huge amount of empty space.
I understand the COADS box numbers for 10 degree or two degree boxes,
but I don't understand what you're saying about levels and link
fields.
Is there a simpler or more detailed description of this data structure
somewhere that I could study? I'd like to understand the requirements
and whether netCDF-4 could eliminate the wasted space while preserving
fast access.
... So how might we
improve on this - why netcdf4! We use vlen's to deal with the
varying number of depths and a compound data type to combine together
the info. Groups and (sub)-Groups provide the same structure that
was "emulated" in the point structure. Or at least that was what we
thought would be a good design given netcdf4's features, but that is
what prompted my email about examples and design - there may well be
better ways of doing this.
However, unless I miraculously find both money and a good C
programmer, or get Fortran wrappers to work, it will be sometime
before it happens.
BTW, at one point I used the hdf4tohdf5 program to convert these
files to HDF5, so they can be read using that library. I don't know
that the conversion worked that cleanly to use with the HDF-EOS5
library, but it should be possible to both read and write using the
libs that are a part of Netcdf4. I actually had a few days to begin
to work on this, hence the series of emails.
-Roy
**********************
"The contents of this message do not reflect any position of the U.S.
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097
e-mail: address@hidden (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/
"Old age and treachery will overcome youth and skill."
