ncdigest V1 #1119

[Jeff Whitaker]

Paul van Delst wrote:
> Jeff Whitaker wrote:
>> Ed Hartnett wrote:
>>> Jeff Whitaker <jswhit at fastmail.fm> writes:
>>>
>>>  
>>>> By the way, while I'm ranting ...  Another potential problem is see is
>>>> nested user-defined types.  Since this is allowed in netcdf-4, I can
>>>> see the potential for creating horribly complicated files with
>>>> compounds of vlens containing compounds containing enums etc.  Don't
>>>> think many people would be crazy enough to use it, but it makes it
>>>> very hard to create client code to read arbitrary netcdf-4 files,
>>>> since you have to check for all those possibilities in general.  I
>>>> wonder whether it might not be wise to sacrifice some generality for
>>>> simplicity and just say that user-defined types can only be composed
>>>> of primitive data types.
>>>>
>>>>     
>>>
>>> Howdy Jeff!
>>>
>>> Here at NetCDF World Headquarters, we also wonder how these new
>>> features will be used by the netCDF community!
>>>
>>> Perhaps some of the HDF5 readers of this mailing list can provide some
>>> real world examples of nested structures.
>>>
>>> To read them, the only general purpose answer I know of (as with
>>> groups) is a recursive function. This is how the netCDF-4 file reading
>>> code handles the problem, and, I believe, how ncdump handles the
>>> problem (Russ will correct me if I'm wrong about ncdump).
>>> Both of these are in C (for example libsrc4/nc4hdf5.c, function
>>> rec_read_metadata().
>>>
>>> The problem, perhaps, is what to do with this information once you
>>> have it.
>>> For example, it's easy to plot a data file full of NC_INT, or
>>> NC_FLOAT, but how do you plot a file full of data in compound types?
>>>
>>> It is a more complicated problem, but I'm sure the very clever
>>> programmers working on the IDV will come up with something. ;-)
>>>
>>> Although these new features present many new complexities, the
>>> CLASSIC_MODEL flag allows users to produce netCDF-4 files which don't
>>> use any of the user-defined types, and thus can be read (and, with a
>>> little modification on the nc_create call, written) by existing
>>> software.
>>>
>>> Thanks!
>>>
>>> Ed
>>>
>>>   
>> Ed:  I think it's relatively easy (and productive)  to deal with 
>> compound types, where the elements are primitive (fixed size) data 
>> types.  I think that allowing nested compound types (or compound 
>> types containing vlens) increases the complexity exponentially 
>> though,   Unless there are some real use cases out there, I wonder if 
>> it is really worth it.
>
> This is an anecdotal datapoint, but I use nested structures now (only 
> one level deep though) in my f95 code. I tend to write "atomic" I/O 
> functions (wrappers around the required f90 API calls) for structures 
> which translates to reading from separate files. It would be nice to 
> be able to "layer" that sort of functionality. I guess it's a 
> file-within-a-file type of thing. 

Paul: This sounds like 'groups' in netcdf-4.  Basically, groups in 
netcdf-4 are like having netcdf files within netcdf files (using 
something like  unix directory structure).  See

http://www.unidata.ucar.edu/software/netcdf/workshops/2007/groups-types/Introduction.html

I've found this to a very useful feature.

-Jeff

-- 
Jeffrey S. Whitaker         Phone  : (303)497-6313
Meteorologist               FAX    : (303)497-6449
NOAA/OAR/PSD  R/PSD1        Email  : Jeffrey.S.Whitaker at noaa.gov
325 Broadway                Office : Skaggs Research Cntr 1D-124
Boulder, CO, USA 80303-3328 Web    : http://tinyurl.com/5telg

==============================================================================
To unsubscribe netcdfgroup, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================