Re: [netcdfgroup] arrays of fixed-length strings



Jeff Whitaker wrote:
John Caron wrote:


Jeff Whitaker wrote:
John Caron wrote:


Jeff Whitaker wrote:
Ed Hartnett wrote:
Jeff Whitaker <jswhit@xxxxxxxxxxx> writes:

In netcdf-4.0, I don't see how to create variables which are arrays of
strings with length > 1.  I see how to create arrays of
single-characters, and arrays of variable-length strings, but not
strings of a specified length.

Am I missing something, or is this not supported by HDF5?


Howdy Jeff!

Strings are variable length by their nature.

How about a two dimensional array of NC_CHAR?

Thanks,

Ed




Ed: I use arrays of fixed length strings, padded with spaces, quite a bit. This simplifies the memory management issues associated with arrays of variable length strings (which have no counterpart in Fortran 90/95, although they are allowed in Fortran 2003). Below is an excerpt from my previous reply which explains why I don't like using 2-D arrays of characters to represent 1-D arrays of fixed-length strings:

Russ: I realize you can use a array of shape ndim,8 to store an array of ndim 8 character strings. Thats the way I've done it with netcdf-3 - it just feels clunky. A typical use case for me is station data, where you want to store the name of the station. I end up the with an array of characters shaped (nstations,ncars) - in fortran I read it into an (nstations,nchars) character(len=1) array (after first finding out what both nstations and nchars are), then reshape it into a (nstations) character(len=nchars) array. I'd rather just read it into a character(len=nchars) array straight off. Not a show stopper for sure, but it would be more convenient. I realize that specifying the data type would be tricky, instead of NC_CHAR, do you have a bunch of new types NC_CHAR1, NC_CHAR2, ... NC_CHAR120? Or a new function datatype = nc_set_chartype(nchars)? However, I bet it would get used a lot more than the esoteric datatypes you have in netcdf-4 already (enums and opaque for example).

Hi Jeff:

Suppose we stuck with fixed length char arrays for this case, but added a convenience method in the API that did the work for you. What would that convenience method look like?

John

John: A convenience method won't really help much - the thing I'd most like to avoid is defining another dimension to hold the number of characters in each string. Essentially, I'd like to have that information transferred to the datatype.

Hi Jeff:

It seems unlikely that we'd want to add a multitude of datatypes for this purpose. The extra dimension seems the right thing if you really want to specify that all the Strings are of the same length.
John

John:  OK.  Since HDF5 defines a fixed-length string datatype like this:


   tid = H5Tcopy(H5T_C_S1);
   H5Tset_size(tid, <string length>);

I thought you might be able to create a datatype on the fly in netcdf with something like

stringtype = nc_set_stringtype(<string length>)

and then use 'stringtype' instead of NC_CHAR when defining the variable.

I guess I'm just used to thinking in fortran, where the length of the string is a property of the datatype (defined when the variable is declared). It seems like dimensions should hold information about the underlying physical grid, not implementation details of how the variable is stored in memory. I guess it depends on whether you think of a string as an array of characters, or as a datatype unto itself.

-Jeff

Hi Jeff:

Its possible that the C API could offer such a feature, that would be up to 
Russ and Ed. But in terms of the underlying data model, I think that Strings 
will be considered variable length. One could consider Fortran's handling of 
Strings to be, uh, clumsy. I dont know of any modern languages that follow 
that, and I vaguely remembering Pascal being criticized for making array 
lengths part of the type.

Its a good point about what info a Dimension should capture. They really start off as 
indicating storage size and layout. Shared use with coordinate variables give them a much 
deeper significance describing the "underlying physical grid", as you say. 
Surprisingly, HDF5 and OPeNDAP data models didnt make them seperate objects that could be 
shared, though both have added other constructs to get similar semantics (dimension 
scales and grid maps, respectively).

As always, these things are a tradeoff.

I appreciate hearing your thoughts about these kinds of things.

John


  • 2007 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: