[netcdfgroup] arrays of fixed-length strings

[Jeff Whitaker]

John Caron wrote:
>
>
> Jeff Whitaker wrote:
>> John Caron wrote:
>>>
>>>
>>> Jeff Whitaker wrote:
>>>> Ed Hartnett wrote:
>>>>> Jeff Whitaker <jswhit at fastmail.fm> writes:
>>>>>
>>>>>  
>>>>>> In netcdf-4.0, I don't see how to create variables which are 
>>>>>> arrays of
>>>>>> strings with length > 1.  I see how to create arrays of
>>>>>> single-characters, and arrays of variable-length strings, but not
>>>>>> strings of a specified length.
>>>>>>
>>>>>> Am I missing something, or is this not supported by HDF5?
>>>>>>
>>>>>>     
>>>>>
>>>>> Howdy Jeff!
>>>>>
>>>>> Strings are variable length by their nature.
>>>>>
>>>>> How about a two dimensional array of NC_CHAR?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Ed
>>>>>
>>>>>
>>>>>   
>>>>
>>>>
>>>> Ed:  I use arrays of fixed length strings, padded with spaces, 
>>>> quite a bit.  This simplifies the memory management issues 
>>>> associated with arrays of variable length strings (which have no 
>>>> counterpart in Fortran 90/95, although they are allowed in Fortran 
>>>> 2003).  Below is an excerpt from my previous reply which explains 
>>>> why I don't like using 2-D arrays of characters to represent 1-D 
>>>> arrays of fixed-length strings:
>>>>
>>>> Russ: I realize you can use a array of shape ndim,8 to store an 
>>>> array of ndim 8 character strings.  Thats the way I've done it with 
>>>> netcdf-3 - it just feels clunky.  A typical use case for me is 
>>>> station data, where you want to store the name of the station. I 
>>>> end up the with an array of characters shaped (nstations,ncars) -  
>>>> in fortran I read it into an  (nstations,nchars) character(len=1) 
>>>> array (after first finding out what both nstations and nchars are), 
>>>> then reshape it into a (nstations) character(len=nchars) array.  
>>>> I'd rather just read it into a character(len=nchars) array straight 
>>>> off.  Not a show stopper for sure, but it would be more 
>>>> convenient.  I realize that specifying the data type would be 
>>>> tricky,  instead of NC_CHAR, do you have a bunch of new types 
>>>> NC_CHAR1, NC_CHAR2, ... NC_CHAR120?  Or a new function datatype = 
>>>> nc_set_chartype(nchars)?  However, I bet it would get used a lot 
>>>> more than the esoteric datatypes you have in netcdf-4 already 
>>>> (enums and opaque for example).
>>>
>>> Hi Jeff:
>>>
>>> Suppose we stuck with fixed length char arrays for this case, but 
>>> added a convenience method in the API that did the work for you. 
>>> What would that convenience method look like?
>>>
>>> John
>>
>> John:  A convenience method won't really help much - the thing I'd 
>> most like to avoid is defining another dimension to hold the number 
>> of characters in each string.  Essentially, I'd like to have that 
>> information transferred to the datatype.
>
> Hi Jeff:
>
> It seems unlikely that we'd want to add a multitude of datatypes for 
> this purpose. The extra dimension seems the right thing if you really 
> want to specify that all the Strings are of the same length.
> John

John:  OK.  Since HDF5 defines a fixed-length string datatype like this:


    tid = H5Tcopy(H5T_C_S1);
    H5Tset_size(tid, <string length>);

I thought you might be able to create a datatype on the fly in netcdf 
with something like

stringtype = nc_set_stringtype(<string length>)

and then use 'stringtype' instead of NC_CHAR when defining the variable.

I guess I'm just used to thinking in fortran, where the length of the 
string is a property of the datatype (defined when the variable is 
declared).  It seems like dimensions should hold information about the 
underlying physical grid, not implementation details of how the variable 
is stored in memory.  I guess it depends on whether you think of a 
string as an array of characters, or as a datatype unto itself.

-Jeff

-- 
Jeffrey S. Whitaker         Phone  : (303)497-6313
Meteorologist               FAX    : (303)497-6449
NOAA/OAR/PSD  R/PSD1        Email  : Jeffrey.S.Whitaker at noaa.gov
325 Broadway                Office : Skaggs Research Cntr 1D-124
Boulder, CO, USA 80303-3328 Web    : http://tinyurl.com/5telg