[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: netcdf (operators and c++ class)

> Organization: Department of Rangeland Ecology and Management, Texas A&M 
> University
> Keywords: 199409301509.AA26387 C++ classes g++ interpreter bugs

Hi Dan,

> I started before I found the operators paper.  We would be happy
> to make it available, but it doesn't fit well with the current
> operators.  e.g. c++, requires libg++ for stack classes, uses
>       a modified c++ class, is intended to work on "growing"
>       netCDF files (does a sleep to wait on records that
>       are not there yet, works on variables one record at a time,
>       does syncs between writes etc.)

We aren't so committed to the way that the netCDF operators were originally
specified that we would ignore a better approach.  In fact, I've recently
talked to Harvey Davies of CSIRO about what we think is a different and
better approach to specifying array sections via a tiny language/notation he
has developed called FAN (file-array notation).  He has offered us a program
called nc2text (and the opposite text2nc) that uses FAN to specify sequences
of data from netCDF variables and incorporates quite a bit of the
functionality of several netCDF operators as well as units and type
conversion.

> The tack I am currently going with looks like the tcl extension
> could use a netCDF enhanced interpreter, and do operators as
> scripts (A lot of my the operator code is in doing arg parses,
> and tcl does that with great ease).  How does that fit in
> with the "approved" operator effort?

We originally thought about the idea of developing an interpreter for netCDF
expressions that would do everything we wanted to do in the operators, but
none of us particularly wanted to design another language.  However I think
this is actually a reasonable approach, as has been proven with various
interpretive languages that have added netCDF interfaces for I/O, such as
IDL and MATLAB as well as Davies FAN.  Davies has also implemented a netCDF
interface to J, which is a functional descendant of APL.

> Here is what I have done, if you want I can send diffs:

Thanks!

>       Bugs:
>               NcFile::add_var(NcToken, NcType, int, const NcDim**)
>               changed to be
>               NcFile::add_var(NcToken, NcType, int, const NcDim*)
>               (I could not make a program pass a list of dims
>               in that it created.)

I don't understand this one.  The last parameter is specified to be of type
const NcDim* in my sources in both netcdf.hh and netcdf.cc.  I don't remember
it ever being const NcDim**.  I just checked what's in the netcdf-2.3.2
distribution, and it's also const NcDim*.  There is an overloaded version of
NcFile:add_var() that takes from 1 to 5 const NcDim* parameters, but in what
source files was add_var ever specified to take a const NcDim** argument?

>               Added if(!data_mode()) return 0; to beginning of
>               NcFile::sync(void)

Good, I've incorporated your fix.

>               NcVar::set_cur() <both> extended the bounds checking
>               test to allow t[i] to be bigger than get_dim(i)->size()
>               if get_dim(i)->is_unlimited(), so you can use set_cur
>               with put to write new records

Yes, I had found this one recently but hadn't folded it into a patch yet.  I
found I needed to changed the statement in netcdf.cc from

                if (t[i] >= get_dim(i)->size())
to
                if (t[i] >= get_dim(i)->size() && ! get_dim(i)->is_unlimited())

>               const long* NcValues::edges(void) const to 
>               long* NcValues::edges(void) const, so a program can
>               do foo = edges(); foo[3] = 1; to get at slabs.

I'll assume you meant "NcVar" instead of "NcValues" here.  This seems like a
good fix, so I've incorporated it.

>       Enhancements for NcVar:
> 
>               get_rec/put_rec/set_rec member functions that use
>               the unlimited dimension to step through records, includes
>               changes to NcVal to allow a size parameter so you get
>               just a record rather than an NcVal of the entire
>               variable.
> 
>               get_index(TYPE *key), a slow linear search that 
>               finds the record number of the first variable that matches
>               some key

These both seem worthwhile.  Can you make the changes available to me?

>       Design change:
> 
>               I dumped the New/Old file concept because I wanted
>               to use inheritance to make a certain type of file
>               (e.g. spatial file, time series file, weather data
>               file) and be able to use it to create or read such
>               a file (the c++ code is really short, a new constructor
>               that adds variables/dims/attributes to be that type
>               of file, is_valid checks for required
>               variables/dims/attributes, and you are done, sometimes
>               there is some special extra function, maybe something
>               like get_rec(time) that knows what the dims look like
>               and will find a time, but not real often.)

I didn't realize the New/Old stuff would get in the way of this, but I see
now that's the case.  Did you change the interfaces of other class members
to keep the functionality of being able to specify whether a file should
already exist when you try to open it?  I'd be interested in seeing what
effect this design change has on the interface.

>       Stuff I'm thinking about:
> 
>               update the record/index stuff to use an arbitrary dim,
>               cache stuff for index lookup (make key,rec pairs, sort
>               etc.)

These sound like good ideas.  I would be interested in any progress you make
on these.

> If your bug fix is not in my bug fix list, send it on down.

The only one I had that wasn't in the FTP-able patches you already got was
the NcVar::set_cur() bug you found and fixed independently.

--
Russ Rew                                                UCAR Unidata Program
address@hidden                                          P.O. Box 3000
http://www.unidata.ucar.edu/                          Boulder, CO 80307-3000