[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Use of NcVar::Put method

> Date: Mon, 25 Nov 1996 16:09:34 +0100
> From: address@hidden
> To: address@hidden
> Subject: Use of NcVar::Put method

Hi Laurent,

> I use the Put method of the NcVar class. To improve performance of a
> netCDF writer, I try to write several values for the same variable at
> a time (10 000 exactely) and it seems that the Put method does not
> buffer the output, writing each value at a time. The result is a very
> poor performance for my writer. Is this a normal behaviour or do you
> think there could be a bug in that method?

All of the overloaded NcVar::put() methods just call the underlying C
function ncvarput() for an array of data.  The ncvarput() function
buffers the data for array elements that are contiguous on disk.  Hence
the NcVar::put() methods should perform just like the corresponding
ncvarput() call.

However, it's possible that the values you are writing are not
contiguous in the disk representation due to use of the UNLIMITED
dimension, so ncvarput() would have to call write() multiple times,
maybe even once per value.

For example, if your file was structured similar to the following:

    netcdf example{
    dimensions:
        rec = UNLIMITED;
    variables:
        float x(rec);
        float y(rec);
    }

then the actual order of values on the disk is

    x[0], y[0], x[1], y[1], x[2], y[2], ...

so if you write all the x values, the result will be a single write call
for each value.  However, if your file was structured like this:

    netcdf example{
    dimensions:
        n = 100;
        rec = unlimited;
    variables:
        float x(rec, n);
        float y(rec, n);
    }

and you wrote all the x values, 100 values would be written with each
call, since the order in which they are stored is:    

    x[0][0], x[0][1], x[0][2], x[0][99], y[0][0], y[0][1], ..., y[0][99],
    x[1][0], x[1][1], x[1][2], x[1][99], y[1][0], y[1][1], ..., y[1][99],
     ...

And for a variable that doesn't use the unlimited dimension at all, all
values are stored contiguously in a single block, so optimal buffering
will occur.

For more details on file structure and how its effect on performance,
see Chapter 9 in the netCDF User's Guide, "NetCDF File Structure and
Performance" or see the on-line version at:

   http://www.unidata.ucar.edu/packages/netcdf/guide_11.html

_____________________________________________________________________

Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu