[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 959515: About NetCDF file size

>Keywords: 199505161456.AA21911 netCDF file size

Hi Francisco,

>       I was trying the NetCDF lib for writing 2D arrays at several
>       timesteps.  The problem is that number of arrays per timestep is
>       variable (sometimes pressure and temperature, other times velocity,
>       etc). I was surprised to discover that the file size was virtually
>       the same irespective the number of actual arrays that i ordered to
>       write. What is happening?

I assume you are using the unlimited dimension for time, so that each
timestep corresponds to one record.  One of the features of netCDF is direct
access to subsets of the data and cross-sections of variables.  Another is
that data can be written in different orders than it is read.  In other
words, it is possible to write the pressure array corresponding to timestep
10, and then later write the pressure array corresponding to timestep 5.
Then the user can request to read a slice of the pressure array for all
timesteps from 0 through 10.

To permit this out-of-order writing of data, each record is of fixed size,
determined by the sum of the sizes of a single record of data for all the
record variables (variables that use the unlimited dimension).

Whenever data is written beyond the last record so far written, "fill
values" are used to write values for data that hasn't been written yet, so
you can detect attempts to read data that has not yet been written
(e.g. trying to read the pressure array for the 8th timestep when only the
10th and 5th timesteps have so far been written).

This may be excess generality for your purposes, but there are users of the
netCDF library that require the capability to write data out of order and in
a different order from which it is read.

For more details on the structure and size of a netCDF file, see the Chapter
on "NetCDF File Structure and Performance" in the "NetCDF User's Guide" at

    http://www.unidata.ucar.edu/packages/netcdf/guide.txn_11.html#SEC139

______________________________________________________________________________

Russ Rew                                           UCAR Unidata Program
address@hidden                              http://www.unidata.ucar.edu