Unidata - To provide the data services, tools, and cyberinfrastructure leadership that advance Earth system science, enhance educational opportunities, and broaden participation. Unidata
         
  advanced  
 

Re: Incorrect netCDF file growth

Greetings,

Glenn Davis has asked me to respond to you regarding the packing of
variables in our implementation of the netCDF abstraction.

First off, a semantic correction is in order.  Contrary to your subject
line, our implementation of netCDF is not incorrect.  Remember that
netCDF is an API and not a data format.  Consequently, only if a netCDF
implementation violates the netCDF API can it be considered incorrect.
Our netCDF implementation does not; therefore, it is correct.

Having said that, it is possible to discuss the efficiency by which an
implementation stores a netCDF object.  We chose to implement the
storage of the contiguous portions of a variable by aways rounding-up
that portion to the nearest multiple of 32 bits.  You have to know a
bit about the structure we chose for implementing netCDF storage (which
is briefly described in section 9 of the netCDF User's Guide) in order
to know what portions of a variable are considered contiguous in our
implementation.  In the context of your examples, however, you should
know that the record dimension breaks the contiguity of each variable
at the record level.  Thus, a byte array of shape (record,2,3) has
(2*3=6 rounded-up to) 8-bytes stored in each record.

As a consequence, the storage space required for a netCDF object
depends on the shape of each variable and whether or not a record
dimension exists -- rather than being a simple scaling relationship.

Regards,
Steve Emmerson           <steve@xxxxxxxxxxxxxxxx>


 
 
  Contact Us     Site Map     Search     Terms and Conditions     Privacy Policy     Participation Policy
 
National Science Foundation (NSF) UCAR Community Programs   Unidata is a member of the UCAR Community Programs, is managed by the University Corporation for Atmospheric Research, and is sponsored by the National Science Foundation.
P.O. Box 3000     Boulder, CO 80307-3000 USA     Tel: 303-497-8643     Fax: 303-497-8690