[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #XFU-202199]: netcdf files



Hi Amanda,

> It is possible to write new data into an existing file while keeping the
> same structure (only changing the dimensions)?

Yes, but there are restrictions with netCDF-3 that are not a problem when
using netCDF-4.

With netCDF-3 (or using the netCDF-4 classic data model for compatibility),
new data can be appended along only one dimension, called the "unlimited
dimension" or "record dimension".  Multiple variables can share that 
dimension, but it must be the most slowly varying dimension in all such
variables.  A good example would be the "time" dimension, and you can 
efficiently append data for new times to all the variables that depend on 
time.

With the netCDF-3 data model, no other dimensions can change.  You can add
new dimensions, variables, and attributes to a file, but this may be 
inefficient for large files, because all the metadata is stored in a header
at the beginning of the file, and expanding the header to make room for more
metadata results in the library moving the data by copying it to a different
location in the file.

With the netCDF-4 data model, multiple unlimited dimensions may be defined.
Variables in different groups may use different time dimensions that grow at
different rates.  Variables may also use multiple time dimensions.  An 
example where this might be useful is having two unlimited dimensions, "time"
and "station", where a station is an observing location.  Observing locations
may change as new locations are added, and observations may still be appended
for all locations at the same time.  You can also define multiple time 
dimensions to model a situation in which different stations have observations 
at different times.

With the netCDF-4 data model and format, new variables, dimensions, and 
attributes may be added efficiently, without moving data.  However, only
dimensions initially defined as unlimited may grow; fixed dimensions cannot
change size.

--Russ

> Le 25/04/2013 19:37, Unidata netCDF Support a écrit :
> > Hi Amanda,
> >
> >> I am a researcher at meteo-france and i am contacting you to get some
> >> advice about generating netcdf files. I would like to generate a netcdf
> >> output file containing at the same time a map of objects which have
> >> single value attributes and also an attribute which is a array with a
> >> variable length. Is it possible to generate this kind of structure? If
> >> yes, how can I do it?
> > By "map of objects which have single value attributes" I'm assuming you
> > mean a collection of objects, possibly of different types, that each have
> > a named attribute of the same name and type.  If each of the objects is
> > represented as a netCDF variable accessible by its name, then such a data
> > type can be represented in a netCDF file as a collection of netCDF 
> > variables.
> > However, this is not a very convenient representation, as the objects each
> > must have distinct names, and cannot be accessed by their attribute values,
> > for example.
> >
> > If you are using netCDF-3 with its simple data model and "classic" data
> > format, you won't be able to easily represent either of those kinds of data
> > structures.  However, using the netCDF-4 enhanced data model, with its
> > support for user-defined types, variable-length types, and groups, I think
> > it is possible to represent what you want.  Alternatively, you may want to
> > also look at HDF5, which supports a "reference" type.  An array of 
> > references
> > pointing to other data objects may but what you want for the map data 
> > structure,
> > and netCDF-4 does not include support for HDF5 references.
> >
> > --Russ
> >
> >
> > Russ Rew                                         UCAR Unidata Program
> > address@hidden                      http://www.unidata.ucar.edu
> >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: XFU-202199
> > Department: Support netCDF
> > Priority: Normal
> > Status: Closed
> >
> >
> 
> --
> Amanda Gounou
> Météo France
> DPREVI/PI
> address@hidden
> 
> 

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: XFU-202199
Department: Support netCDF
Priority: Normal
Status: Closed