[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Support #MWB-826938]: NetCDF .cdl to .nc conversion using ncgen

Chad,

By "create a file but don't write all the data", I meant that you define
various
multidimensional variables, but don't supply all the values for them.  For
example, you have:

 netcdf njwxnet_4D {
 dimensions:
         level = 1 ;
         latitude = 1573 ;
         longitude = 1573 ;
         time = UNLIMITED ; // (? currently)
 variables:
         float pressure(time, level, latitude, longitude) ;

so pressure is a 4-dimensional array variable that should have
(1 x 1 x 1573 x 1573) values for each time, which will require about
10MB per time.  Similarly, the other variables windspmaxdir, precip,
..., windsp are each 4-dimensional variables that you have defined to
require 1 x 1573 x 1573 values per time record, abd this all adds up
to about 130 MB.

When I looked at your CDL the first time, I didn't notice the declaration
for the time variable:

        float time(time, level, latitude, longitude) ;

This probably isn't what you intended.  I suspect you meant to define time
as one of the coordinate variables, like level, latitude, and longitude:

        float time(time) ;

If you don't mean for the data to be on a 1573 x 1573 grid, but rather to be
defined for scattered stations that are not on a cartesian grid, I suggest
this sort of structure instead:

netcdf st {
dimensions:
        station = 121 ;    // number of stations
        id_len = 8;        // length of station_id strings
        time = UNLIMITED ; // (? currently)
variables:
        char station_id(station, id_len) ;
        float latitude(station) ;
        float longitude(station) ;
        int time(time) ;

        float pressure(time, station) ;
        float windspmaxdir(time, station) ;
        float precip(time, station) ;
        float windspmax(time, station) ;
        float temperature(time, station) ;
        float winddir(time, station) ;
        float dewpoint(time, station) ;
        float record_date(time, station) ;
        float solarradiation1(time, station) ;
        float solarradiation2(time, station) ;
        float relhumid(time, station) ;
        float windsp(time, station) ;
data:
        pressure = _, _, _, 1012, 1012,
                _, _, _, 1014, _,
 ...
}

If this is the sort of thing you have in mind, ncgen will produce
a 706KB file that only has 1-D and 2-D variables, but it would still
be mostly fill values, as the CDL doesn't provide all the data for
all the times.

When I suggested using "ncgen -x", I hadn't noticed that most of the
space was being taken up by variables for the first record, because
time was declared as a 4-D variable.  I mistakenly thought that not
writing the fill values would save space in the resulting file, so
sorry to send you on a wild goose chase.

--Russ


--Russ





Russ Rew                                         UCAR Unidata Program
address@hidden                     http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: MWB-826938
Department: Support netCDF
Priority: Normal
Status: Closed