Georeferenced Gridded Data Conventions
This document was completed 3/20/95 by Peggy Bruehl of Unidata.
This document was updated 4/21/95 by Peggy Bruehl of Unidata.
Discussion of netCDF Conventions for Georeferenced Gridded Data
NUWG Meeting 2/15/95
Peggy Bruehl
Introduction:
This document is intended to be used in conjunction with sample gridded
data CDL's:
Sample Georeferenced Gridded Data CDL's
History:
At the 2/15 NUWG meeting, I presented a summary of our NUWG gridded
data netCDF conventions. A few unresolved issues regarding these
conventions recently arose during email traffic on the NUWG mailing
list. In addition, I am trying to finish up a project for adding a
netCDF interface for gridded data to GEMPAK, so I had some questions
too. I presented the following summary, in an effort to get 'approval'
by all NUWG committe members present, and to hammer out some solutions
to the remaining issues. We were reasonably successful, and the
summary of our discussion is below. I've added some verbage from
Susan Jesuroga's document "General Conventions approved by NUWG" in
order to flesh out some of the topics discussed.
I'm very open to comments and suggestions about the presentation and
content of the material below. Also, if I've misrepresented any
discussions, topics, or decisions in the following document, please
let me know so I may make any necessary corrections.
NUWG netCDF Conventions for Georeferenced Gridded Data
General Summary
In general, the NUWG has decided to require the following minimum
set of conventions for data exchange:
1) No naming conventions for variables
This allows the author of the netCDF full discretion for naming
variables. It is recognized by the NUWG that truely generic
software for reading and accessing netCDF data can not be
variable name dependent and will require usage of the netCDF
inquire funcitons or will employ some sort of table look-up
algorithm.
2) No type conventions
Exception: time variables must be of type double. This is required
in order to support a variety of time scales and precisions.
3) Variables must have a "long_name" attribute
This convention is for self documentation, and is particularly
important because of our decision not to implement specific
naming conventions. The "long_name" attribute will provide the
human recipient of the netCDF file, a description of the quantity,
regardless of it's name.
4) Where appropriate, variables must have a "units" attribute
The "units" attribute must conform to the UDUNITS library and
be consistent for a given variable. By consistent, we mean
if temperatures are stored in a given variable "temp", and the
units attribute is set to "degF", all the temperatures must
be in Fahrenheit. There are some quantities which do not have
units, and for those a "units" attribute is not required.
The "long_name" attribute is very important in this case.
The UDUNITS distribution is freely available via anonymous FTP
from the Unidata Program Center unidata.ucar.edu in the file
pub/udunits/udunits.tar.Z. Included in this distribution is the
udunits database, udunits.dat, which contains a list of all
acceptable units strings.
5) Other attributes follow netCDF documentation
For example, _FillValue, valid_range, scale_factor, add_offset, etc.
A good example of how these attributes may be implemented is
contained in the document "Conventions for the standardization
of NetCDF files" sponsored by the Cooperative Virtual Data Center
(http://ferret.wrc.noaa.gov/noaa_coop/coop_cdf_profile.html).
Again, we stress that all the NUWG conventions should be considered a
MINIMUM set of conventions for data exchange. The data provider is
always free to add additional fields.
The rest of this document will concern three special topics: time,
vertical coordinates and levels representation, and navigations issues.
Special Topics:
(1) Time
The NUWG agrees that all time fields will be represented such to be
compatible with the UDUNITS library. We agree that time should
always be represented as a double, even if it is not required by
the precision of the data. This gives the most flexibility to
the widest range of data sets.
Initially, our gridded data conventions were designed to support
model output grids. Model output grids generally require two
times to define the grid, a reference time (T0) and a valid
(or forecast) time. Later, we broadened our description to support
not only model output grids with reference and valid times, but
also simple grids requiring only a single time. Here is our
generic convention for representing time in a gridded data file:
double reftime(record); // reference time of the grid
reftime:long_name = "reference time";
reftime:units = "hours since 1992-1-1";
:record = "reftime"
The particular problem associated with storing multiple model output
grids in a single netCDF file is the need to specify time as the
unlimited dimension. This is complicated by having a reference
time and valid time associated with each grid. In our NUWG
conventions, we introduced the concept of a "record" to be the
unlimited dimension. The goal is to define referential attributes
of a variable such as:
double reftime(record); // reference time of the model
reftime:long_name = "reference time";
reftime:units = "hours since 1992-1-1";
double valtime(record); // forecast time (for which model
//is valid)
valtime:long_name = "valid time";
valtime:units = "hours since 1992-1-1";
:record = "reftime, valtime" ; // "dimension attribute" -- means
// reftime and valtime uniquely
// determine record
This indicates that the "nth" record is defined by the "nth" value
in both the "valtime" and "reftime" variables. They contain
the actual times for the grid. This allows us to refer to both
the valid and reference time as a single unlimited dimension.
This convention, when applied to grids requiring only a single time,
results in the time convention shown above.
Summary of conventions:
1) Time variables are always type double
2) Time variables are indexed by another variable (can
be the unlimited dimension)
3) The names given to the time variables and the indexing variables
are not subject to convention
4) This convention supports grids requiring any number of times
to fully describe the data
Special Topics:
(2) Vertical Coordinate Systems/Levels
The simple case of a grid defined in a single vertical coordinate
system, by a list of set levels, can be represented as:
dimensions:
z = 20;
variables
float T(record, level, y, x) ;
T:long_name = "temperature" ;
T:units = "degK" ;
float level(level) ;
level:long_name = "level" ;
level:units = "hectopascals" ;
data:
level = 1000, 950, 900, 850, 800, 750, 700, 650, 600, 550, ...
This convention is representative of the way gridded data defined
in traditional vertical coordinate systems (pressure, potential
temperature, height, etc...) should be stored.
Occasionally, a grid may require two levels to fully specify the
data. In this case, the referential concept introduced in the
time conventions can be useful. An example of this is a grid defining a
quantity over a layer, such as the boundary layer.
dimensions:
bndry = 3;
variables:
float T-bndry(record, bndry, y, x) ;
T-bndry:long_name = "temperature in boundary layer" ;
T-bndry:units = "degK" ;
:bndry = "bndry_bot, bndry_top"; // (bndry_bot, bndry_top) uniquely
// determine bndry
float bndry_bot(bndry) ;
bndry_bot:long_name = "bottom level of boundary layer between 2
levels at specified pressure differences from ground to levels";
bndry_bot:units = "hectopascals";
float bndry_top(bndry) ;
bndry_top:long_name = "top level of boundary layer between 2
levels at specified pressure differences from ground to levels";
bndry_top:units = "hectopascals";
data:
bndry_bot = 0, 60, 150;
bndry_top = 30, 90, 180;
Finally, the referential concept is also useful for describing hybrid
grids. Hybrid grids contain a relative vertical coordinate "z". Here's
an example: the value of the u component of the horizontal wind at a
given grid point may be described by a vertical level in the pressure
coordinate, the value of which is stored in variable "p", or it may be
described by a vertical level in the virtual potential temperature
coordinate, the value of which is stored in variable "vpt". The
referential attribute is used as follows:
float u(record, z, x, y );
u:z = "vpt, p";
u:long_name = "u component of horizontal wind";
u:units = "meters/second";
This specifies that for grid point u(1,2,1,1), the value of the
"z" dimension (vertical level) can be found in p(1,2,1,1) or in
vpt(1,2,1,1).
Summary of conventions:
1) When necessary, a referential variable can be used as an index
into associated variables
2) This referential indexing is indicated by a variable or global
attribute with the same name as the dimension
Special Topics:
(3) Navigation
Of all the special topics, the conventions concerning navigation are
the least mature. Thus far, we have agreed that the navigation information
associated with a grid will be stored in a suite of navigation variables.
These variables are grouped together into a psuedo-structure. Each element
of the pseudo-structure is dimenensioned by the "nav" dimension. In addition,
there is a variable, "nav_model", which describes the source of the navigation
parameterization and provides the context in which to interpret all the other
variable names.
For example:
char nav_model(nav, nav_len) ; // navigation parameterization
nav_model:long_name = "navigation model name";
...
For GRIB-centric data, its value is:
nav_model = "GRIB1" ;
but for parameterizations based on the Federal Geographic Data Committee
Content Standards for Digital Geospatial Metadata it could be:
nav_model = "FGDC-1994" ;
and for parameterizations based on the geo-TIFF model, it might be
nav_model = "geo-TIFF version 1" ;
Notice that it is possible to use multiple navigation parameterizations
within the same netCDF file with this mechanism.
We haven't yet agreed on how many values of nav_model our generic
applications should support, but we want to support at least "GRIB1".
We will be discussing this in upcoming meetings.
For the case of "nav_model" = GRIB1, we take the navigation parameterization
from the GRIB Edition 1 document by John Stackpole in the section on the
GDS (Grid Description Section) octets 7-44. This document is available
as text (abbreviated version) and as a PostScript file.
The actual set of variables needed for any given navigation parameterization
will depend on that navigation. For example, the variables needed to
describe a polar stereographic grid are different than the variables needed to
describe a simple lat/lon grid.
In the case of GRIB Edition 1 navigation parameterizations, each suite of
navigation variables must contain a numeric ID containing the grid
identification number, and an indication of the originating center,
both assigned by the GRIB Edition 1 document. Missing values may be
used if the particular grid is not described by the GRIB document.
For example, the navigation variables "grid_number" and "center_id"
may be used.
Each variable that is defined on the grid must have the "navigation"
variable attribute associated with it. The string defined in this
attribute gives the name of the dimension by which all navigation
variables are dimensioned. In this way, the "navigation" groups all the
navigaton variables together (in the same sense that a structure groups
quantities of varying types together). The "navigation" attribute
also indicates which variables (temperature, pressure, etc.) in a
netCDF file are defined on the grid.
Here is an example taken from the CDL used by Unidata to hold the
RUC MAPS model. The RUC MAPS model output is defined on a Lambert
Conic Conformal grid projection. Note that the ordering of the
dimensions which define the grid variables (x,y) or (lat,lon) are
not subject to convention, nor are the names assigned to these dimensions.
However, the names assigned to these dimensions are defined by the
navigation variables "x_dim" and "y_dim".
dimensions:
x = 93;
y = 65;
nav = 1; // For navigation. Variables that use
// this dimension define a mapping between
// (x,y) indices and (lat, lon) coords.
nav_len = 100 ; // Max length for navigation character strings
variables:
float Z(record, level, y, x) ;
Z:long_name = "geopotential height" ;
Z:units = "gp m" ;
Z:navigation = "nav" ;
char nav_model(nav, nav_len) ; // navigation parameterization
nav_model:long_name = "navigation model name";
char grid_type(nav, nav_len) ;
grid_type:long_name = "GRIB-1 grid type" ;
char grid_name(nav, nav_len) ;
grid_name:long_name = "grid name" ;
short grid_number(nav) ;
grid_number:long_name = "GRIB-1 catalogued grid number" ;
long center_id(nav) ;
center_id:long_name = "WMO centers table";
char earth_shape(nav, nav_len) ;
earth_shape:long_name = "assumed earth shape" ;
char x_dim(nav, nav_len) ;
x_dim:long_name = "x dimension name" ;
char y_dim(nav, nav_len) ;
y_dim:long_name = "y dimension name" ;
short Nx(nav) ;
Nx:long_name = "number of points along x-axis" ;
short Ny(nav) ;
Ny:long_name = "number of points along y-axis" ;
float La1(nav) ;
La1:long_name = "latitude of first grid point" ;
La1:units = "degrees_north" ;
float Lo1(nav) ;
Lo1:long_name = "longitude of first grid point" ;
Lo1:units = "degrees_east" ;
byte ResCompFlag(nav, rescompdim) ;
ResCompFlag:long_name = "resolution and component flags" ;
float Lov(nav) ;
Lov:long_name = "orientation of the grid" ;
Lov:units = "degrees_east" ;
float Dx(nav) ;
Dx:long_name = "x-direction grid length" ;
Dx:units = "km" ;
float Dy(nav) ;
Dy:long_name = "y-direction grid length" ;
Dy:units = "km" ;
byte ProjFlag(nav) ;
ProjFlag:long_name = "projection center flag" ;
float Latin1(nav) ;
Latin1:long_name = "first intersecting latitude" ;
Latin1:units = "degrees_north" ;
float Latin2(nav) ;
Latin2:long_name = "second intersecting latitude" ;
Latin2:units = "degrees_north" ;
float Dj(nav) ;
Dj:long_name = "j grid increment" ;
Dj:units = "degrees" ;
data:
nav_model = "GRIB1" ;
// Navigation
grid_type = "GRIB1 Lambert conformal" ;
grid_name = "AWIPS grid 211: Regional CONUS" ;
grid_number = 211 ;
center_id = 7 ;
earth_shape = "oblate spheroid (IAU 1965)" ;
x_dim = "x";
y_dim = "y";
Nx = 93;
Ny = 65;
La1 = 12.190;
Lo1 = 226.541;
ResCompFlag = 0,0,0,0,1,0,0,0;
Lov = 265.0;
Dx = 81.2705;
Dy = 81.2705;
ProjFlag = 0;
Latin1 = 25.0;
Latin2 = 25.0;
Summary of Conventions:
1) Navigation information is stored in variables and dimensioned
by the value of the variable attribute "navigation"
2) All grid variables have the "navigation" attribute
3) The "nav_model" variable defines a string containing a key
word which indicates the source of the navigation parameterization.
Thus far, the NUWG has officially accepted "GRIB1" as the key word
for navigation parameterizations defined in the GRIB edition 1 document
GDS octets 7-44 "Grid description" (Table C). This document defines
the content of the navigation parameterization, not necessarily the
explicit variable names to be used.
4) For GRIB1 parameterizations, a numeric ID listing the Grid Identification
number and an originating center ID from the GRIB edition 1 document must
be included in the navigation variables. Missing data values are OK for
grids not described by a GRIB document.
5) Ordering or naming of grid dimensions not subject to convention.
Dimensions defining grid variables defined by "x_dim" and "y_dim"
navigation variables
