Here's some additional issues concerning the gridded data NetCDF conventions. There are email messages about 1) the ordering of the X/Y coordinates of grid variables; 2) the representation of levels (single, multiple, and hybrid B); 3) the representation of the navigation variables; and 4) a couple of older messages laying out how to implement grids described by only one time (rather than a reftime and valtime). Peggy ____________________________________________________________________________ Peggy Bruehl Unidata Program Center peggy@unidata.ucar.edu UCAR, PO Box 3000 (303) 497-8641 Boulder, CO 80307-3000 ---------------------------------------------------------------------------- Unidata WWW Service URL http://www.unidata.ucar.edu/ GEMPAK Home Page URL http://www.unidata.ucar.edu/packages/gempak/index.html **************************************************************************** Organization: Unidata From: Peggy Bruehl Date: Thu, 2 Feb 1995 11:22:02 -0700 To: davis@fsl.noaa.gov, nuwg@comet.ucar.edu Subject: Grid netCDF Conventions Cc: peggy@unidata.ucar.edu, support-gempak@unidata.ucar.edu Darien, Susan Jesuroga asked me if I would answer your question, since I've been actively thinking about the grid netCDF conventions. I am CC:'ing the nuwg mailing list as well. >Date: Wed, 1 Feb 1995 17:59:16 +0000 (GMT) >From: Darien Davis >To: jesuroga@comet.ucar.edu >Subject: clarification of grid netcdf x,y,z ordering > >Linda Wharton has been looking through the cdl specifications for grids and >are concerned by the ordering on variables of z,x,y. We looked at some of >Jeff Pedigo's actual CDL files and the ordering is z,y,x...this is much >preferred. Is it possible that this may be a typo? To write out a 3-d slab >would be somewhat difficult with this ordering. Can it be changed? > >Thanks for your enlightening comments on this. > >Darien >davis@fsl.noaa.gov We have a sample CDL available on our gopher and WWW servers that Jeff Pedigo put together. The reference is ftp://ftp.unidata.ucar.edu/pub/netcdf/Conventions/NUWG/grid.cdl.draft In addition, Unidata is currently working on rewritting our CDLs to conform to the NUWG conventions. Russ Rew has placed our CDL for the RUC/MAPS model in the same directory (ftp://ftp.unidata.ucar.edu/pub/netcdf/Conventions/NUWG/ruc.cdl). Please note that Russ is still working on this CDL and it is subject to change. With respect to your question about the ordering of x and y, there is no NUWG convention about the order of the grid indices, so you could, in principle, use any or all of the orderings (zxy, zyx, xyz, ...). It's probably a good idea to pick one and use it consistently, but if it should change, generic reader software should not break. In looking at Jeff's CDL, a couple other issues cropped up. 1) Jeff uses an attribute called "z_dim" to describe the vertical coordinate for each variable. I think the reasoning behind this was to support hybrid levels (?): float cp(record, z, x, y); cp:record_dim = "valtime, reftime"; cp:_Navigation_var = "nav"; cp:z_dim = "vpt, p"; cp:long_name = "condensation pressure"; cp:units = "pascals"; cp:valid_range = 100.f, 600.f; cp:_FillValue = -9999.f; The Unidata CDL's have implemented a different philosophy in order to support multiple vertical coordinates in the same data file: float RH(record, level, y, x) ; RH:long_name = "relative humidity" ; RH:units = "percent" ; RH:valid_range = -10.f, 110.f ; RH:_FillValue = -9999.f ; RH:navigation_dim = "nav" ; float v-bndry(record, bndry, y, x) ; v-bndry:long_name = "v-component of wind in boundary layer" ; v-bndry:units = "meters/second" ; v-bndry:valid_range = -200.f, 200.f ; v-bndry:_FillValue = -9999.f ; v-bndry:navigation_dim = "nav" ; "level" is a variable containing pressure levels, and "bndry" is an index into two varibles, bndry_bot and bndry_top. (Please see the Unidata RUC CDL for details). As the author of the GEMPAK netCDF interface, I am willing (I think) to support both implementations, as long as I can count on an attribute named "z_dim" to point me to the correct vertical coordinate for any given variable. In any case, I'd like to open to discussion about this topic among interested parties. 2) There is a significant difference between Jeff's CDL and Unidata's CDL represents the navigation information. I think I remember the NUWG group deciding on a using a suite of navigation variables, rather than one "nav" variable with lots of attributes. Jeff, do you remember differently? All interested parties should take a look at both representations. (Note that the current GEMPAK-netCDF requires a varible that contains a unique "GRIB-1 catalogued grid number". The name of the variable, grid_number in our CDL, can be anything, but I must have some number to index into my table of grid navigations.) Is it time to all get together and discuss these final issues? Also, is it time to start thinking about a set of conventions for satellite or radar data? Peggy ____________________________________________________________________________ Peggy Bruehl Unidata Program Center peggy@unidata.ucar.edu UCAR, PO Box 3000 (303) 497-8641 Boulder, CO 80307-3000 ---------------------------------------------------------------------------- Unidata WWW Service URL http://www.unidata.ucar.edu/ GEMPAK Home Page URL http://www.unidata.ucar.edu/packages/gempak/index.html **************************************************************************** Organization: FSL To: Peggy Bruehl Cc: davis@fsl.noaa.gov, nuwg@comet.ucar.edu, support-gempak@unidata.ucar.edu Subject: Re: Grid netCDF Conventions Date: Thu, 02 Feb 1995 21:55:31 +0000 From: "Jeff Pedigo" Hi Peggy, I think you have summed up the Gridded NUWG work well. There are a few explanations (and an excuse or two) that I should add: >1) Jeff uses an attribute called "z_dim" to describe the vertical >coordinate for each variable. I think the reasoning behind this >was to support hybrid levels (?): > > float cp(record, z, x, y); > cp:record_dim = "valtime, reftime"; > cp:_Navigation_var = "nav"; > cp:z_dim = "vpt, p"; > cp:long_name = "condensation pressure"; > cp:units = "pascals"; > cp:valid_range = 100.f, 600.f; > cp:_FillValue = -9999.f; > >The Unidata CDL's have implemented a different philosophy in order >to support multiple vertical coordinates in the same data file: > > float RH(record, level, y, x) ; > RH:long_name = "relative humidity" ; > RH:units = "percent" ; > RH:valid_range = -10.f, 110.f ; > RH:_FillValue = -9999.f ; > RH:navigation_dim = "nav" ; > > float v-bndry(record, bndry, y, x) ; > v-bndry:long_name = "v-component of wind in boundary layer" ; > v-bndry:units = "meters/second" ; > v-bndry:valid_range = -200.f, 200.f ; > v-bndry:_FillValue = -9999.f ; > v-bndry:navigation_dim = "nav" ; > >"level" is a variable containing pressure levels, and "bndry" is an index >into two varibles, bndry_bot and bndry_top. (Please see the Unidata RUC CDL >for details). > Apples .vs. oranges comparison here. Your technique works for RUC that has been interpolated to isobaric levels. We do the same thing with isobaric levels. I remember this as the convention for fixed levels. It doesn't work for Hybrid-B levels used within FSL. Hybrid-B was a "worst case" scenerio where the vertical dimension was dependent on the values of other model variables. When we were working on the conventions I presented Hybrid-B as an example of using attributes to describe the meaning (or interpretation) of a dimension. I don't think that NMC will be distributing RUC Hybrid-B data operationally for some time. At least I haven't heard of any plans to do that. Some of the folks at FSL use Hybrid-B RUC received from NMC but that is a different animal (or fruit) from the AWIPS RUC that Unidata is storing (use Peggy's CDL for AWIPS RUC). >2) There is a significant difference between Jeff's CDL and Unidata's >CDL represents the navigation information. I think I remember the >NUWG group deciding on a using a suite of navigation variables, rather >than one "nav" variable with lots of attributes. Jeff, do you remember >differently? As I recall it there was considerable philosophical evolution occuring in relation to grid navigation while we were meeting. I think your "multiple variable" approach was the one we finally developed a consensus on. Unfortunately, I was developing gridded netCDF storage programs to meet some tight deadlines. I developed some software philosophy during an intermediate evolutionary phase. My "missing link" navigation should be corrected. With the best intentions of going back and fixing things later... here we are. I like your technique better. I think it is the conventional method that we agreed upon. >All interested parties should take a look at both >representations. (Note that the current GEMPAK-netCDF requires >a varible that contains a unique "GRIB-1 catalogued grid number". >The name of the variable, grid_number in our CDL, can be anything, >but I must have some number to index into my table of grid navigations.) > >Is it time to all get together and discuss these final issues? > Soon I hope. It's long overdue. >Also, is it time to start thinking about a set of conventions for >satellite or radar data? > We do need to get moving again. Best regards, Jeff Organization: Unidata From: Peggy Bruehl Date: Thu, 13 Jan 1994 13:03:00 -0700 To: nuwg@comet.ucar.edu Subject: one vs two grid times Cc: pegggy@unidata.ucar.edu NUWG folk, At yesterday's NUWG meeting, we discussed my question about how to specify the time on a grid that is not associated with a forecast model. I would like to summarize my ideas and propose a convention. In our original meetings to discuss the time convention on grids, we only considered grids that resulted from model runs. These grids, by definiton, have two times associated with them, the time of the model run, and the time for which the forecast is valid. Here's how we decided that these times should be represented in a netCDF file: double reftime(record); // reference time of the model reftime:long_name = "reference time"; reftime:units = "hours since 1992-1-1"; double valtime(record); // forecast time (for which model is valid) valtime:long_name = "valid time"; valtime:units = "hours since 1992-1-1"; (Take note that while the NUWG may recommend the variable names "reftime" and "valtime", there is not necessarily a convention for the name. The variables could be named "time1", and "time2".) I contend that the above example (a grid with two associated times) is a special case of the more general grid category. In a general sense, gridded data is simply data arranged at regular intersections between sets of parallel lines (in a given coordinate system). There is nothing inherent in a grid which requires two times to describe it. The NUWG group is very familiar with the forecast grid case, because that's the only kind of grid that we've discussed in our meetings. In retrospect, I believe we should have discussed grids in the general sense first, and then refined our conventions to include a recommendation on the special case of forecast grids. In my previous note, I talked about gridded data that resulted from an objective analysis of irregularly spaced data. My intention was for an objective analysis grid to serve as an example of the general case of gridded data. So, here's my proposal. As our major convention, let's take a step back from forecast grids and consider the more general grid case where only a single time is required to temporally specify the grid. double reftime(record); // reference time of the grid reftime:long_name = "reference time"; reftime:units = "hours since 1992-1-1"; Whether we suggest that the name be "reftime", "time", or "tImE" (that's for you FSL people) is irrelevant. The important point is that there is ONE time used to specify the grid. Now as a special case in our gridded data conventions, let's assert that gridded data originating from *forecast* models have a second time used to specify the grid (i.e. "valtime", or "time2"). This makes sense, since having two times associated with a given piece of data implies the "forecast" nature of the data, whereas a single time is simply the time associated with the data. Here is a practical example of how we would implement these two levels of convention *in the same file*. In preparing input to computational forecast models, high frequency components are usually smoothed out of gridded data before it is used to initialize the model run. I am curious about the ETA model, and I would like to compare the temperature grid used to initialize the model with an objective analysis of observed temperatures over a specific geographical area. I choose a grid coincident with the ETA model output, I do the objective analysis and I write the data out into netCDF. I also copy the ETA model temperature intialization grid into my netCDF file for ease of comparison. Here's part of the CDL: double reftime(record); // reference time of the grid reftime:long_name = "reference time"; reftime:units = "hours since 1992-1-1"; reftime:_FillValue = -9999.f; double valtime(record); // forecast time (for which model is valid) valtime:long_name = "valid time"; valtime:units = "hours since 1992-1-1"; valtime:_FillValue = -9999.f; :record = "reftime, valtime"; float temp(record, level, lat, lon); temp:long_name = "temperature"; temp:units = "degrees celcius"' temp:_FillValue = -9999.f; temp:navigation_dim = "nav"; Now, the grid resulting from my objective analysis will be stored in the "temp" variable, with a given record number that corresponds to an appropriate reftime, and a FillValue in valtime. The ETA model intialization grid will also be stored in the "temp" variable, with the next record number corresponding to the same number of hours for reftime and valtime. This example does not force us to assign reference and valid times to the grid that doesn't require two times, and it allows us to distinquish between a general case grid and the special case forecast grid. That's my proposal, and I welcome questions and comments. Peggy ________________________________________________________________________ Peggy Bruehl Unidata Program Center peggy@unidata.ucar.edu UCAR, PO Box 3000 (303) 497-8641 Boulder, CO 80307-3000 ---------------------------------------------------------------------------- Unidata Gopher Service gopher.unidata.ucar.edu (port 70) **************************************************************************** Full-Name: Russ Rew To: Peggy Bruehl Subject: Re: one vs two grid times Organization: UCAR Unidata Program Date: Tue, 18 Jan 1994 09:25:39 -0700 From: Russ Rew Peggy, > At yesterday's NUWG meeting, we discussed my question about how to specify > the time on a grid that is not associated with a forecast model. I would > like to summarize my ideas and propose a convention. ... > That's my proposal, and I welcome questions and comments. Your proposal seems fine to me. We need to make sure that a function can be implemented that will return the record number, given the value of the record attribute and values for the corresponding time variable(s). This should be possible even if one of the times is the FillValue. --Russ