Organization: FSL From: "Andrew Sundelin" Date: Tue, 25 Apr 1995 13:02:57 -0600 To: nuwg@comet.ucar.edu Subject: Navigation Information Query Hi NUWGies, I am working up a CDL for NEXRAD Level II data and have come across an interesting problem for representing the navigation of the data in a way consistent with our conventions. To help illustrate the problem the following CDL excerpt has been simplified: // everything starts out normally byte Z(elevs, radials, refs); Z:long_name = "Reflectivity"; Z:units = "dBZ"; Z:navigation = "navZ"; byte V(elevs, radials, vels); V:long_name = "Velocity"; V:units = "meters / second"; V:navigation = "navV"; // This navigation information is specific to Z or V // so the convention works okay -- rangeZ is dimensioned by navZ and // rangeV is dimensioned by navV float rangeZ(navZ, elevs, refs); rangeZ:long_name = "Radial reflectivity range"; rangeZ:units = "meters"; float rangeV(navV, elevs, vels); rangeV:long_name = "Radial velocity range"; rangeV:units = "meters"; // Then all of the following information applies to both Z and V. // So to stick to the convention should I dimension these by navV or // by navZ or by both or something completely different or what exactly? float radialAzim(nav?, elevs, radials); radialAzim:long_name = "Radial azimuth angle"; radialAzim:units = "degrees"; float radialElev(nav?, elevs, radials); radialElev:long_name = "Radial elevation angle"; radialElev:units = "degrees"; float siteLat(nav?); siteLat:long_name = "Latitude of site"; siteLat:units = "degrees_north"; float siteLon(nav?); siteLon:long_name = "Longitude of site"; siteLon:units = "degrees_east"; float siteAlt(nav?); siteAlt:long_name = "Altitude of site above mean sea level"; siteAlt:units = "meters"; Hopefully, there is an easy answer to this that is just eluding me. Does anybody have any ideas? Thanks, Andrew Sundelin sundelin@fsl.noaa.gov Organization: UCAR Unidata Program To: "Andrew Sundelin" From: Russ Rew Cc: nuwg@comet.ucar.edu Date: Tue, 25 Apr 1995 14:32:04 -0600 Subject: Re: Navigation Information Query Andrew, > I am working up a CDL for NEXRAD Level II data and have come across an > interesting problem for representing the navigation of the data in a way > consistent with our conventions. This is an interesting problem that we hadn't considered. > To help illustrate the problem the following CDL excerpt has been > simplified: > > // everything starts out normally > > byte Z(elevs, radials, refs); > Z:long_name = "Reflectivity"; > Z:units = "dBZ"; > Z:navigation = "navZ"; > > byte V(elevs, radials, vels); > V:long_name = "Velocity"; > V:units = "meters / second"; > V:navigation = "navV"; > > // This navigation information is specific to Z or V > // so the convention works okay -- rangeZ is dimensioned by navZ and > // rangeV is dimensioned by navV > > float rangeZ(navZ, elevs, refs); > rangeZ:long_name = "Radial reflectivity range"; > rangeZ:units = "meters"; > > float rangeV(navV, elevs, vels); > rangeV:long_name = "Radial velocity range"; > rangeV:units = "meters"; > > // Then all of the following information applies to both Z and V. > // So to stick to the convention should I dimension these by navV or > // by navZ or by both or something completely different or what exactly? > > float radialAzim(nav?, elevs, radials); > radialAzim:long_name = "Radial azimuth angle"; > radialAzim:units = "degrees"; > > float radialElev(nav?, elevs, radials); > radialElev:long_name = "Radial elevation angle"; > radialElev:units = "degrees"; > > float siteLat(nav?); > siteLat:long_name = "Latitude of site"; > siteLat:units = "degrees_north"; > > float siteLon(nav?); > siteLon:long_name = "Longitude of site"; > siteLon:units = "degrees_east"; > > float siteAlt(nav?); > siteAlt:long_name = "Altitude of site above mean sea level"; > siteAlt:units = "meters"; > > > > Hopefully, there is an easy answer to this that is just eluding me. > > Does anybody have any ideas? I think in this case you have to use a third navigation variable, say "nav", for all the navigation information common to the two navigations "navZ" and "navV" and use other mechanisms to represent the relations between "nav", "navZ", and "navV". What we have here is an inheritance hierarchy. "navZ" inherits everything from "nav" and in addition adds some "navZ"-specific parameters like "rangeZ". Similarly, "navV" inherits from "nav" and adds additional parameters like "rangeV". A general solution requires that there be some way to express inheritance hierarchies like: nav / \ / \ navZ navV This would be easy if "nav", "navZ", and "navV" were variables (just use a "parent_var" or "inherits_from_var" attribute that names the parent variable), but there is no obvious way to represent such relations among netCDF dimensions. A netCDF dimension only has a size and a name. One possibility is using a convention to describe an arbitrary inheritance hierarchy with attributes of a single "_Inheritance" variable, such as: variables: ... int _Inheritance; // to represent an inheritance hierarchy _Inheritance:navZ = "nav"; // navZ inherits from nav _Inheritance:navV = "nav"; // navV inherits from nav This has some advantages: + It is quite general, permitting arbitrary inheritance hierarchies and even multiple inheritance, using lists of dimensions. + It doesn't take much space. + It permits multiple hierarchies in the same netCDF file. and some disadvantages: - It requires a new reserved variable name, such as "_Inheritance". - It is a new convention for a variable-specific attribute with the same name as a dimension. I don't think it collides with our other convention, but it might be confusing. - It is possible to express nonsensical inheritance "hierarchies" by representing loops, etc. - It may be overkill if the problem it solves is rarely encountered. There are other possibilities, such as just repeating all the common information using different variable names, but I don't see any easy answers. --Russ Organization: FSL From: "Andrew Sundelin" Date: Wed, 26 Apr 1995 09:31:52 -0600 To: nuwg@comet.ucar.edu Subject: Re: Navigation Information Query Hi NUWGies, One possible, if ugly solution (that doesn't generalize to other problems as nicely as Russ' suggestion) would be to represent the following: > // Then all of the following information applies to both Z and V. > // So to stick to the convention should I dimension these by navV or > // by navZ or by both or something completely different or what exactly? > > float radialAzim(nav?, elevs, radials); > radialAzim:long_name = "Radial azimuth angle"; > radialAzim:units = "degrees"; > > float radialElev(nav?, elevs, radials); > radialElev:long_name = "Radial elevation angle"; > radialElev:units = "degrees"; etc. as float radialAzim(navZ, navV, elevs, radials); radialAzim:long_name = "Radial azimuth angle"; radialAzim:units = "degrees"; float radialElev(navZ, navV, elevs, radials); radialElev:long_name = "Radial elevation angle"; radialElev:units = "degrees"; etc. If you got into more navigation variables this would obviously get uglier and uglier, but I don't find it too irritating for this simple case. In fact, I would argue that if you're getting into a situation where you are using 3 or more navigation variables that your data would probably be better off in separate netCDF files. Also, to some degree this dodges the bullet of coming up with new conventions. On the other hand, I think Russ' suggestion quite possibly solves some other problems or can be applied to other things that we haven't thought of yet. As for space issues, I don't think its particularly wasteful, but I really don't know if it is or not. Any Unidatans have any comment on that? This solution is so brain numbingly simple that I hesitate to present it, but what do you think? Andrew Sundelin sundelin@fsl.noaa.gov Organization: NCAR ACD From: caron@dilbert.acd.ucar.edu (John Caron) Subject: Re: Navigation Information Query To: nuwg@comet.ucar.edu Date: Wed, 26 Apr 1995 10:03:58 -0600 (MDT) Heres how I would think about this problem using the principle that the dimensions describe the grid via coordinate variables and coordinate reference variables, and the "nav" structure defines the mapping of the grid to world coordinates. For reference, here is a repeat of my proposal for "Coordinate References", which are an extension of "Coordinate Variables". (actually I would like to reword it some to make this case more explicitly covered, as I wasnt thinking of non-indeependent coordinate systems at the time). Coordinate Variables A variable with the same name as a dimension is a "coordinate variable", and its data values are the coordinate values for that dimension. The variable must be indexed by the dimension. Coordinate Reference Coordinate systems cannot always be described by coordinate variables which are single valued and have the same cardinality as a dimension. A "coordinate reference" is one or more variables that also describe a coordinate system, and may have different sizes than the dimension it describes. A "coordinate reference variable" is a variable whose data values describe the coordinate values for a dimension. Its cardinality is a function of the size of the dimension. A coordinate reference is defined by an attribute with the same name as a dimension. The value of the attribute is the name(s) of coordinate reference variable(s). An attribute that is global becomes the default description for that dimension in the file, and may be overridden by a variable's attribute. > // everything starts out normally > > byte Z(elevs, radials, refs); > Z:long_name = "Reflectivity"; > Z:units = "dBZ"; > Z:navigation = "navZ"; > > byte V(elevs, radials, vels); > V:long_name = "Velocity"; > V:units = "meters / second"; > V:navigation = "navV"; So we have a 3D grid. The two variables share two of the dimensions. The third is different. In the simplest case, we define 4 coordinate variables: float elevs(elevs); elevs:long_name = "elevation angles"; elevs:units = "radians"; float radials(radials); radials:long_name = "radial angles"; // or something radials:units = "radians"; float refs(refs); refs:long_name = "reflectivity range gates"; // or something refs:units = "meters"; float vels(vels); vels:long_name = "velocity range gates"; // or something vels:units = "meters"; This assumes that the coordinates they are independent from each other. In the post, it appears they are not independent, so how do we handle that case? The first two coord. vars are probably correct: float elevs(elevs); elevs:long_name = "elevation angles"; elevs:units = "radians"; float radials(radials); radials:long_name = "radial angles"; // or something radials:units = "radians"; And we need a way to specify the other coord as a function of elev angle: float rangeZ(elevs, refs); rangeZ:long_name = "Radial reflectivity range"; rangeZ:units = "meters"; float rangeV(elevs, vels); rangeV:long_name = "Radial velocity range"; rangeV:units = "meters"; So heres the coordinate reference, defined as a global or variable-specific attribute: : refs = "rangeZ"; : vels = "rangeV"; Now, its the job of the nav structure to map the grid coordinate system, (defined by the coord vars. "elevs", "radials", and the coord reference vars. "rangeZ", "rangeV") to world coords, say (lat, lon, altitude). So we imagine we have a tranformation function t(azi, zen, rho) -> (lat, lon, z). What does it need to know? Probably just the lat, lon position of the radar. Then you just feed it azi = radial(i), zen = elevs(j), and rho = rangeZ(j,k) or rangeV(j,k). Note t() can be a very general function in this way. So you just need, for the nav structure: char nav_model(nav, nav_len) ; // navigation parameterization nav_model:long_name = "navigation model name"; char projection_type(nav, nav_len) ; float siteLat(nav); siteLat:long_name = "Latitude of site"; siteLat:units = "degrees_north"; float siteLon(nav); siteLon:long_name = "Longitude of site"; siteLon:units = "degrees_east"; float siteAlt(nav); siteAlt:long_name = "Altitude of site above mean sea level"; siteAlt:units = "meters"; data: nav_model = "Unidata projection library" ; projection_type = "radar spherical coordinates"; Now, if there were some parameters to the projection function that depended on whether you were transforming the reflectivity or the velocity, you might come back to either seperate nav structures (navZ and navV as in the original post, or to Russ' heirarchy. In this case, I'm guessing there isn't. In any case, the advantage of this approach might be in reducing the complexity of the nav structure. The overall complexity of the problem is not obviously reduced, other than obviating the need for 2 nav structures or a heirarchy of nav structures. I would argue that the real advantage, however, is making explicit the seperation of the grid description from the world mapping. Each is straightforward in itself, and a little bit confusing munged together, especially from the perspective of an automatic file reader. Looking at Stackpole's summary of GRIB grids, I can see how tempting it is to use their system. After all, there it all is, nicely packaged and described. All the work is apparently done for you. Its glaring weakness is that it describes very specific grids, rather than a family of grids based on user-settable parameters. Such a family would be exactly what I mean by a projection function. Note that the projection function maps any x,y in the projection plane to lat,lon, so it can handle any grid topology. The number of projections in common use is reasonably small, but the number of different possible grids is infinite. I am going to continue these thoughts more abstractly in the email thread to "Georeferenced gridded data conventions", but it would probably be useful to include this example in that thread. To: caron@dilbert.acd.ucar.edu (John Caron) Cc: nuwg@comet.ucar.edu Subject: Re: Navigation Information Query Organization: UCAR Unidata Program Date: Thu, 27 Apr 1995 09:51:26 -0600 From: Russ Rew John, Before considering the rest of your proposal, I have to correct what I think may be a misunderstanding: You wrote: > ... Looking at Stackpole's summary of GRIB grids, I can > see how tempting it is to use their system. After all, there it all is, > nicely packaged and described. All the work is apparently done for > you. Its glaring weakness is that it describes very specific grids, rather > than a family of grids based on user-settable parameters. Such a family > would be exactly what I mean by a projection function. Note that the > projection function maps any x,y in the projection plane to lat,lon, so it > can handle any grid topology. The number of projections in common use is > reasonably small, but the number of different possible grids is infinite. and in a second posting: > If I read Stackpole's GRIB document right, the grids described are fixed and > not parameterized. (Does anyone understand differently?). If so, it cant be > used for general georefrenced data. The FGDC clearly parameterizes their > projections, and also seperates the grid description from the projection. I > would prefer that the grid be described by coordinate and coordinate > reference variables, and the projection only be described by the nav > variable. However, if in order to support FGDC we had to put also the grid > description into FGDC form, I think I could live with it. Since I'm not > sure of the details of implementing the FGDC, its possible that the > coordinate and coordinate reference variables might even satisfy what the > FGDC means by "content standards for metadata". Stackpole's summary of GRIB grids copies the WMO standard specification of a GRIB Grid Description Section (GDS) and adds some particular frequently-used NMC-specific grids that NMC identifies with a small integer grid ID. The GDS provides a way to describe infinite families of grids based on user-settable parameters. It's the parameterizations in the GDS for GRIB that I intended to use for grid descriptions. That's one of the reasons I stated at our last meeting that I wanted to change our primary reference to the WMO GRIB document rather than Stackpole's version of it with all the NMC-specific additions. The netCDF files we will produce from GRIB products have a grid parameterization based on the WMO GRIB GDS, even when an NMC grid ID is also available. There are some "International Exchange Grids" for which grid IDs have been assigned that are independent of the originating center, but my GRIB decoder always manufactures a GDS parameterization for these even when it's not sent with the GRIB product. Also for any of the infinite number of non-cataloged grids, a grid ID of 255 is used which specifically means the grid is specified in the GDS. As far as I can see, the GRIB GDS is very similar to the FGDC Content Standards for Geospatial Data parameterizations for grids. Each parameterization includes some families of grids not included by the other, but they both include all common projections. The GRIB1 GDS standard includes some unique parameterizations, such as the "quasi-regular" grids that are arguably not even grids, but they are already so commonly used and that we have to deal with them. If you want to see an on-line version of the WMO GRIB1 GDS specification look at: ftp://nic.fb4.noaa.gov/pub/nws/nmc/docs/gribguide/guide.txt which is a version of Stackpole's document without most of the NMC-specific additions. I don't know of any on-line copy of the original WMO standard. This is all independent of your suggestion to separate the description of the grid from the description of the projection. I'm still looking at that ... --Russ Organization: NCAR ACD From: caron@dilbert.acd.ucar.edu (John Caron) Subject: Re: Navigation Information Query To: russ@unidata.ucar.edu (Russ Rew) Date: Thu, 27 Apr 1995 10:31:20 -0600 (MDT) Cc: nuwg@comet.ucar.edu Thanks for clarifying that, that makes GRIB conventions infinitely more useable. Do you have an answer to this: > A related GRIB question: knowing the GRIB nav structure, how do you find out what the > world coords are for each grid cell? Is there a standard library, a lookup table or what? > Because whatever technique you use, there is probably an implicit ordering of the > coordinates that you just have to know. So lets make it explicit (by convention), so that