Writing an IOSP: Example - Radial Data

NEXRAD Level III data are radar products generated from the Level II base data.  There are a total of 41 Level-III products routinely available from NCDC. The ucar.nc2.iosp.nids.Nidsiosp class provides access to all Nexrad Level-III datasets distributed through Unidata Internet Data Distribution system. The Level III data contains  radial, raster, and alphanumeric data types.    


Sample Level 3 Dataset

Look at structure of the example dataset in ToolsUI:

Look at the example dataset as a radial type and a raster type:

Data Structure

The Level III data is  separated into the following blocks:

  1.  Message Header
  2.  Production Description
  3. Production Symbology - symbology data is then broken up into layers, the Symbology Block has a header plus a number of data layers.  These layers can be graphical such as radial and rastor scan data or could be textual or symbolic.
    1. Symbology Layer - each layer is split up into packets
  4. Graphic Data
  5. Tabular Data

Data Type


Radial Data Type:  it contains values for each bin within a particular radial. Each radial is defined by a start and end angle and by distance. There are multiple radials that define a full sweep.

Raster Data Type: it is pixelized data defined by rows and columns. Its start coordinate on -2048 to 2047 coordinate system. It is used to provide image display,  contour, and shaded data plots.

Alphanumeric Data Type: it gives image/graphic locations of where to plot the lines, IDs, symbols etc. It includes the meso, tvs, hail, and sti products

IOSP Details

public boolean isValidFile(RandomAccessFile raf) throws IOException;

We identify that this is a Level III data file by reading some of the header record. We check the first 25 byte of the data file to search for the "data set ID" information which contains a string: "SDUS", that is a LDM identifier of NEXRAD Level III product.

public void open(RandomAccessFile raf, NetcdfFile ncfile, CancelTask cancelTask) throws IOException;

In this API we get all of the information: dimensions, attributes, and variables and use them to create NetcdfFile object. The information needed to construct these objects is contained throughout the whole dataset.

       
      for(int i=0; i<sinfo.nlayers ; i++ ){
          while (icount < Divlen_length ) {
              switch(pkcode)
              {
                 case 18:  //  DPA
                 case 17: //   (pkcode == 0x11)   Digital Precipitation Array
                      break;
                 case 10:    //     unlinked vector packet
                      break;
                 case 1:   // text and special symbol packet
                      break;
                 case 2:        // text and special symbol packets             
                      break;
                 case 8:       //text string 
                      break;
                 case 3:   // Special graphic symbol packet
                 case 11:   // Special graphic symbol packet

                 case 25:   // Circle packet
                      break;
                 case 12:     //TVS
                      break;
                 case 13:    // hail  positive
                      break;
                 case 14:     // hail probable
                      break;
                 case 4:    // wind barb
                      break;
                 case 5:  //   Vector Arrow Data
                   break;
                 case 23:   // past position data packet
                 case 24:  // forecast position data packet
                      while (poff < plen){
                          int pcode = bos.getShort()
                          switch (pcode)   {
                             case 2:  // text and special symbol packets           
                            break;
                             case 6:   // linked vector packet
                            break;
                             case 25:   // Circle packet
                                 break;
                      }
                      break;
                 default:
                      if ( pkcode == 0xAF1F ) {              /* radial image                  */
                          hedsiz += pcode_radial( bos, hoff, hedsiz, isZ, uncompdata, pinfo.threshold) ;
                          break;
                      }
                      else if (pkcode == 0xBA0F ||pkcode == 0xBA07 ) {      /* raster image                  */
                          hedsiz += pcode_raster( bos, (short)pkcode, hoff, hedsiz, isZ, uncompdata);
                          break;
                      }
                      else {
                          out.println( "error reading pkcode equals " + pkcode);
                          throw new IOException("error reading pkcode, unable to handle the product with code " + pkcode);
                      }                   
              } //end of switch
          }
          klayer = klayer + Divlen_length + 6;
     }


Here we are following the basic attribute conventions from the netCDF Users Guide with the "long_name", "units", "scale_offset", and "add_offset" attributes. We are also following the more recent Coordinate attribute convention which describes attributes for describing more general coordinate systems than supported by coordinate variables as defined in the netCDF Users Guide.

public Array readData( Variable v2, List section ) throws IOException, InvalidRangeException;

The readData() method is where we implement reading the data from disk.

  1. For all the data read from the dataset, we use offset information to calculate the location in the file for each variable.
  2. This API always read all data for each requested variable, subsetting (if requested) is done using ucar.ma2.Array subsetting.
  3. Coordinate variables of raster data type are cached.

How to Represent Level III Data in netCDF Data Model

Radial

The level III  data with radial data formats can be contained in two dimensional arrays:

 dimensions:
   azimuth = 367;
   gate = 230;
 variables:
   byte BaseReflectivity_RAW(azimuth=367, gate=230); 
     :long_name = "BREF: Base Reflectivity";
     :units = "dbZ";
     :_CoordinateAxes = "elevation azimuth gate rays_time latitude longitude altitude";
   float BaseReflectivity(azimuth=367, gate=230);
     :long_name = "BREF: Base Reflectivity";
     :units = "dbZ";
     :_CoordinateAxes = "elevation azimuth gate rays_time latitude longitude altitude";



Level III data with radial data format uses polar coordinates (elevation, azimuth, distance) to describe the location of its points in space, and it is  referred as Radial Coordinate System. A Radial Coordinate System has a Elevation, Azimuth, and Distance coordinate axis. It may also has a Time coordinate axis. Generally, in our level II and level III radar products, there is a time variable at radial (elevation, azimuth), or sweep (elevation) level, so it is considered as a variable, rather than coordinate axis.


Raster

The level III  data with raster data formats can be contained in two dimensional
arrays:




 dimensions:
   x = 116;
   y = 116;
 variables:
   byte EchoTop_RAW(y=116, x=116); 
     :long_name = "TOPS: Echo Tops";
     :units = "K FT";
     :_CoordinateAxes = "x y time latitude longitude altitude";
   float EchoTop(y=116, x=116); 
     :long_name = "TOPS: Echo Tops";
     :units = "K FT";
     :_CoordinateAxes = "x y time latitude longitude altitude";



Level III data with raster data format uses FlatEarth Projection. This
projection surface is tangent at some point (lat0, lon0) and has a y
axis rotated from true North by some angle. We call it "flat" because
it should only be used where the spherical geometry of the earth is not
significant.  




Alphanumeric



Some level III data with alphanumeric format is represented with
structures:




dimensions:
   textStringSize = 1;
   row = 13;
   box = 13;
 variables:
   byte PrecipArray_0(Row=131, Box=131);
     :long_name = "PRET: Hourly Digital Precipitation Array at Symbology Layer 0";
     :units = "KM";
   Structure {
     short x_start;
       :units = "KM";
     short y_start;
       :units = "KM";
     String textString;
       :units = "";
   } textStruct_code1(textStringSize);
       :long_name = "text and special symbol for code 1";