Unidata - To provide the data services, tools, and cyberinfrastructure leadership that advance Earth system science, enhance educational opportunities, and broaden participation. Unidata
         
  advanced  
 

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [netcdf-java] Grib File Dump not correct?/Grib Code to read Grib Parameter Names & Values



Chad,

I've been answering many Grib questions lately, it seems that most programmers can use the CDM layer above the grib layer and remove themselves from the details of the Grib file, ie such as offsets, etc.
The CDM layer should have all the information that you need.


On Tue, 6 Oct 2009, Chad Saxon wrote:

Robb,
i have been looking at the doc and been trying out some code. By the way thanks
for the updates and quick replies. 

The crux of what i want to do is really read in and store:

- the unit parameter of the data (i.e. temperature)
- the data coupled with the lat/lon in each record
- the offset of lat/lon(resolution) so i can loop through the data and store
appropriate parameter values.

Basically you would open the file as GridDataset, this will perform all the low level Grib actions for you.


Here's a tutorial to help get you get started: http://www.unidata.ucar.edu/software/netcdf-java/tutorial/

 I would look at GridDatatype, NetcdfFile, and NetcdfDatasets

There are also javaDocs at: http://www.unidata.ucar.edu/software/netcdf-java/v2.2.20/javadoc/index.html

Sample code: http://www.unidata.ucar.edu/mailing_lists/archives/netcdf-java/2009/msg00233.html
http://www.unidata.ucar.edu/mailing_lists/archives/netcdf-java/2009/msg00235.html



The above messages have urls to javaDocs about GridDatasets, if you do some searching I believe you will be able to get all the information that you need. On the netcdf-java home page:
http://www.unidata.ucar.edu/software/netcdf-java


There is a reference to a program ToolsUI GUI that will permit you to look at the files. I would launch it by webstart so you can get the feel for it. There isn't any documentation but it intutitive. Hope this gets you out the low levels of Grib.

Robb...


right now i have this:

*testfile = whateverfile.grb
    List<GridRecord> gribRecords;

ucar.unidata.io.RandomAccessFile gribfile = new
ucar.unidata.io.RandomAccessFile(testfile, "rw");
   index = new GribReadIndex();
   gridindex = new GridIndex();
   gridindex = index.open(testfile);

   gribRecords = gridindex.getGridRecords();

   i wanted to declare gribRecords to the class GribGridRecord since it seems i
have access to the offset parameters i would need to do my seek that you 
suggested
a while back for looking at all the data but the getGridRecords only returns of
type GridRecord which, according to the doc, doesnt have any functionality to 
look
at the offsets, etc.

   Once i get the list of records i just want to be able to do a seek on each 
one,
loop through its data and start on the next record. The outdated Index class
seemed to have that capability before. Maybe i am just overlooking something.
thanks for your help...

Chad




On Sep 14, 2009, at 10:07 AM, Robb Kambic wrote:

      Chad,

      Thanks for pointing out that the Java Docs need to be updated on the
      Decoders page, I'll do it sometime this afternoon.

      Robb...



      On Mon, 14 Sep 2009, Chad Saxon wrote:

            Robb,

            thanks for the input and feedback. i went ahead and got
            the grib-8.0 library

            and have to work through some of the depreciated stuff in
            my code now but i

            was wondering the the javadoc reflects the grib 8.0
            changes? Some of the

            functionality like, for
            example, Grib1GridDefinitionSection.getGdsVars();

            doesnt appear in the API javadoc that i am looking at.

            thanks...

            Chad

            On Sep 8, 2009, at 2:03 PM, Robb Kambic wrote:


     On Mon, 7 Sep 2009, Chad Saxon wrote:


           I am writing some low level code(because i need to),

                       to read grib files and all the values that i
            print out

                       on the console do not match the values when i
            dump

                       them out to a file. The grib file that i am
            testing

                       against is including in this e-mail for those
            that

                       want to take a look.


           Also i seem to be having problems getting actual

                       String literals for certain codes.. like the
            parameter

                       name for code 11 is Temperature and the Unit
            type is K

                       in this case but it seems that the way to get
            these

                       are not very intuitive to get using Index and
            my

                       GribRecords variable in the code below. Maybe
            someone

                       could shed some light on this??


           In advance, i want to thank all those that can take

                       the time to look through this.


           --Chad


     Chad,


     The Grib library usually reads the records of the grib file in

                 sequence of the sections, there are 5 sections to
            Grib1. See:

                 http://www.nco.ncep.noaa.gov/pmb/docs/on388/


     To read a particular section, the filePointer must be moved to the

                 start of the section to have a proper read. There is
            an index file

                 that has some of the offsets to the sections, such as
            the GDS

                 section. Here's the index of your sample file:


     ------------------------------------------------------------------

                 10 0 -1 11 96 1 0.0 255 0.0 2009-09-07T06:00:00Z 24
            284494432 36

                 68 1 false 7 0 2

                 10 0 -1 11 96 1 0.0 255 0.0 2009-09-07T06:00:00Z 48
            284494432 390

                 422 1 false 7 0 2

                 10 0 -1 11 96 1 0.0 255 0.0 2009-09-07T06:00:00Z 72
            284494432 744

                 776 1 false 7 0 2

            
     ------------------------------------------------------------------


     The offsets 36, 390, and 744 are the GDS offsets for the 3 records

                 in the file. If you would modify you code to do the
            seeks and then

                 read the GDS, then the values will be correct.


     gribfile.seek( 36 );

                 Grib1GridDefinitionSection gds =

                 Grib1GridDefinitionSection(gribfile);


     Here are some files that will help you understand the how the

                 index works.

                 IndexFormat.txt  locate in the root directory of the
            package.

                 ucar/grib/grib1 directory:

                 Grib1WriteIndex.java

                 Grib1Dump.java

                 Grib1GetData.java

                 Grib1Data.java

                 Grib1Input.java


     also look at the following files on how to extract particular PDS

                 and GDS values;


     GribReadIndex.java

                 ShowGribIndex.java


     I must warn you that the Grib library has been refactored, so the

                 Index api will be slightly different then the 6.0
            release. I would

                 suggest that you get the latest 8.0 release on the
            decoders page

                 located at:


     http://www.unidata.ucar.edu/software/decoders/


     Robb...


           here is my code below with output:


           public class GribPlugin {


           java.io.PrintStream ps;


           String[] fileDump;


           Grib1GridDefinitionSection gridSection;


           String testfile = "gfs20090907162501123.grb";


           public GribPlugin() throws IOException,

                       NoValidGribException, NotSupportedException{


           ucar.unidata.io.RandomAccessFile gribfile = new

                       ucar.unidata.io.RandomAccessFile(testfile,
            "rw");


           type = file.getEdition(gribfile);


           System.out.println("File read in: " + testfile + ",

                       Type of Grib File is " + type);


           fileDump = new String[3];


           fileDump[0] = "gfs20090907162501123.grb";


           fileDump[1] = "gribdump.out";


           fileDump[2] = "true";


           try{


           gridSection = new

                       Grib1GridDefinitionSection(gribfile);


           System.out.println("defining a new gridsection");


           }


           catch(IOException e){


           throw new IOException("Grid Section " + e);


           }


           System.out.println("grid definition section");


           System.out.println("dx: " + gridSection.getDx() + ",

                       dy: " + gridSection.getDy() + ", La1: " +

                       gridSection.getLa1() + ", La2: " +

                       gridSection.getLa2() + ", Lo1: " +


           gridSection.getLo1() + ", Lo2: " +

                       gridSection.getLo2() + ", Nx: " +
            gridSection.getNx()

                       + ", Ny: " + gridSection.getNy() + ", Grid
            Section

                       Name: " + gridSection.getName());


           System.out.println("File Dump Information");


           gribDump.main(fileDump);


           System.out.println("Dump GridFile Info Out To File");


           ps = new java.io.PrintStream("grib.out");


           index = gribIndex.writeFileIndex(gribfile, ps, true);


           //this is printing out the grib records


           System.out.println("Grib Records");


           gribRecords = index.getGribRecords();


           System.out.println("Number of Grib Records: " +

                       gribRecords.size());


           for(int i = 0; i < gribRecords.size(); i++){


           System.out.println(gribRecords.get(i).category + " " +


           gribRecords.get(i).center + " " +


           gribRecords.get(i).discipline + " " +


                  gribRecords.get(i).paramNumber + " " + <--how

                       do you convert this to the equivalent param
            name??


           gribRecords.get(i).productType + " " +


           gribRecords.get(i).decimalScale + " " +


                  gribRecords.get(i).gdsKey + " " +


           gribRecords.get(i).typeGenProcess + " " +


           gribRecords.get(i).forecastTime + " " +


                  gribRecords.get(i).refTime + " " +


           gribRecords.get(i).subCenter + " " +

                       gribRecords.get(i).bmsExists);


           } //end for


           } //end public class constructor


           } // end class GribPlugin


           Console Output:


           File read in: gfs20090907162501123.grb, Type of Grib

                       File is 1


           GribPDSLevel: Table 3 level 96 is not implemented yet


           PDS: Time Unit 9 is not yet supported


           Using default table:resources/grib/tables/wmo_2_v1.tab

                       (0:0:-1)


           defining a new gridsection


           Grid 28 not configured yet


           grid definition section


           dx: 2.632, dy: 0.0, La1: -8355.851, La2: 591.622, Lo1:

                       65.536, Lo2: 0.256, Nx: 540, Ny: 24583, Grid
            Section

                       Name: Latitude/Longitude Grid


           File Dump Information


           gds exception was caught


           Dump GridFile Info Out To File


           Grib Records


           Number of Grib Records: 3


           -1 7 0 11 10 1 2350466037 96 24 2009-09-07T06:00:00Z 0

                       false


           -1 7 0 11 10 1 2350466037 96 48 2009-09-07T06:00:00Z 0

                       false


           -1 7 0 11 10 1 2350466037 96 72 2009-09-07T06:00:00Z 0

                       false


           *****


           Now you will notice that my grid definition section

                       does not match any of the parameters from the
            grid

                       dump below(except for the grid section name).
            My

                       question is, is why is this and/or what am i
            doing

                       wrong here??? Those numbers that i get from the

                       Console output just do not make any sense to me
            at

                       all.


           *****


           GridDump Output:


           Header : GRIB1


                            Discipline : 0 Meteorological

                       Products


                          GRIB Edition : 1


                           GRIB length : 354


                    Originating Center : 7 US National Weather

                       Service (NCEP)


                Originating Sub-Center : 0 WMO Secretariat


                    Product Definition : 10 product valid at

                       reference time P1


                    Parameter Category : -1 Meteorological

                       Parameters


                        Parameter Name : 11 TMP Temperature


                       Parameter Units : K


                        Reference Time : 2009-09-07T06:00:00Z


                            Time Units : hour


                  Time Range Indicator : product valid at RT + P1


                           Time 1 (P1) : 24


                           Time 2 (P2) : 0


               Generating Process Type : 96 Global Forecast

                       System Model (formerly known as the Aviation)


                            Level Type : 1 surface


                         Level Value 1 : 0.0


                         Level Value 2 : 0.0


                            GDS Exists : true


                            BMS Exists : false


                 Number of data points : 3447


                             Grid Name : Latitude/Longitude Grid


                             Grid Shape: 0 spherical


                 Spherical earth radius: 6367.47


                                    Nx : 16


                                    Ny : 15


                                   La1 : 0.0


                                   Lo1 : -100.0


           Resolution & Component flags : 128


                                 Winds : True


                                   La2 : 29.0


                                   Lo2 : -70.0

            
=============================================================================

                 ==

                 Robb Kambic           Unidata Program Center

                 Software Engineer III   Univ. Corp for Atmospheric
            Research

                 rkambic@xxxxxxxxxxxxxxxx   WWW:
            http://www.unidata.ucar.edu/

            
=============================================================================

                 ==



      
===============================================================================
      Robb Kambic           Unidata Program Center
      Software Engineer III   Univ. Corp for Atmospheric Research
      rkambic@xxxxxxxxxxxxxxxx   WWW: http://www.unidata.ucar.edu/
      
===============================================================================





=============================================================================== Robb Kambic Unidata Program Center Software Engineer III Univ. Corp for Atmospheric Research rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/ ===============================================================================

 
 
  Contact Us     Site Map     Search     Terms and Conditions     Privacy Policy     Participation Policy
 
National Science Foundation (NSF) UCAR Community Programs   Unidata is a member of the UCAR Community Programs, is managed by the University Corporation for Atmospheric Research, and is sponsored by the National Science Foundation.
P.O. Box 3000     Boulder, CO 80307-3000 USA     Tel: 303-497-8643     Fax: 303-497-8690