[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: 19990127: grib to netcdf



Missy,

Good news, I think Chiz and I found the ruc2 problem for parameter 185. 
The ruc2 grid files are larger than the older grid files and the
gribtonc code didn't provide the adequate space to properly read in the
raw data.  The fix is to increase the MAX_GRIB_SIZE to 100000.  I'll
attach the grib1.h to be inserted into the src dir that has the fix.  You
need to do:

% make clean
% make

So the grib1.h file is included properly.

Robb...



On Thu, 4 Mar 1999, Melissa Richey wrote:

> 
> Hi Robb-
> 
> I know it has been a while, but we are still interested in using this
> decoder to convert grib format to netcdf format. We are still having
> trouble obtaining the entire water vapor mixing ratio variable. I have
> tested it on some other ruc2 files, and continute to get the same
> problem. The decoder successfully degribs the last 11 levels of water
> vapor mixing ratio, but the first 29 levels give  'oversize GRIB
> product' . Right now, we are converting the grib format into a
> non-standard format used in only part of our division. This degribber
> is able to degrib water vapor mixing ratio, and when I compare the
> last 11 levels of wvmr to the ones in the netcdf file, the values are the
> same. Becaause our degribber successfully extracts wvmr, that would
> seem to imply that the raw grib file is not corrupted. 
> 
> Although we are already converting grib into a usable format, it is a
> format which is soon to become obsolete and no longer supported in our
> division. We are interested in using netcdf format because it is a
> standard format, widely supported, and is familiar to outside groups
> with whom we are working.
> 
> Were you able to obtain a ruc2 file or make any more progress in
> solving the mystery of water vapor mixing ratio?
> 
> Thanks,
> Missy 
> 
> 
> 
> + 
> + On Thu, 11 Feb 1999, Melissa Richey wrote:
> + 
> + > 
> + > Hi Robb-
> + > 
> + > I have been looking at gribtonc using the debugger, so I thought I
> + > would let you know what I have found. I can't figure out why it is
> + > doing what it is doing, but I thought it might mean something to
> + > you... 
> + > 
> + > The function 'get_prod' controls the reading of one grib product into
> + > a buffer. In reading a normal grib product, 'look_for_mark'
> + > successfully finds the start of the product on the first
> + > try. (FOUND_START)
> + > It then successfully fills the rest of the buffer and finds the end of
> + > the product. (FOUND_END)
> + > For the bad mixing ratio product, it first finds the end of the
> + > product, then finds the start, then fills the data buffer before ever
> + > finding the end again, (NOT_FOUND) resulting in "oversize grib
> + > product."
> + 
> + Missy,
> + 
> + My first impression is that the grid for the water vapor mixing ratio is
> + somehow corrupted or the the grid immediately before is corrupted. Does
> + this happen on all your raw files, I would test it on more than one. 
> + After looking at the code, I wonder if the FOUND header state machine code
> + is parsing correctly wvmixr grid. We don't have a raw ruc2 presently, I
> + should have one tomorrow.  Then I can take a first hand look.
> + 
> + Robb...
> + 
> + ps Have you tried od (octal dump) on the ruc2 raw file? It could be a
> + sanity check.
> + > 
> + > Just my two cents worth.
> + > 
> + > Missy
> + > 
> + > 
> + > 
> + > + Missy,
> + > + 
> + > + That's what I started to do before I found out about the conflict of 
> 185. 
> + > + The only suggestion I have is to use the brute force method, that is 
> get a
> + > + raw grib file including param 185 and use a debugger on gribtonc. My
> + > + feeling gribtonc is trying to process 185 according to the NWS manner
> + > + instead of the FSL manner.  I'm not the originator of gribtonc and I 
> have
> + > + not added/changed params before, so I would have to research this. I'll
> + > + look at the code today to see if there is something else. Also, as a 
> guide
> + > + line I would look at how the other hybrid ruc2 variables are processed.
> + > + Have you changed the cdl wvmixr variable to match your modified 
> gribtonc?
> + > + 
> + > + Just some thoughts,
> + > + Robb...
> + > 
> + 
> + 
> ===============================================================================
> + Robb Kambic                            Unidata Program Center
> + Software Engineer III                          Univ. Corp for Atmospheric 
> Research
> + address@hidden                 WWW: http://www.unidata.ucar.edu/
> + 
> ===============================================================================
> + 
> + 
> 

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================
/*
 *   Copyright 1995 University Corporation for Atmospheric Research
 *   See ../COPYRIGHT file for copying and redistribution conditions.
 */
/* $Id: grib1.h,v 1.4 1995/06/01 05:06:36 russ Exp $ */

/*
 * NMC "GRIB" Edition 1 Data structures, taken from WMO FM 92-X GRIB
 */

#ifndef _GRIB1_H
#define _GRIB1_H

#ifdef __OBJECTCENTER__
#define const                   /* workaround for bug with const parameters */
#endif

#include "gribtypes.h"          /* g1int, g2int, ..., and conversion functions 
*/
#include "gds.h"                /* Grid Description Section (GDS) raw data 
layout */
#include "get_prod.h"           /* for prod structure */

#define MAX_GRIB_SIZE 100000    /* The biggest we've seen on HRS is 6045 */
#define HAS_GDS         0x80
#define HAS_BMS         0x40    
#define NONCATALOGED_GRID 255   /* indicates grid is defined by GDS */

/*
 * Indicator Section
 */
typedef struct ids {
    char grib[4];               /* 'G', 'R', 'I', 'B' */
    g3int len;                  /* length of entire GRIB message */
    g1int edition;              /* GRIB edition number */
} ids;

/*
 * Product Definition Section
 */
typedef struct pds {
    g3int len;                  /* length of section in bytes */
    g1int table_version;        /* GRIB tables version number */
    g1int center;               /* id of center */
    g1int model;                /* model id, allocated by center */
    g1int grid;                 /* grid id, allocated by center */
    g1int db_flg;               /* whether GDS, BMS included */
    g1int param;                /* parameter id, from GRIB table 2 */
    g1int level_flg;            /* type of level, from GRIB table 3 */
    union {                     /* height, pressure, etc of levels (table 3) */
        g2int lev;              /* one level from 2 bytes */
        g1int levs[2];          /* two 1-byte levels */
    } level;
    g1int year;                 /* reference time of data */
    g1int month;
    g1int day;
    g1int hour;
    g1int minute;
    g1int tunit;                /* unit of time range, from GRIB table 4 */
    g1int tr[2];                /* periods of time or interval */
    g1int tr_flg;               /* time range indicator, from GRIB table 5 */
    g2int avg;                  /* number in average, if any */
    g1int missing;              /* number missing from averages or accums */
    g1int century;              /* 20 for years 1901-2000 */
    g1int subcenter;            /* reserved in GRIB1 standard */
    g2sint scale10;             /* (signed) units decimal scale factor */
    unsigned char reserved1[12];        /* reserved; need not be present */
    unsigned char reserved2[GRIB_ARB]; /* reserved for local center use;
                                   need not be present */
} pds;

/*
 * Bit Map Section
 */
typedef struct bms {
    g3int len;                  /* Length of section in bytes */
    g1int nbits;                /* Number of unused bits at end of section */
    g2int map_flg;              /* 0 if bit map follows, otherwise catalogued
                                   bit map from center */
    unsigned char bits[GRIB_ARB];       /* the bit map */
} bms;


#define BDS_KIND        0x80    /* From GRIB Table 11, kind of data bit.  0
                                   for grid-point data, 1 for spherical
                                   harmonic coefficients */
#define BDS_PACKING     0x40    /* From GRIB Table 11, kind of packing bit.
                                   0 for simple packing, 1 for complex
                                   (spherical harmonics) or second-order
                                   (grid) packing */
#define BDS_DATATYPE    0x20    /* From GRIB Table 11, data type bit.  0 if
                                   floating-point values, 1 if integer
                                   values */
#define BDS_MORE_FLAGS  0x10    /* From GRIB Table 11, additional flags bit.
                                   0 if no additional flags, 1 if more flag
                                   bits for grid-point data second-order
                                   packing */
#define BDS_MATRIX      0x40    /* From GRIB Table 11, matrix values.  0 for
                                   single datum at each grid point, 1 for
                                   matrix of values at each grid point */
#define BDS_SECONDARY   0x20    /* From GRIB Table 11, secondary bit maps.
                                   0 if no secondary bit maps, 1 if
                                   secondary bit maps present */
#define BDS_WIDTHS      0x10    /* From GRIB Table 11, widths flag.  0 if
                                   second-order values have constant width,
                                   1 second-order values have different
                                   widths */
/*
 * Binary Data Section
 */
typedef struct bds {
    g3int len;                  /* Length of section in bytes */
    g1int flg;                  /* High 4 bits are flag from GRIB table 11.
                                   Low 4 bits are no. of unused bits at end. */
    g2sint scale;               /* (signed) scale factor */
    g4flt ref;                  /* Reference value (min of packed values) */
    g1int bits;                 /* Number of bits containing each packed val */
    unsigned char data[GRIB_ARB]; /* The packed data */
} bds;

/*
 * The GRIB1 raw struture
 */
typedef struct grib1 {
    char *hdr;                  /* Header such as WMO Header, if any */
    ids *idsp;                  /* Indicator section */
    pds *pdsp;                  /* Product definition section */
    gds *gdsp;                  /* Grid description section (optional) */
    bms *bmsp;                  /* Bit map section (optional) */
    bds *bdsp;                  /* Binary data section */
} grib1;

#ifdef __cplusplus
/* Overlay a preallocated raw grib1 on a raw product */
extern "C" int make_grib1(prod*, grib1*);
/* allocate a new grib1 and overlay it on a raw product */
extern "C" grib1* new_grib1(prod*);
extern "C" void free_grib1(grib1*);     /* free a grib1 */
#elif defined(__STDC__)
/* Overlay a preallocated raw grib1 on a raw product */
extern int make_grib1(prod*, grib1*);
/* allocate a new grib1 and overlay it on a raw product */
extern grib1* new_grib1(prod*);
extern void free_grib1(grib1*); /* free a grib1 */
#else
/* Overlay a preallocated raw grib1 on a raw product */
extern int make_grib1( /* prod*, grib1* */);
/* allocate a new grib1 and overlay it on a raw product */
extern grib1* new_grib1( /* prod* */ );
extern void free_grib1( /* grib1* */ ); /* free a grib1 */
#endif

#endif /* _GRIB1_H */