[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: GRIB1 and BUFR in TDS

Hi,

There was some minor problems in decoding the files but they are now fixed
and ci.  the grib file had a strange char in the header and the bufr
decoder was trying to decode the qc section.  the test files are a bit strange,
each parameter was broken into 2 or more files, PRO, 000001, 000002, 000003...
each file has a PRO as the header and data starts in the 000001.  this is not
supported by either of Unidata's  grib or bufr libraries because it's not in
either formal GRIB or BUFR specification. i only tested the 000001 files
because the others would of required code changed to test.

so if the users want to use our libraries without modification then the files
need to be combined into 1 file, that can contain a header and data
without any auxiliary headers in the file. i assume that the files were
broken into parts to make for easier data transfers. Also, i would suspect
that there is a program to reassemble the files without the additional
headers and no additional characters being inserted into the files.
i would like to test some of the combined files if they are available.


robb...



On Fri, 8 Dec 2006, Ethan Davis wrote:

> Hi Robb,
>
> I got this email last week but guess I never forwarded it on to you. Can
> you take a look at the data files and see what it would take to read
> them? I've saved them in /upc/share/testdata/valentijnGRIB_BUFR. Sorry,
> I'm not sure which ones are GRIB and which are BUFR.
>
> Thanks,
>
> Ethan
>
> Valentijn Venus wrote:
> > Hi Ethan,
> >
> > I hope you are fine/
> >
> > I just read the support message "TDS and BUFR" below (You, John, and
> > Robb). Can you look into the attached example files as we receive them
> > currently through EUmetcast from Eumetsat
> > (http://www.eumetsat.int/Home/Main/Access_to_Data/Meteosat_Meteorologica
> > l_Products/BUFR___GRIB2/DF_ACC2DATA_METP_BUFR?l=en). Can you see what is
> > needed to support them within the TDS?
> >
> > Cheers, Tyn
> >
> > You may be helped with the following details:
> >
> > I have attached one GRIB1 dataset of Total Ozone from Meteosat-8/SEVIRI:
> > <<L-000-MSG1__-MPEF________-TOZ______-PRO______-200511010845-__>>
> > <<L-000-MSG1__-MPEF________-TOZ______-000001___-200511010845-__>>
> >
> > Also find attached two BUFR datasets (Cloud Layer Analysis):
> >
> > *CLA* for which the corresponding BUFR tables can be found in here:
> > http://www.eumetsat.int/groups/ops/documents/document/pdf_mpef_local_buf
> > r_descript.pdf
> >
> > The navigation is of the GRIB1  data is
> >
> >> still a bit foreign. Its the projection used by Eumetsat
> >> (www.eumetsat.int) for its Meteosat-8 imagery. If it is the same as is
> >> defined in the current HRIT MSG server code (msgtaget2.f) than it
> >> wouldn't be to complicated. The 90 Space View Perspective or
> >> Orthographic projection is defined in documents EUM/MSG/ICD/105 and
> >> EUM/MSG/SPE/022 (available at their website under
> >> Publications/Technical Documentation). Here is a section from it:
> >>
> >> --------------------------------------------------------------------
> >>                        Header : GRIB
> >>                    Discipline : 3 Space products
> >>                  GRIB Edition : 2
> >>                   GRIB length : 3444913
> >>            Originating Center : 254 EUMETSAT Operation Centre
> >>        Originating Sub-Center : 0
> >> Significance of Reference Time : 3 Observation time
> >>                Reference Time : 2005:4:18T12:0:0
> >>                Product Status : 1 Operational test products
> >>                  Product Type : 6 Processed satellite observations
> >>         Number of data points : 13778944
> >>                     Grid Name : 90 Space View Perspective or
> >>
> > Orthographic
> >
> >>                     Grid Shape: 3 Earth oblate spheroid with axes
> >> specified by produ cer
> >>        Oblate earth major axis: 6378.14
> >>        Oblate earth minor axis: 6356.7554 Number of points along
> >> parallel: 3712 Number of points along meridian: 3712 Latitude of
> >> sub-satellite point: 0.0
> >>  Longitude of sub-satellite pt: 0.0
> >>  Resolution & Component flags : 0
> >>         i direction increment : 3566.0
> >>         j direction increment : 3561.0  X-coordinate of sub-satellite
> >> : 1856.0001  Y-coordinate of sub-satellite : 1856.0001
> >>                 Scanning mode : 192
> >>                   Basic angle : 0
> >>        Altitude of the camera : 7.4533146E8
> >>        X-coordinate of origin : 0.0
> >>        Y-coordinate of origin : 0.0
> >>            Product Definition : 30 Satellite product
> >>            Parameter Category : 0 Image_format_products
> >>                Parameter Name : 7 Cloud_mask
> >>               Parameter Units :
> >>       Generating Process Type : 8
> >>                  ForecastTime : 0
> >>            First Surface Type : 0 Reserved
> >>
> >
> >
> >
> >
> > Re: TDS and BUFR
> >
> >     * To: Ethan Davis <edavis@xxxxxxxxxxxxxxxx>
> >     * Subject: Re: TDS and BUFR
> >     * From: John Caron <caron@xxxxxxxxxxxxxxxx>
> >     * Date: Sat, 19 Aug 2006 12:47:11 -0600
> >     * Cc: James Gallagher <jhrg@xxxxxxx>, "'support-thredds'"
> > <support-thredds@xxxxxxxxxxxxxxxx>
> >     * In-reply-to: <address@hidden>
> >     * Organization: UCAR/Unidata
> >     * References: <address@hidden>
> > <address@hidden>
> >     * User-agent: Mozilla Thunderbird 1.0.6 (Windows/20050716)
> >
> > All correct.
> >
> >
> > We are just getting around to testing BUFR, I would consider it alpha
> > software for now. The CDM data objects will likely evolve some more.
> >
> > Ethan Davis wrote:
> >
> >
> >     Hi James,
> >
> >
> >     The short answer is yes but not all types of BUFR files are
> > currently supported.
> >
> >     Now for a slightly longer answer that John may want to correct as
> > I'm not as up on our BUFR work as he is ...
> >
> >     John and Robb (also here at Unidata) have developed a ucar.bufr
> > library. The netCDF-Java library, which TDS uses for reading data files,
> > uses ucar.bufr to read BUFR files. I think there is a lot of variation
> > in the data structure represented in BUFR files. Each type of BUFR files
> > require some mapping into the netCDF/CDM data model. This is the hard
> > part and will require on going work as we find new types of BUFR files.
> > However, I believe we already have a number of "standard" BUFR file
> > types being read by netCDF-Java. Another complication is that, like
> > GRIB, BUFR also external tables (for parameter info and such) like in
> > GRIB. And, as with GRIB, the different "centers" can extend the
> > "standard" tables. So that adds to the work.
> >
> >     Ethan
> >
> >
> >     James Gallagher wrote:
> >
> >
> >         Ethan, John,
> >
> >
> >         Does the TDS support BUFR?
> >
> >
> >         James
> >         --
> >         James Gallagher                jgallagher at opendap.org
> >         OPeNDAP, Inc                   406.723.8663
> >
> >
> >
>
> --
> Ethan R. Davis                                Telephone: (303) 497-8155
> Software Engineer                             Fax:       (303) 497-8690
> UCAR Unidata Program Center                   E-mail:    address@hidden
> P.O. Box 3000
> Boulder, CO  80307-3000                       http://www.unidata.ucar.edu/
> ---------------------------------------------------------------------------
>
>

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================