Re: [thredds] Help with GRIB encoding - what is "reference time of data" ??

  • To: "Benno Blumenthal" <benno@xxxxxxxxxxxxxxxx>
  • Subject: Re: [thredds] Help with GRIB encoding - what is "reference time of data" ??
  • From: "TOYODA Eizi" <toyoda.eizi@xxxxxxxxx>
  • Date: Fri, 29 Nov 2013 19:38:44 +0900
Hi Benno,

1) I had similar question before.  I'm sorry I can't remember the conclusion 
for GRIB, but for BUFR, there are several element descriptors whose 
definition changed over time.  WMO is trying to maintain upper 
compatibility, but sometimes something happened unfortunately.  I guess it 
is safer to assume GRIB is in similar status.

2)  Again I'm sorry the table posted at following site is the only one 
maintained.
http://www.wmo.int/pages/prog/www/WMOCodes/WMO306_vI2/LatestVERSION/LatestVERSION.html
Collecting local tables is I think the next step.

3) You'll need the code of the originating centre (and probably sub-centre) 
to identify the local table.  By the way I think NCEP and ECMWF are using 
different short names for GRIB parameters.  So I don't think it is easy for 
WMO to establish its standard name.

Sorry about unconfortable answers....
Eizi
  ----- Original Message ----- 
  From: Benno Blumenthal
  To: TOYODA Eizi
  Cc: John Caron ; THREDDS community ; Java NetCDF
  Sent: Thursday, November 28, 2013 2:33 AM
  Subject: Re: [thredds] Help with GRIB encoding - what is "reference time 
of data" ??


  I have been looking at grib2, and was hoping I could get some help on some 
lower-level metadata questions.


  1) wmo code tables have version numbers which are also in the data 
records -- is it sufficient to use a code table that has version number 
greater than or equal to the number in the record, or do we have to match 
the version number in the record?  WMO is on version 12, my test data of 
interest is version 1, I would much rather have a single latest-and-greatest 
that keep all twelve-and-counting versions.


  2) there are xml versions of the wmo code tables, which can be converted 
into something more programmatically useful.  Is there a standard set of 
converted tables, i.e. XML with clean discipline, category, version coding? 
Can the same be said of local tables, e.g. NCEP for starters?


  3) my understanding so far is that to identify a scientific parameter 
(a.k.a standard_name) in grib we need wmo table version or local table 
version, data source (if local table), discipline, category, parameter 
number, and where (and if) we find this info depends on the PDS template 
number.
  NCEP maps this to short names, though WMO does not. And you are presumably 
mapping this to standard_names as well.   Are these mappings available, i.e. 
in XML format?



  On Wed, Nov 27, 2013 at 5:05 AM, TOYODA Eizi <toyoda@xxxxxxxxxxxxxx> 
wrote:

    Hi John,

    Yes JMA still issues GRIB edition 1 for:
    - bulletins sent to GTS since long time ago, and
    - the reanalysis project JRA-25 and JRA-55 (coming).
    In the reanalysis there are some products like monthly average, for 
which
    multiple model runs are used in a single message.  In that case my
    understanding is that the reference time is set to the beginning of the 
time
    range.

    Situations are similar in GRIB2.  I'm hoping the ECMWF reanalysis will 
start
    using GRIB2 sometime.  I'm afraid I don't know actual data using 
"Verifying
    time of forecast" (CT1.2:2).

    The reason why GRIB distinguishes "Analysis" time (CT1.2:0) and "Start 
of
    forecast" (CT1.2:1) might be rather historic nowadays.  It has a root in 
the
    difference between data, i.e. analysis and initialized analysis 
(sometimes
    called data for ft=0).

    Analysis is a best estimate of the state of the atmosphere, made from
    observation and past numerical forecast.  In the past NWP centers used
    methods called nudging or optimal interpolation (the latter is same as 
what
    geographers call kriging).  The bad thing is the result may contain
    inbalance between wind and pressure, which causes unnatural gravity wave 
in
    the forecast.  So people had to remove such inbalance after analysis. 
That
    is the "initialization" heavily mentioned in GRIB edition 1.

    Recent NWP systems use variational assimilation, which has less such
    problem.  So the analysis simply becomes the initial field without much
    changes.  I don't think somebody still issues "initialized analysis"
    separated from "(uninitialized) analysis".  In GRIB edition 2, the 
concept
    remains in a footnote in the Code table 1.4, but usually the analysis 
and
    forecast are put in a single GRIB message for Forecast products 
(CT1.4:1).
    In short, people no longer cares.

    Best,
    Eizi

    ----- Original Message ----- From: "John Caron" <caron@xxxxxxxxxxxxxxxx>
    To: "Eizi TOYODA" <toyoda@xxxxxxxxxxxxxx>
    Cc: "THREDDS community" <thredds@xxxxxxxxxxxxxxxx>; "Java NetCDF" 
<netcdf-java@xxxxxxxxxxxxxxxx>
    Sent: Wednesday, November 27, 2013 12:09 AM
    Subject: Re: [thredds] Help with GRIB encoding - what is "reference time 
of data" ??




      Hi Eizi:

      Thanks for clarifying that terminology, I see that my understanding 
has been a bit fuzzy. I think I do indeed mean "Start of forecast", not the 
actual time the model was run.

      Does your agency still use GRIB-1? If so, do you ever code anything 
other than the "Start of forecast" in the reference time ?

      If you use GRIB-2, do you set "Significance of reference time" equal 
to 1 = "Start of forecast" ?

      Finally, what in your opinion is the meaning of the other 
"Significance of reference time" codes, esp how does "Analysis" differ from 
"Start of forecast" ?

      thanks,
      John

      On 11/26/2013 2:23 AM, Eizi TOYODA wrote:

        Hi John,

        I'm not sure what do you mean by "run time".

        If you want the date/time at which the forecast model started, I 
don't
        think GRIB contains it.

        If you mean the initial time of forecast model, that is "Start of
        forecast" and it is really common to use this as reference time.

        Best,
        --
        Eizi TOYODA: Japan Meteorological Agency
        Associate member of WMO/CBS/OPAG-ISS/IPET-DRMM

        Best Regards,
        --
        Eiji (aka Eizi) TOYODA
        http://www.google.com/profiles/toyoda.eizi


        On Sat, Nov 23, 2013 at 3:03 AM, John Caron <caron@xxxxxxxxxxxxxxxx
        <mailto:caron@xxxxxxxxxxxxxxxx>> wrote:

            The question is whether the CDM can assume that GRIB "Reference 
time
            of data" is the "run time" of a forecast model.

            In GRIB-1 docs, in the PDS there is:

            "Reference time of data – date and time of start of averaging or
            accumulation period"



            In GRIB-2 in Identification section, there is:

            12 Significance of reference time (see Code table 1.2)

            Reference time of data:

            13–14 Year (4 digits)
            15 Month
            16 Day
            17 Hour
            18 Minute
            19 Second

            And Code table 1.2 has the following:

            Code Table Code table 1.2 - Significance of reference time (1.2)
                 0: Analysis
                 1: Start of forecast
                 2: Verifying time of forecast
                 3: Observation time
                -1: Reserved
                -1: Reserved for local use
               255: Missing


            None of this obviously refers to "run time", although I suspect
            that's how many centers use it. However, it appears that when 
you
            want to define a time interval, say "average of the temperature,
            starting 12 hours and ending 24 hours from reference, you may 
use
            the reference time to define the start of that interval. In 
which
            case, its not the runtime. Im hoping thats not the case, that
            reference time is the same as the run time for forecast models.

            So if you know how to interpret these for any or all datasets,
            please send me a note, or post to this group. Please pass this
            question on to anyone who might be willing to contribute.

            Thanks!

            John

            _______________________________________________
            thredds mailing list
            thredds@xxxxxxxxxxxxxxxx <mailto:thredds@xxxxxxxxxxxxxxxx>
            For list information or to unsubscribe,  visit:
            http://www.unidata.ucar.edu/mailing_lists/






    _______________________________________________
    thredds mailing list
    thredds@xxxxxxxxxxxxxxxx
    For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/





  -- 
  Dr. M. Benno Blumenthal          benno@xxxxxxxxxxxxxxxx
  International Research Institute for climate and society
  The Earth Institute at Columbia University
  Lamont Campus, Palisades NY 10964-8000   (845) 680-4450 
  • 2013 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the thredds archives: