ECMWF Cycle 48r1 went live when these issues began to occur. Relevant links
to the compression methods are below.
https://www.ecmwf.int/en/elibrary/81320-impact-grib-compression-weather-forecast-data-and-data-handling-applications
https://confluence.ecmwf.int/display/FCST/Implementation+of+IFS+Cycle+48r1
https://public.ccsds.org/Pubs/121x0b3.pdf
On Thu, Jun 29, 2023 at 11:43 AM Shay Carter <srcarter@xxxxxxxx> wrote:
> Hi Pete,
>
> Thanks for the heads up!
>
> One of my coworkers filled me in with this pertinent information:
>
> A couple of days ago the ECMWF Open Data changed (grib2 compression type)
>> and most software is having problems reading it now. Wgrib2 can convert it
>> to a different compression which other software can then read, so there is
>> a work-around.
>
>
> So, in order to get ECMWF data back into your EDEX, I think you just need
> to change the compression which can be done with wgrib2. The same coworker
> linked
> this post <http://gradsusr.org/pipermail/gradsusr/2023-June/043431.html> about
> the issue, with the relevant code being something along the lines of this:
>
> wgrib2 IN.grb -set_grib_type c3 -grib_out OUT.grb*
>
> You'll need to change compression before inserting it into the EDEX queue .
> Let me know if this makes sense, if you're able to test it out, and how it
> goes?
>
> Thanks!
>
> --Shay
>
> On Thu, Jun 29, 2023 at 10:29 AM Manousos, Peter C via awips2-users <
> awips2-users@xxxxxxxxxxxxxxxx> wrote:
>
>> (apologies if you are both on the GEMBUD and AWIPS email lists as this
>> will be a duplicate…)
>>
>>
>>
>>
>>
>> Greetings. As of 06 28 2023 12z it seems the ECMWF’s freely available
>> WMO Essential grib2 data has changed (here
>> <https://data.ecmwf.int/forecasts/20230628/12z/0p4-beta/oper/>). It is
>> no longer fully processable by dcgrib2 or AWIPS EDEX
>>
>>
>>
>> We did not see any change notice on ECMWF web site or Google and were
>> wondering if anyone else is experiencing the following (attached is a
>> one-field <500 mb HGT> grib2 file), and if you did would you happen to know
>> the cause and any workarounds?
>>
>>
>>
>> wgrib2 -v -V test.grb2 shows what looks like a viable grib2 file
>>
>>
>>
>> 1:0:vt=2023062803:500 mb:3 hour fcst:HGT Geopotential Height [gpm]:
>>
>> ndata=405900:undef=0:mean=5562.91:min=4689.51:max=5955.01
>>
>> grid_template=0:winds(N/S):
>>
>> lat-lon grid:(900 x 451) units 1e-06 input WE:NS output WE:SN res 48
>>
>> lat 90.000000 to -90.000000 by 0.400000
>>
>> lon 180.000000 to 179.600000 by 0.400000 #points=405900
>>
>> basic_ang=0 sub_angle=1000000 units=0.000001
>>
>> unscaled lat=90000000 to -90000000 lon=180000000 to 179600000
>>
>>
>>
>> As additional info a wgrib2 -v –V on a grib2 file previous to June 28
>> shows the same thing as above.
>>
>>
>>
>> Submitting to EDEX shows the following error (at the end in bold with
>> preceding traceback messages)
>>
>>
>>
>> ERROR 2023-06-28 19:02:42,848 4308 [Ingest.GribDecode-6] grib: EDEX -
>> Error processing file
>> [/awips2/data_store/manual/grib/20230628/19/test.grb2]:
>>
>> Traceback (most recent call last):
>>
>> File
>> "/awips2/edex/lib/plugins/com.raytheon.edex.plugin.grib.jar/GribDecoder.py",
>> line 184, in decode
>>
>> gribDictList = grib2.decode(gribFile, self.startPosition,
>> self.messageLength)
>>
>> *error: Failed to get grib field.*
>>
>>
>>
>>
>>
>> If you have NAWIPS…
>>
>>
>>
>> NAWIPS 7.14.0.1 dcgrib2 -d temp.log -v 100 test.grd < test.grb2 gives
>> the following error (in bold)
>>
>>
>>
>> [27248] 230628/1520[DCINIT 3] Version 7.14.0.1
>>
>> [27248] 230628/1520[DC 2] read 102399/102399 bytes strt 0 newstrt 102399
>>
>> [27248] 230628/1520[DC 2] read 1/1 bytes strt 102399 newstrt 102400
>>
>> [27248] 230628/1520[DC 2] read 83204/102397 bytes strt 0 newstrt 83204
>>
>> *[27248] 230628/1520[CED_GNUM -1] No GID KXKY [900 451] LL [90.000000
>> 180.000000] UR [-90.000000 179.600006]*
>>
>> *[27248] 230628/1520[CED_GNUM -1] No GID KXKY [900 451] LL [90.000000
>> 180.000000] UR [-90.000000 179.600006]*
>>
>> [27248] 230628/1520[DC 2] read 0/19196 bytes strt 83204 newstrt 83204
>>
>> [27248] 230628/1520[DCGRIB -9] End of Data
>>
>> [27248] 230628/1520[DC 5] Normal termination.
>>
>> [27248] 230628/1520[DC 2] Number of bulletins read and processed: 1
>>
>> [27248] 230628/1520[DC 6] Shutting down.
>>
>>
>>
>> And nagrib2 also gives an error (in bold)
>>
>>
>>
>> getdrstemplate: DRS Template 5.42 not defined.
>>
>> *[NAGRIB2 -25] Error unpacking GRIB2 metadata*
>>
>> 0 GRIB messages were read or scanned from the GRIB file:
>>
>>
>> test.grb2
>>
>> 0 grids were written to the GEMPAK file:
>>
>>
>> test.grd
>>
>> Parameters requested: GBFILE,GDOUTF,PROJ,GRDAREA,KXKY,MAXGRD,CPYFIL,GAREA,
>>
>> OUTPUT,G2TBLS,G2DIAG,OVERWR,PDSEXT.
>>
>>
>>
>>
>>
>>
>>
>> ------------------------------
>>
>> * The information contained in this message is intended only for the
>> personal and confidential use of the recipient(s) named above. If the
>> reader of this message is not the intended recipient or an agent
>> responsible for delivering it to the intended recipient, you are hereby
>> notified that you have received this document in error and that any review,
>> dissemination, distribution, or copying of this message is strictly
>> prohibited. If you have received this communication in error, please notify
>> us immediately, and delete the original message. *
>> _______________________________________________
>> NOTE: All exchanges posted to Unidata maintained email lists are
>> recorded in the Unidata inquiry tracking system and made publicly
>> available through the web. Users who post to any of the lists we
>> maintain are reminded to remove any personal information that they
>> do not want to be made public.
>>
>>
>> awips2-users mailing list
>> awips2-users@xxxxxxxxxxxxxxxx
>> For list information, to unsubscribe, or change your membership options,
>> visit: https://www.unidata.ucar.edu/mailing_lists/
>>
>
>
> --
> *(she/her/hers)*
> Software Engineer III
> *UCAR - Unidata*
> *True Colors Chair*
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> awips2-users mailing list
> awips2-users@xxxxxxxxxxxxxxxx
> For list information, to unsubscribe, or change your membership options,
> visit: https://www.unidata.ucar.edu/mailing_lists/
>
