Hi Ryan,
Glad you got this issue resolved. There are two ways to get/ingest the GOES
16/17 ABI data.
1. Use the NOAAPort feed which get stored as the .gini files (The top
option below)
2. Use the NIMAGE data Unidata is sending out which are the netcdf files
stitched together, with the WMO header stripped so the data can also be
read into other software (the bottom option below)
#
# GOES 16/17 ABI
# NOAAPort
NOTHER ^(TI....) (KNES) (......) (...)
FILE -close -edex
/awips2/data_store/GOES/GINI/\1_\2_\3_\4_(seq).gini
#
# GOES 16/17 ABI
# Unidata-stitched
NIMAGE
^/data/ldm/pub/native/satellite/GOES/([^/]*)/Products/CloudAndMoistureImagery/([^/]*)/([^/]*)/([0-9]{8})/([^/]*).nc
FILE -close -edex
/awips2/data_store/GOES/CMI/\5.nc4
In-house we are ingesting the NIMAGE netcdf files, but either way works.
Thanks,
~Tiffany
On Wed, Dec 30, 2020 at 11:49 AM Ryan Hickman <ryan@xxxxxxxxxxxxxxxx> wrote:
> I noticed my goof. You need to edit `distribution/goesr.xml` not
> satellite.xml.
>
> "<regex>^TI[SR]</regex>" becomes "<regex>^TI[SRU]</regex>" and all is well.
>
> INFO 2020-12-30 17:49:16,588 2820 [Ingest.GOESR-1] Ingest: EDEX: Ingest -
>> goes-r::
>> /awips2/data_store/goesr/noaaport/TIUR13_KNES_301748_PAA_921505.gini
>> processed in: 0.0600 (sec) Latency: 0.0830 (sec)
>>
>
> Consider this resolved.
>
> On Wed, Dec 30, 2020 at 10:17 AM Ryan Hickman <ryan@xxxxxxxxxxxxxxxx>
> wrote:
>
>> I am unable to get EDEX to ingest the GOES-17 Mesoscale sectors from
>> NOAAPORT. This started with a report of that data missing and I was able to
>> find entries in the "unrecognized files" log.
>>
>> WARN 2020-12-30 17:01:41,160 6124 [external.dropbox-2] RouteFailedLog:
>>> TRDA01 KWBC 301600
>>> WARN 2020-12-30 17:01:41,161 6125 [external.dropbox-5] RouteFailedLog:
>>> THBA01 KWBC 301500
>>> WARN 2020-12-30 17:01:41,162 6126 [external.dropbox-4] RouteFailedLog:
>>> TRDA01 KWBC 301700
>>> WARN 2020-12-30 17:01:41,166 6127 [external.dropbox-5] RouteFailedLog:
>>> TRAA01 KWBC 301600
>>> WARN 2020-12-30 17:01:41,167 6128 [external.dropbox-4] RouteFailedLog:
>>> TRBA01 KWBC 301500
>>> WARN 2020-12-30 17:01:41,171 6129 [external.dropbox-5] RouteFailedLog:
>>> TRDA01 KWBC 301600
>>> WARN 2020-12-30 17:01:41,171 6130 [external.dropbox-4] RouteFailedLog:
>>> THDA01 KWBC 301600
>>> WARN 2020-12-30 17:01:41,174 6131 [external.dropbox-2] RouteFailedLog:
>>> THDA01 KWBC 301600
>>> WARN 2020-12-30 17:01:42,308 6132 [external.dropbox-5] RouteFailedLog:
>>> TRBA01 KWBC 301500
>>> WARN 2020-12-30 17:01:42,314 6133 [external.dropbox-5] RouteFailedLog:
>>> THDA01 KWBC 301600
>>>
>>
>> The pqact.conf entry is the default one that was included with EDEX.
>>
>>> #
>>> # GOES 16/17 ABI
>>> #
>>> NOTHER ^(TI....) (KNES) (......) (...)
>>> FILE -close -edex
>>> /awips2/data_store/goesr/noaaport/\1_\2_\3_\4_(seq).gini
>>>
>>
>> Since the `distribution/satellite.gini.xml` includes
>> "<regex>.*.gini.*</regex>" you'd figure it would get ingested. Regardless,
>> I added "<regex>TIU... KNES</regex>" and did an `edex restart ingest` which
>> got the data flowing to the right ingester.
>>
>> INFO 2020-12-30 17:09:30,981 9392 [Ingest.Gini-1] Ingest: EDEX: Ingest -
>>> satellite::
>>> /awips2/data_store/goesr/noaaport/TIUR06_KNES_301708_PAA_917280.gini
>>> processed in: 0.0040 (sec) Latency: 0.0240 (sec)
>>> INFO 2020-12-30 17:09:33,202 9397 [Ingest.Gini-1] Ingest: EDEX: Ingest
>>> - satellite::
>>> /awips2/data_store/goesr/noaaport/TIUR08_KNES_301708_PAA_917282.gini
>>> processed in: 0.0030 (sec) Latency: 0.1670 (sec)
>>> INFO 2020-12-30 17:09:33,211 9400 [Ingest.Gini-1] Ingest: EDEX: Ingest
>>> - satellite::
>>> /awips2/data_store/goesr/noaaport/TIUR04_KNES_301708_PAA_917283.gini
>>> processed in: 0.0040 (sec) Latency: 0.1760 (sec)
>>>
>>
>> However the data is still not displayable in AWIPS CAVE ("No Data
>> Available") and the log above seems to indicate it's being processed as a
>> generic "Gini-1" instead of "GOESR-2" like the rest of the GOES-17 products.
>>
>> Has anyone else had similar issues with the NOAAPORT GOES-17 Mesoscale
>> data? Is there another configuration file I might need to change to get it
>> to recognize these files as GOES-17 HDFs?
>>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> awips2-users mailing list
> awips2-users@xxxxxxxxxxxxxxxx
> For list information, to unsubscribe, or change your membership options,
> visit: https://www.unidata.ucar.edu/mailing_lists/
>
--
Tiffany Meyer
AWIPS Lead Software Engineer IV
UCAR - Unidata
