[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[AWIPS #AND-239869]: AWS GOES ABI Data not being processed by EDEX

> Greetings.   Since the upgrade to 18.2.1-5 (RHEL 7) we have not been able to 
> get EDEX to process the AWS ABI files properly.  Here are the details
> 
> We pull a file from the AWS server
> OR_ABI-L2-CMIPF-M6C13_G16_s20221882200216_e20221882209536_c20221882210036.nc
> 
> After submitting via pqinsert we get a normal message which looks good...
> 20220707T221949.575845Z pqinsert[30638]             pqinsert.c:main:439       
>           INFO    27385307 20220707221945.924362 NIMAGE 000  
> OR_ABI-L2-CMIPF-M6C13_G16_s20221882200216_e20221882209536_c20221882210036.nc
> 
> And we see in /awips2/data_store/GOES/abi a corresponding listing which looks 
> good...
> -rw-r--r-- 1 root  root   27385307 Jul  7 18:20 
> OR_ABI-L2-CMIPF-M6C13_G16_s20221882200216_e20221882209536_c20221882210036.nc
> 
> But we see in our log file 
> /awips2/edex/logs/edex-ingest-satellite-20220707.log  the following warning...
> WARN  2022-07-07 22:20:00,816 4793 [Ingest.GOESR-1] GoesrNetcdfDecoder: No 
> valid records were found in file: 
> OR_ABI-L2-CMIPF-M6C13_G16_s20221882200216_e20221882209536_c20221882210036.nc
> 
> We did make the following configurations too..
> 
> 1.  cd 
> /awips2/edex/data/utility/common_static/base/satellite/goesr/descriptions/Level2
> 
> wget 
> https://downloads.unidata.ucar.edu/awips2/current/files/CMIP_Descriptions.tar
> 
> tar -xvf CMIP_Descriptions.tar (contains GOESCMIDataDescription.xml  
> GOESCMIEastSectors.xml GOESCMITestScectors.xml  GOESCMIWestSectors.xml )
> 
> 
> 
> 1.  Added at the end of /awips2/ldm/etc/pqact.conf (where [TAB] is a literal 
> tab)
> 
> # ABI GOES R FE
> 
> NIMAGE[TAB]OR_ABI-L2-(.*).nc
> 
> [TAB]FILE[TAB]-close -edex[TAB]/awips2/data_store/GOES/abi/OR_ABI-L2-\1.nc
> 
> 
> 
> 1.  As awips user  sudo edex stop    (verified all was stopped including the 
> ldm) then restarted with sudo edex start (and verified all was running  with 
> edex status)
> [edex status]
> postgres    :: running :: pid 11483
> pypies      :: running :: pid 5387
> qpid        :: running :: pid 11971
> EDEXingest  :: running :: pid 7963 12682
> EDEXgrib    :: running :: pid 6798 12815
> EDEXrequest :: running :: pid 9434 12963
> ldmadmin    :: running :: pid 43208
> 
> We are not sure why the decoder is stating "no valid records" and hence not 
> processing the files and suspect either we did NOT need to make these configs 
> with the upgrade OR there is a piece we are missing.
> 
> Could your recommend a next step for us to solve this issue?
> 

Hi Pete,

Try moving the untarred files (GOESCMI*) from the Level2 directory to 
Sectorized_CMI and then restart EDEX. 

/awips2/edex/data/utility/common_static/base/satellite/goesr/descriptions/Sectorized_CMI

I just tested this with the exact file from AWS that you were working with and 
it ended up working. I'll also add the CMIP tar files into our base release so 
next time you shouldn't have to worry about it. 

Thanks,

Tiffany Meyer
AWIPS Lead Software Engineer
UCAR-Unidata

If you're interested, please feel free to fill out a survey about the support 
you receive:  
https://docs.google.com/forms/d/e/1FAIpQLSeDIkdk8qUMgq8ZdM4jhP-ubJPUOr-mJMQgxInwoAWoV5QcOw/viewform

Ticket Details
===================
Ticket ID: AND-239869
Department: Support AWIPS
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.