[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #GJS-316993]: Issue Ingesting NationalBlend Data on AWS Cloud Instance

Greetings,

> Unidata Support Team,
> I am experiencing difficulty ingesting NationalBlend data from my AWS cloud
> instance for our AWIPS application. I have been in conversation with my
> teammate, Andy Just from NOAA Federal, who is receiving the NationalBlend
> data without any issues using another AWS cloud instance. We compared our
> configurations and observed the following differences:
> 
> My current ldmd.conf contains the following REQUEST lines:
> REQUEST NGRID ".*" idd.unidata.ucar.edu
> REQUEST NEXRAD3 ".*" idd.unidata.ucar.edu
> REQUEST HDS ".*" idd.unidata.ucar.edu
> REQUEST IDS|DDPLUS ".*" idd.unidata.ucar.edu
> REQUEST SATELLITE ".*" idd.unidata.ucar.edu
> REQUEST NIMAGE ".*" idd.unidata.ucar.edu
> REQUEST NOTHER ".*" idd.unidata.ucar.edu
> 
> Another team configuration includes:
> 
> REQUEST ANY ".*" awips-ldm

I don't know what's on "awips-ldm" is it possible that contains different data 
than the IDD?


> Unidata had previously advised us to use multiple REQUEST lines to avoid
> data backup issues that were occurring with the REQUEST ANY ".*" awips-ldm
> line. This approach resolved our initial performance problems. However, it
> appears that my current setup may still be causing issues specifically with
> ingesting NationalBlend data.

What NBM data are you expecting to find; what patterns are you matching 
against?  There are a few different types of NBM data on NOAAPort (gridded, 
MOS, etc.).  If you can share those pattern(s) with me I could also try to look 
if we received those or not.


> My teammate mentioned that the NationalBlend data can be visualized in the
> Volume Browser in CAVE by selecting SfcGrid => NationalBlend under the
> Sources section and choosing Temp under the Fields section, followed by Sfc
> under the Planes section. They provided instructions to load the data using
> the context menu option "Load as NationalBlend Sfc Temp(F) + NationalBlend
> Sfc Temp Img (F)".

Okay it sounds like you are using gridded data.  I believe that is found on the 
NGRID feed but I could be wrong, I'll look into it.


> Given this context, it seems that the REQUEST line for the NationalBlend
> data is likely REQUEST HDS ".*" idd.unidata.ucar.edu. Could you please
> assist us in diagnosing why my instance is not ingesting this data
> correctly? Is there a specific configuration or additional REQUEST line
> that I should include to ensure successful data ingestion? --See attached
> files

I see in the ldmd.conf file you send that all of the EXEC actions are commented 
out.  Is there a chance those need to be uncommented?

If not, the two things that would help me now are any PQACT entries that you 
are trying to capture this data with, and any logging errors that exist.  I 
guess what I'm asking is, how do you know you are missing data?

Lastly, this is a small thing I think but I noticed in your ldmd.conf you have 
(EXEC "pqact -e"),  I don't believe there is a -e option for pqact 
(https://docs.unidata.ucar.edu/ldm/man/pqact.html) so I'm wondering if that's 
there for some purpose or just one of those things that got copy/pasted over 
since the dawn of time.


Best,
-Mik...   [slams on the breaks!]

I just got your second email.  OKAY, I see what data you are referring to now, 
and yes it looks like it's on the IDD, that's reassuring.  You were right & I 
was wrong, that's on HDS... TIL! 

If you're requesting HDS you should be receiving that data.  This is the NBM 
which comes in often enough, try running: `notifyme -v -h localhost -f HDS -p 
NationalBlend -o 3600` to see if you've seen it in your product queue in the 
past hour.  If you have then you are receiving the data but then nothing 
happens, i.e. it is not being acted upon, which would lead back to my above 
questions/conclusions.  If that command comes back empty, try removing the "-p 
NationalBlend" bit to see if you're getting any HDS data at all.  Let us know 
how that goes.


Best,
-Mike


Ticket Details
===================
Ticket ID: GJS-316993
Department: Support LDM
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with NSF Unidata User Support are recorded in the 
Unidata inquiry tracking system and then made publicly available through the 
web.  If you do not want to have your interactions made available in this way, 
you must let us know in each email you send to us.