[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[AWIPS #LFY-616594]: notifyAWIPS2-unidata.py question

Hi Travis,

> Thanks again for the help yesterday. Per your recommendation, I was able to 
> include
> a for loop that executes notifyAWIPS2-unidata.py for the files in my 
> directory.
> However, when I go into the hdf5 directory for my new model in edex, not all 
> of
> the files are there. I have 65 files in that directory to be ingested, but 
> only
> 11 appear (even after about 20 mins). Does it take a long time for edex to 
> process
> them, or did something go wrong with the ingest? When I ran this in a 
> terminal, I 
> could see the connection to Qpid opened and closed for each of the files 
> followed
> by a 0.

Good questions!  So, to answer your initial assumption 1 ingested file does not 
always
equal 1 decoded HDF5 file.  Data from several of the ingested files (raw files) 
can
end up in one HDF5 file, which could explain why you have less HDF5 files than 
you had
raw files.

There are a couple things you can do to verify all of your data is indeed 
getting 
ingested though.  The first is,  you can watch the edex-ingest log as you're 
running
your script with the for loop.  (You can also view it afterwards, so you don't 
have
to "watch" it in real time).  But, if you wanted to watch it, you'd open a new 
terminal
and type:
  tail -f /awips2/edex/logs/edex-ingestGrib-[date].log
Then that will update as things are written to the log.  So you could then run 
your 
script with the for loop, and you will see each of the individual files 
entering EDEX.

Another thing to do, you could add a simple `echo` statement in your current 
script,
to read out the name of the file you're passing to notifyAWIPS2-unidata.py.  
That way
you could verify all of the files you're expecting, do indeed get called with 
the notify
script.

Finally, you could open another terminal (like before), and watch the qpid 
queue.  You 
do this by running:
  edex qpid
This should show you the individual entries for each of your raw files.

Let me know if this makes sense and if it helps?
Thanks!

--Shay Carter

She/Her/Hers
AWIPS Software Engineer
UCAR - Unidata

If you're interested, please feel free to fill out a survey about the support 
you receive: 
https://docs.google.com/forms/d/e/1FAIpQLSeDIkdk8qUMgq8ZdM4jhP-ubJPUOr-mJMQgxInwoAWoV5QcOw/viewform

Ticket Details
===================
Ticket ID: LFY-616594
Department: Support AWIPS
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.