Re: [ldm-users] NEXRAD2 Latency/Fragmented Data?

• To: ldm-users@xxxxxxxxxxxxxxxx
• Subject: Re: [ldm-users] NEXRAD2 Latency/Fragmented Data?
• From: Gilbert Sebenste <sebenste@xxxxxxxxxxxxxxxxxxxxx>
• Date: Mon, 9 Jul 2012 23:50:27 -0500 (CDT)

On Mon, 9 Jul 2012, Blair Trosper wrote:

We are (and have been for some time) seeing a huge latency (between 30 to
90 minutes) on the NEXRAD2 feed.  At present, our constellation of three
servers are pulling from Texas A&M (bigbird & idd) and Michigan Weather
Systems.
Are there network issues or something I might be missing?

Not that I have been seeing...mine has been fine all day today.

Additionally, I'm noticing that the data we do receive is often
incomplete.  For example, the most recent scan I received for KFWS (who has
been in 212 for the last few hours) included only 7 of the 14 tilts for
reflectivity, and some of the volume scans (like VIL) were absent entirely.

That sounds like network congestion, or...

It's entirely possible that I need to tweak my pqact.conf settings, but it
seems like we didn't start having issues until the latency kicked up in the
last week or two.  Here is what we have now (an excerpt from
pqact.conf...feel free to correct me if we've got this wrong):

***

NEXRAD2 ^L2-BZIP2/(....)/([0-9]{8})([0-9]{4})[0-9]{2}/([0-9]{1,4}).*/[SI]
       FILE    data/nexrad2/\1/\1_\2-\3.part
NEXRAD2 ^L2-BZIP2/(....)/([0-9]{8})([0-9]{4})[0-9]{2}/([0-9]{1,4}).*/E
       FILE    -close data/nexrad2/\1/\1_\2-\3.part
NEXRAD2 ^L2-BZIP2/(....)/([0-9]{8})([0-9]{4})[0-9]{2}/([0-9]{1,4}).*/E
       EXEC    safemv data/nexrad2/\1/\1_\2-\3.part
data/nexrad2/\1/\1_\2-\3

Nothing out of the ordinary. What is your ldmd.conf entries look like?
You can erase the requested machine for privacy.

Gilbert

*******************************************************************************
Gilbert Sebenste                                                    ********
(My opinions only!)                                                  ******
Staff Meteorologist, Northern Illinois University                      ****
E-mail: sebenste@xxxxxxxxxxxxxxxxxxxxx                                  ***
web: http://weather.admin.niu.edu                                      **
Twitter: http://www.twitter.com/NIU_Weather                            **
Facebook: http://www.facebook.com/niu.weather                           *
*******************************************************************************

• References:
  • [ldm-users] NEXRAD2 Latency/Fragmented Data?
    • From: Blair Trosper