Re: [ldm-users] NEXRAD2 Latency/Fragmented Data?

Both.  The stats from the UNIDATA sites show our servers almost an hour behind 
a lot of the time, and we only get partial L2 scans.

I'm changing to your syntax in pqact.conf to see if that sorts it out.

Sorry for the delay in replying, and thanks for the help.

Blair

On 9 Jul, 2012, at 22:14 , Jeff Lake wrote:

> Are you seeing the latency incoming??
> Or disk write ??
> 
> I'm not seeing it on the system your feeding from ..
> 
> your pqact is a little odd ...
> here is what I'm using
> 
> NEXRAD2    
> ^L2-BZIP2/(....)/([0-9][0-9][0-9][0-9][0-1][0-9][0-3][0-9])([0-2][0-9][0-5][0-9]).*
>     FILE    -close    /usr/local/ldm/data/raw/level2/\1/\1_\2_\3
> -Jeff Lake
> MichiganWxSystem.com
> WeatherMichigan.net
> TheWeatherCenter.net
> GRLevelXStuff.com
> On 7/9/2012 22:18, Blair Trosper wrote:
>> We are (and have been for some time) seeing a huge latency (between 30 to 90 
>> minutes) on the NEXRAD2 feed.  At present, our constellation of three 
>> servers are pulling from Texas A&M (bigbird & idd) and Michigan Weather 
>> Systems.
>> 
>> Are there network issues or something I might be missing?
>> 
>> Additionally, I'm noticing that the data we do receive is often incomplete.  
>> For example, the most recent scan I received for KFWS (who has been in 212 
>> for the last few hours) included only 7 of the 14 tilts for reflectivity, 
>> and some of the volume scans (like VIL) were absent entirely.
>> 
>> It's entirely possible that I need to tweak my pqact.conf settings, but it 
>> seems like we didn't start having issues until the latency kicked up in the 
>> last week or two.  Here is what we have now (an excerpt from 
>> pqact.conf...feel free to correct me if we've got this wrong):
>> 
>> # L2 data
>> NEXRAD2 ^L2-BZIP2/(....)/([0-9]{8})([0-9]{4})[0-9]{2}/([0-9]{1,4}).*/[SI]
>>         FILE    data/nexrad2/\1/\1_\2-\3.part
>> NEXRAD2 ^L2-BZIP2/(....)/([0-9]{8})([0-9]{4})[0-9]{2}/([0-9]{1,4}).*/E
>>         FILE    -close data/nexrad2/\1/\1_\2-\3.part
>> NEXRAD2 ^L2-BZIP2/(....)/([0-9]{8})([0-9]{4})[0-9]{2}/([0-9]{1,4}).*/E
>>         EXEC    safemv data/nexrad2/\1/\1_\2-\3.part data/nexrad2/\1/\1_\2-\3
>> 
>> If anyone else would be willing to provide an "ALLOW" entry for one or two 
>> of our servers for NEXRAD2, we would be very appreciative.  (We're bringing 
>> on several new servers over the next two or three months…with at least one 
>> being 1Gbps…and we will be happy to share the bandwidth and spread the love.)
>> 
>> Thanks,
>> 
>> Blair Trosper
>> Weather Data / Updraft Networks
>> blair.trosper@xxxxxxxxx
>> 
>> 
>> _______________________________________________
>> ldm-users mailing list
>> ldm-users@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe,  visit: 
>> http://www.unidata.ucar.edu/mailing_lists/ 
> 
> 
> _______________________________________________
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/

  • 2012 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: