Generally a larger volume of data and too small of a queue. I'm
assuming a sustained rate of 6-10Mbits for the NEXRAD II feed, that
means to keep an hours worth of data in the queue would take a queue
size of 2.7GBytes to 4.5GBytes. Of course you don't have to have an
hours worth of data in the queue as long as the pqact can process the
data in the queue before they drop out of the queue. By processing I
mean to pass it down stream and pqact.
You can multi-stream the pqacts and gain some performance.
When Dual Pol comes around you'll need a 9 GByte queue and then the
AVSET might add another 20% onto that.
Alan.
Gilbert Sebenste said the following on 7/9/2009 10:40 AM:
On Thu, 9 Jul 2009, Gilbert Sebenste wrote:
Started yesterday, central region for sure, I haven't checked the
other regions yet. Here are my latest files for KLOT
(Romeoville/Chicago)...
-rw-rw-r-- 1 ldm users 2255367 Jul 9 08:58 KLOT_20090709_1322
Notice, for example, the 1322Z data file is twice the normal size and
came in about 40 minutes late.
I also just saw something using pqmon. (Darryl, just saw your message).
I would have to increase my queue size to nearly 6 GB to stay ahead of
this. No, no, and heck, no! I'm trying to run my queue out of
/dev/shm, to which I have 1.3 GB to do so. It says I am only saving
"558" seconds worth of data with 1.2 GB of queue. That's insane! Even
if the products get rejected, I still have to reject them by getting
them first, correct?
OK. So when I use idd.aos.wisc.edu, flood.atmos.uiuc.edu,
idd.unidata.ucar.edu or bigbird.tamu.edu singularly, that is, as the
primary feed with no backups, I don't get problems. But I just tried
it now with
request NEXRAD2 ".*" notam.atmos.uiuc.edu
request NEXRAD2 ".*" bigbird.tamu.edu
In order...and I get the double feed. I tried it in reverse, same
thing (with bigbird as primary). Tried it with other sites...same
results. And when I do an ldmadmin watch, look what I see:
Jul 09 14:36:01 pqutil INFO: 38876 20090709135846.462 NEXRAD2 49042
L2-BZIP2/KJAX/20090709135626/49/42/I/V03/0
Jul 09 14:36:01 pqutil INFO: 30687 20090709133606.451 NEXRAD2
626047 L2-BZIP2/KOAX/20090709133327/626/47/I/V03/0
Jul 09 14:36:01 pqutil INFO: 15952 20090709135846.302 NEXRAD2
507050 L2-BZIP2/KVNX/20090709135452/507/50/I/V03/0
Jul 09 14:36:01 pqutil INFO: 53219 20090709135845.193 NEXRAD2
570006 L2-BZIP2/KGWX/20090709135641/570/6/I/V04/0
Jul 09 14:36:01 pqutil INFO: 23850 20090709133606.815 NEXRAD2
906043 L2-BZIP2/KPUX/20090709133401/906/43/I/V03/0
Jul 09 14:36:01 pqutil INFO: 51674 20090709143600.836 NEXRAD2
533008 L2-BZIP2/KILX/20090709143434/533/8/I/V03/0
Jul 09 14:36:01 pqutil INFO: 18361 20090709133606.928 NEXRAD2
803027 L2-BZIP2/KIWA/20090709133443/803/27/I/V03/0
Jul 09 14:36:01 pqutil INFO: 45660 20090709135845.333 NEXRAD2 57012
L2-BZIP2/KRLX/20090709135810/57/12/I/V03/0
Jul 09 14:36:01 pqutil INFO: 15858 20090709143601.012 NEXRAD2
233018 L2-BZIP2/KBLX/20090709143228/233/18/I/V03/0
Jul 09 14:36:01 pqutil INFO: 21409 20090709135846.469 NEXRAD2
384016 L2-BZIP2/KLVX/20090709135726/384/16/I/V03/0
Jul 09 14:36:01 pqutil INFO: 10574 20090709135846.597 NEXRAD2 87029
L2-BZIP2/KFSD/20090709135708/87/29/I/V03/0
Jul 09 14:36:01 pqutil INFO: 32569 20090709143559.869 NEXRAD2
786017 L2-BZIP2/KINX/20090709143517/786/17/I/V03/0
Jul 09 14:36:01 pqutil INFO: 42332 20090709135847.305 NEXRAD2
701014 L2-BZIP2/KDMX/20090709135804/701/14/I/V03/0
Jul 09 14:36:01 pqutil INFO: 20409 20090709143600.580 NEXRAD2 61033
L2-BZIP2/KRLX/20090709142705/61/33/I/V03/0
Jul 09 14:36:01 pqutil INFO: 38611 20090709135847.733 NEXRAD2
777042 L2-BZIP2/KINX/20090709135647/777/42/I/V03/0
Jul 09 14:36:01 pqutil INFO: 10000 20090709143601.246 NEXRAD2
512014 L2-BZIP2/KARX/20090709143452/512/14/I/V04/0
Jul 09 14:36:01 pqutil INFO: 29056 20090709143559.812 NEXRAD2
689019 L2-BZIP2/KLZK/20090709143510/689/19/I/V03/0
Jul 09 14:36:01 pqutil INFO: 67204 20090709133606.339 NEXRAD2
709013 L2-BZIP2/KIND/20090709133526/709/13/I/V03/0
Obviously delayed duplicates, and I JUST cleared/deleted and remade
the queues as the above was happening, so my queues weren't full. This
never used to happen before. What's going on?
*******************************************************************************
Gilbert Sebenste
********
(My opinions only!)
******
Staff Meteorologist, Northern Illinois University
****
E-mail:
sebenste@xxxxxxxxxxxxxxxxxxxxx ***
web: http://weather.admin.niu.edu **
*******************************************************************************
_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
begin:vcard
fn:Alan D. Hall
n:Hall;Alan D.
org:NOAA's National Climatic Data Center;US Department of Commerce
adr:;;151 Patton Ave.;Asheville;NC;28801-5001;USA
email;internet:Alan.Hall@xxxxxxxx
title:IT Specialist
tel;work:(828)271-4071
tel;fax:(828)271-4022
url:www.ncdc.noaa.gov
version:2.1
end:vcard
