Okay, here's what I've been able to dig up. It looks like there's a delay of
30 minutes or more in _some_ CR products. A few seconds of ldmadmin watch
turned up these (I ignored anything that was less than 10 minutes late):
Jul 04 14:54:13 pqutil INFO: 61365 20090704142559.640 NEXRAD2 558018
L2-BZIP2/KSGF/20090704142506/558/18/I/V03/0
Jul 04 14:54:14 pqutil INFO: 15786 20090704142559.931 NEXRAD2 84028
L2-BZIP2/KLNX/20090704141924/84/28/I/V03/0
Jul 04 14:54:20 pqutil INFO: 48059 20090704142600.113 NEXRAD2 586004
L2-BZIP2/KDMX/20090704142551/586/4/I/V03/0
Jul 04 14:54:21 pqutil INFO: 55683 20090704142600.513 NEXRAD2 697049
L2-BZIP2/KDVN/20090704142326/697/49/I/V03/0
Jul 04 14:54:27 pqutil INFO: 8958 20090704142602.097 NEXRAD2 541063
L2-BZIP2/KOAX/20090704142141/541/63/I/V03/0
Jul 04 14:54:32 pqutil INFO: 23970 20090704142602.264 NEXRAD2 929039
L2-BZIP2/KICT/20090704142354/929/39/I/V03/0
Jul 04 14:54:35 pqutil INFO: 60772 20090704142600.912 NEXRAD2 647018
L2-BZIP2/KLSX/20090704142507/647/18/I/V03/0
Jul 04 14:54:37 pqutil INFO: 68477 20090704142602.760 NEXRAD2 414046
L2-BZIP2/KILX/20090704142320/414/46/I/V03/0
Jul 04 14:54:39 pqutil INFO: 31960 20090704142603.111 NEXRAD2 768023
L2-BZIP2/KDLH/20090704142124/768/23/I/V03/0
Jul 04 14:54:51 pqutil INFO: 21836 20090704142617.992 NEXRAD2 659003
L2-BZIP2/KMQT/20090704142554/659/3/I/V03/0
Jul 04 14:54:52 pqutil INFO: 43730 20090704142618.445 NEXRAD2 281057
L2-BZIP2/KMPX/20090704142317/281/57/I/V03/0
Jul 04 14:54:53 pqutil INFO: 27219 20090704142618.479 NEXRAD2 311038
L2-BZIP2/KARX/20090704142314/311/38/I/V04/0
Jul 04 14:54:53 pqutil INFO: 22742 20090704142618.867 NEXRAD2 697053
L2-BZIP2/KDVN/20090704142326/697/53/I/V03/0
So I went back and looked at file write times. It looks like CR sites started
seeing a delay (generally ~30 min) around 4Z on 2 July. This was fairly
consistent in the sites I looked at. Sites in other regions did not exhibit
the same latency. (Time stamps are in UTC-4)
-rw-r--r-- 1 ldm ldm 1130776 Jul 1 23:32 KDMX_20090702_0322
-rw-r--r-- 1 ldm ldm 1152867 Jul 1 23:42 KDMX_20090702_0332
-rw-r--r-- 1 ldm ldm 1163772 Jul 1 23:51 KDMX_20090702_0342
-rw-r--r-- 1 ldm ldm 1864256 Jul 2 00:15 KDMX_20090702_0351
-rw-r--r-- 1 ldm ldm 2328948 Jul 2 00:40 KDMX_20090702_0401
-rw-r--r-- 1 ldm ldm 2323290 Jul 2 00:57 KDMX_20090702_0411
-rw-r--r-- 1 ldm ldm 1752044 Jul 2 01:08 KDMX_20090702_0421
-rw-r--r-- 1 ldm ldm 1865551 Jul 2 01:17 KDMX_20090702_0431
-rw-r--r-- 1 ldm ldm 1817208 Jul 2 01:27 KDMX_20090702_0441
-rw-r--r-- 1 ldm ldm 984866 Jul 1 23:25 KILX_20090702_0315
-rw-r--r-- 1 ldm ldm 979181 Jul 1 23:34 KILX_20090702_0325
-rw-r--r-- 1 ldm ldm 977704 Jul 1 23:44 KILX_20090702_0334
-rw-r--r-- 1 ldm ldm 975361 Jul 1 23:54 KILX_20090702_0344
-rw-r--r-- 1 ldm ldm 1946615 Jul 2 00:25 KILX_20090702_0354
-rw-r--r-- 1 ldm ldm 1958218 Jul 2 00:44 KILX_20090702_0403
-rw-r--r-- 1 ldm ldm 1918879 Jul 2 00:58 KILX_20090702_0413
-rw-r--r-- 1 ldm ldm 1484241 Jul 2 01:08 KILX_20090702_0423
-rw-r--r-- 1 ldm ldm 1546451 Jul 2 01:17 KILX_20090702_0432
-rw-r--r-- 1 ldm ldm 1700754 Jul 2 01:27 KILX_20090702_0442
-rw-r--r-- 1 ldm ldm 1650739 Jul 2 01:38 KILX_20090702_0452
-rw-r--r-- 1 ldm ldm 1451028 Jul 2 01:47 KILX_20090702_0501
-rw-r--r-- 1 ldm ldm 1903809 Jul 1 23:38 KOAX_20090702_0334
-rw-r--r-- 1 ldm ldm 1935148 Jul 1 23:43 KOAX_20090702_0338
-rw-r--r-- 1 ldm ldm 1948071 Jul 1 23:48 KOAX_20090702_0343
-rw-r--r-- 1 ldm ldm 1940662 Jul 1 23:52 KOAX_20090702_0348
-rw-r--r-- 1 ldm ldm 2830923 Jul 2 00:08 KOAX_20090702_0352
-rw-r--r-- 1 ldm ldm 3889692 Jul 2 00:15 KOAX_20090702_0357
-rw-r--r-- 1 ldm ldm 3909672 Jul 2 00:30 KOAX_20090702_0402
-rw-r--r-- 1 ldm ldm 3953484 Jul 2 00:40 KOAX_20090702_0406
-rw-r--r-- 1 ldm ldm 2449726 Jul 2 00:52 KOAX_20090702_0411
-rw-r--r-- 1 ldm ldm 2403103 Jul 2 00:59 KOAX_20090702_0417
-rw-r--r-- 1 ldm ldm 2155576 Jul 2 01:06 KOAX_20090702_0424
-rw-r--r-- 1 ldm ldm 1971551 Jul 2 01:13 KOAX_20090702_0430
Here's current data.
-rw-r--r-- 1 ldm ldm 2769972 Jul 4 11:40 KDMX_20090704_1507
-rw-r--r-- 1 ldm ldm 2762001 Jul 4 11:45 KDMX_20090704_1512
-rw-r--r-- 1 ldm ldm 2731136 Jul 4 11:48 KDMX_20090704_1517
-rw-r--r-- 1 ldm ldm 2344602 Jul 4 11:26 KDMX_20090704_1521
-rw-r--r-- 1 ldm ldm 2343846 Jul 4 11:31 KDMX_20090704_1526
-rw-r--r-- 1 ldm ldm 2314881 Jul 4 11:36 KDMX_20090704_1531
-rw-r--r-- 1 ldm ldm 2280884 Jul 4 11:41 KDMX_20090704_1536
-rw-r--r-- 1 ldm ldm 2280884 Jul 4 11:46 KDMX_20090704_1541
-rw-r--r-- 1 ldm ldm 1755661 Jul 4 11:48 KDMX_20090704_1546
-rw-r--r-- 1 ldm ldm 6049570 Jul 4 11:43 KILX_20090704_1509
-rw-r--r-- 1 ldm ldm 5955236 Jul 4 11:47 KILX_20090704_1514
-rw-r--r-- 1 ldm ldm 5823187 Jul 4 11:49 KILX_20090704_1519
-rw-r--r-- 1 ldm ldm 5616058 Jul 4 11:28 KILX_20090704_1523
-rw-r--r-- 1 ldm ldm 5597159 Jul 4 11:33 KILX_20090704_1528
-rw-r--r-- 1 ldm ldm 5592560 Jul 4 11:37 KILX_20090704_1533
-rw-r--r-- 1 ldm ldm 5589902 Jul 4 11:43 KILX_20090704_1537
-rw-r--r-- 1 ldm ldm 5568192 Jul 4 11:47 KILX_20090704_1543
-rw-r--r-- 1 ldm ldm 1021825 Jul 4 11:06 KOAX_20090704_1432
-rw-r--r-- 1 ldm ldm 904902 Jul 4 11:11 KOAX_20090704_1437
-rw-r--r-- 1 ldm ldm 927664 Jul 4 11:17 KOAX_20090704_1442
-rw-r--r-- 1 ldm ldm 553907 Jul 4 11:27 KOAX_20090704_1448
-rw-r--r-- 1 ldm ldm 556596 Jul 4 11:35 KOAX_20090704_1458
-rw-r--r-- 1 ldm ldm 549581 Jul 4 11:48 KOAX_20090704_1509
-rw-r--r-- 1 ldm ldm 543498 Jul 4 11:49 KOAX_20090704_1519
-rw-r--r-- 1 ldm ldm 512042 Jul 4 11:40 KOAX_20090704_1530
I've checked my LDM box for sanity, and it appears to be sane. I wonder
though... it looks like castor.itap.purdue.edu has MUCH higher latency than
you're seeing, so perhaps it's a problem with our top-tier site? Where are the
others who are seeing this feeding from? I've bothered the admin for
castor.itap.purdue.edu and pollux.itap.purdue.edu via his roommate on Facebook,
so perhaps he'll weigh in later this afernoon. I'm told he's already seen our
messages and is looking into the problem.
--
Ben Cotton
Unix System Administrator
Department of Earth and
Atmospheric Sciences
Purdue University
http://www.purdue.edu/eas/resources/it
CIVL 4212
O: 765 49-61546
C: 765 404-7986
-----Original Message-----
From: Gerry Creager [mailto:gerry.creager@xxxxxxxx]
Sent: Sat 7/4/2009 9:03 AM
To: Cotton, Benjamin J
Cc: sebenste@xxxxxxxxxxxxxxxxxxxxx; ldm-users@xxxxxxxxxxxxxxxx
Subject: Re: [ldm-users] Level 2 radar issues from NWS CRH
I've checked my data cache, and I don't see any "interesting" latencies
on those three sites. I suppose I could set up a script to do a better
job of monitoring things like that, though.
Last-write times were consistent with file-size for clear-air mode, and
similarly, for normal "storm" mode. Until I get more coffee, I'm not
likely to decode the files to look for when folks changed to a
storm-related VCP.
I had about an 800 sec spike in Level II latencies, as seen at
http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?NEXRAD2+bigbird.tamu.edu
about or shortly after 03/00z
gerry
Cotton, Benjamin J wrote:
> I specifically remember seeing delays for KICT, KDMX, and KLOT. That
> was only from a quick peek and I haven't looked again since this
> morning. I'll take a deeper look Saturday morning.
>
> --
> (Sent via Blackberry).
> Ben Cotton, KC9FYX
> Unix System Administrator
> Department of Earth and
> Atmospheric Sciences
> Purdue University
> http://www.purdue.edu/eas/resources/it
>
> CIVL 4212
> O: 765 49-61546
> C: 765 404-7986
>
> ----- Original Message -----
> From: ldm-users-bounces@xxxxxxxxxxxxxxxx
> <ldm-users-bounces@xxxxxxxxxxxxxxxx>
> To: Gilbert Sebenste <sebenste@xxxxxxxxxxxxxxxxxxxxx>
> Cc: ldm-users@xxxxxxxxxxxxxxxx <ldm-users@xxxxxxxxxxxxxxxx>
> Sent: Sat Jul 04 00:07:48 2009
> Subject: Re: [ldm-users] Level 2 radar issues from NWS CRH
>
> I've not done an exhaustive search, so this isn't significant from a
> sampling viewpoint, but the several sites I've looked at from CR didn't
> show a lot of latency. I recalled the CRH maintenance, like Gilbert
> did, and thought that could be playing some roll, but I'm not sure.
>
> What sites are folks seeing a problem drawing from?
>
> gerry
>
> Gilbert Sebenste wrote:
> > FYI...could this be somewhere in the UNIDATA chain?
> >
> >
> *******************************************************************************
> >
> > Gilbert Sebenste
> > ********
> > (My opinions only!)
> ******
> > Staff Meteorologist, Northern Illinois
> University ****
> > E-mail:
> sebenste@xxxxxxxxxxxxxxxxxxxxx ***
> > web: http://weather.admin.niu.edu **
> >
> *******************************************************************************
> >
> >
> > ---------- Forwarded message ----------
> > Date: Fri, 03 Jul 2009 17:59:55 -0500
> > From: Tim.D.Crum@xxxxxxxx
> > To: Gilbert Sebenste <sebenste@xxxxxxxxxxxxxxxxxxxxx>
> > Subject: Re: Level 2 radar issues from NWS CRH
> >
> > Gilbert,
> >
> > Please tell me more. The ERC, NWS and OU web sites show low Central
> > Region latencies. I spoke with ERC and they are not seeing duplicate
> > copies of Central Region Level II data.
> >
> >
> > Tim
> >
> >
> > ----- Original Message -----
> > From: Gilbert Sebenste <sebenste@xxxxxxxxxxxxxxxxxxxxx>
> > Date: Friday, July 3, 2009 1:40 pm
> > Subject: Level 2 radar issues from NWS CRH
> >> Hi Tim,
> >>
> >> Level2 radar data from Central Region appears to be going out in
> >> duplicate, with the second copy of the file delayed by up to an
> >> hour.
> >> This is only happening with NWS Central Region sites; other
> >> regions are
> >> fine. Can those delayed duplicates be shut off? This started with
> >> network
> >> maintenance last night at CR; they said they were going to do work
> >> from 9
> >> PM to midnight, and as they announced it was completed, that's
> >> when the
> >> trouble started.
> >>
> >>
> > ***********************************************************************
> > ********
> >> Gilbert Sebenste
> >> ********
> >> (My opinions only!)
> >> ******
> >> Staff Meteorologist, Northern Illinois University
> >> ****
> >> E-mail: sebenste@xxxxxxxxxxxxxxxxxxxxx
> >> ***
> >> web: http://weather.admin.niu.edu
> >> **
> >>
> > ***********************************************************************
> > ********
> >>
> >
> > _______________________________________________
> > ldm-users mailing list
> > ldm-users@xxxxxxxxxxxxxxxx
> > For list information or to unsubscribe, visit:
> > http://www.unidata.ucar.edu/mailing_lists/
>
> --
> Gerry Creager -- gerry.creager@xxxxxxxx
> Texas Mesonet -- AATLT, Texas A&M University
> Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
> Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843
>
> _______________________________________________
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
--
Gerry Creager -- gerry.creager@xxxxxxxx
Texas Mesonet -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843
