[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[CONDUIT #MJC-410449]: Re: [conduit] Huge CONDUIT latencies, lost data starting ~ 00 UTC last night

Hi again Becky,

Quick update with some more information about problems experienced in
REQUESTing data from ncepldm1.woc.noaa.gov:

I just logged onto the UW/AOS machine that was unable to REQUEST data
from ncepldm1.woc.noaa.gov last week (idd.aos.wisc.edu) and reviewed
its LDM log files.   It looks like data feeds from ncepldm1 to
idd.aos.wisc.edu failed up until 10:10:42 CST on March 1 (16:10:42 UTC).
The errors reported looked like the following snippit from the LDM log
file on idd.aos.wisc.edu:

Mar  1 10:10:33 idd ncepldm1.woc.noaa.gov[15835] NOTE: LDM-6 desired 
product-class: 20130301151033.057 TS_ENDT {{CONDUIT,  "[27]$"},{NONE,  
"SIG=1e15f4a14a5ba32b3754adc5a4a2f3b7"}}
 ...
Mar  1 10:10:36 idd ncepldm1.woc.noaa.gov[15835] ERROR: Disconnecting due to 
LDM failure; nullproc_6 failure to ncepldm1.woc.noaa.gov; RPC: Unable to 
receive; errno = Connection reset by peer

At 16:10:42 UTC, the feed REQUESTs were honored and data started flowing
from ncepldm1 to idd.aos.wisc.edu.  There were no changes on idd.aos.wisc.edu
at this time, so it looks like some change was made on ncepldm1 itself, or
in a firewall through which feed REQUESTs to ncepldm1 would traverse.

That things are working well now is evidenced by real-time stats plots
for the UW/AOS and UPC machines that are REQUESTing data from ncepldm1
and ncepldm4:

UW/AOS - idd.aos.wisc.edu
http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd.aos.wisc.edu

UPC - daffy.unidata.ucar.edu
http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+daffy.unidata.ucar.edu

The setup on each machine is:

idd.aos.ssec.wisc.edu REQUESTs CONDUIT data redundantly from 
ncepldm1.woc.noaa.gov,
                               ncepldm4.woc.noaa.gov and idd.unidata.ucar.edu

daffy.unidata.ucar.edu REQUESTs CONDUIT data redundantly from 
ncepldm1.woc.noaa.gov
                                and ncepldm4.woc.noaa.gov and sends all of the 
data
                                to idd.unidata.ucar.edu
                            
Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: MJC-410449
Department: Support CONDUIT
Priority: Normal
Status: Closed