Re: [conduit] High CONDUIT latencies from vm-lnx-conduit2.ncep.noaa.gov

• To: "anne.myckow@xxxxxxxx" <anne.myckow@xxxxxxxx>, "dustin.sheffler@xxxxxxxx" <dustin.sheffler@xxxxxxxx>, "ncep.list.pmb-dataflow@xxxxxxxx" <ncep.list.pmb-dataflow@xxxxxxxx>
• Subject: Re: [conduit] High CONDUIT latencies from vm-lnx-conduit2.ncep.noaa.gov
• From: Pete Pokrandt <poker@xxxxxxxxxxxx>
• Date: Fri, 20 Aug 2021 16:53:53 +0000
• Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=aos.wisc.edu; dmarc=pass action=none header.from=aos.wisc.edu; dkim=pass header.d=aos.wisc.edu; arc=none
• Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KzekOz1Ih+nI52wg8MbeU4LEHUS1ZQeg2CFTRFng1ME=; b=jX/dHeJK2sLrLqmeJcEX29Wke1jeIAW5xhcg4S/ymktv6XQuSeC/c/C2NLlg84Vj8lkNs8Hp42cEY6S04faxjDsncdCTSOxtqnYg7l08QvFqnn7iPiCBJaWpus35BXZc5LrM6sBcZnmYBeZ7RrhlQzJIpHdtiWlwnn7viCUwruwnsCdoQZog0aR3nyKEoHznEFO0xJen1Yj1hhuFKYlWWLNTHt22CeypBFR853tCkQ42BxGxd8IqEH5Q+5TIKkhFiF+ycvRzI7nfmR7XxCz3FGBr/l7UWQ5Y0sEQSJg7d+3/WGhRg3ilsBdLI2wyBUHc3nT3qSAMRbmoc/BK/fVZDw==
• Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LYnk3EnOqSJcvtGPLuEa+CVIyk1QXFUkbIYRsPBUMVTYVyM7hPYIaFVbdsXOXUnkLe9jqQMZs8le8iuZrID1r4LB1sAe9mm7yEjGuhpH7kouFGpVWp20+xzmCPteDzzKkPNWiSvRQpQg54y/EEnrfayWvL7v+UqO+7/rnOeA3i2x3HA/TVdXs5AHafCV0WujJpguDUh/PHgpcwR/taIbadQMi1YhvgsnvlDpls5wtp+EFMrFNJakehR3FssWjUmDRDOkPLZysFWiF+bj+YjaBA9wZktDrI7YvcyFkF2taXM22jUiwdB9Vel0nZBUOFToDkLmh6SDfddqtPZaqCfd5g==

Dear Anne, Dustin and all,

Did you see this? We are still experiencing high latencies of 800-1000 seconds 
on our CONDUIT feeds during the times when the GFS comes through that appear to 
be coming from the host

vm-lnx-conduit2.ncep.noaa.gov

Here are the most recent lags. Any ideas?

Thanks,
Pete

https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd-agg.aos.wisc.edu

[cid:26e210d0-e3ca-4fe0-9523-f364138e9bd8]



https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+conduit.unidata.ucar.edu
[cid:f260bb3b-acb1-4e23-a4e8-f995d651c517]



<http://www.weather.com/tv/shows/wx-geeks/video/the-incredible-shrinking-cold-pool>-----
Pete Pokrandt - Systems Programmer
UW-Madison Dept of Atmospheric and Oceanic Sciences
608-262-3086  - poker@xxxxxxxxxxxx

________________________________
From: Pete Pokrandt
Sent: Wednesday, August 18, 2021 3:02 PM
To: anne.myckow@xxxxxxxx <anne.myckow@xxxxxxxx>; dustin.sheffler@xxxxxxxx 
<dustin.sheffler@xxxxxxxx>; ncep.list.pmb-dataflow@xxxxxxxx 
<ncep.list.pmb-dataflow@xxxxxxxx>
Cc: conduit@xxxxxxxxxxxxxxxx <conduit@xxxxxxxxxxxxxxxx>; 
support-conduit@xxxxxxxxxxxxxxxx <support-conduit@xxxxxxxxxxxxxxxx>
Subject: High CONDUIT latencies from vm-lnx-conduit2.ncep.noaa.gov

Dear Anne, Dustin and all,

Recently we have noticed fairly high latencies on the CONDUIT ldm data feed 
originating from the machine vm-lnx-conduit2.ncep.noaa.gov. The feed 
originating from vm-lnx-conduit1.ncep.noaa.gov does not have the high 
latencies. Unidata and other top level feeds are seeing similar high latencies 
from vm-lnx-conduit2.ncep.noaa.gov.

Here are some graphs showing the latencies that I'm seeing:

>From 
>https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+idd-agg.aos.wisc.edu
> - latencies for CONDUIT data arriving at our UW-Madison AOS ingest machine

[cid:cb4fda71-be88-450e-bf39-4346eb77ba50]

>From 
>https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/siteindex?conduit.unidata.ucar.edu
> (latencies at Unidata)

[cid:50b9a476-503f-4e75-b22b-4bc99eff3cdc]

At least here at UW-Madison, these latencies are causing us to lose some data 
during the large GFS/GEFS periods.

Any idea what might be causing this?

Pete





<http://www.weather.com/tv/shows/wx-geeks/video/the-incredible-shrinking-cold-pool>-----
Pete Pokrandt - Systems Programmer
UW-Madison Dept of Atmospheric and Oceanic Sciences
608-262-3086  - poker@xxxxxxxxxxxx

• Follow-Ups:
  • Re: [conduit] High CONDUIT latencies from vm-lnx-conduit2.ncep.noaa.gov
    • From: Anne Myckow - NOAA Federal

• References:
  • [conduit] High CONDUIT latencies from vm-lnx-conduit2.ncep.noaa.gov
    • From: Pete Pokrandt