Re: [conduit] 20191123: Spotty CONDUIT reception last 24 hours

  • To: Gilbert Sebenste <gilbert@xxxxxxxxxxxxxxxx>
  • Subject: Re: [conduit] 20191123: Spotty CONDUIT reception last 24 hours
  • From: Pete Pokrandt <poker@xxxxxxxxxxxx>
  • Date: Sun, 24 Nov 2019 02:07:57 +0000
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=aos.wisc.edu; dmarc=pass action=none header.from=aos.wisc.edu; dkim=pass header.d=aos.wisc.edu; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XaNeQmj0yOk4sG4+BUGC8PtGVSMn/ox6BAkIqvhVXJg=; b=obi/BntzP2SuSZryX9zZJZYPx9Q/TjPs4WoyaH79seZ3rr3GLH4RKOtpYQdjAACptSbKJIxzDsJ9cJg78UJQY5Li3drRtlpL7eSpdxtEdjtoouAokA/uzIQkQ6EayGm+tIUYtdmP2jF1DVsOSMunk795HfL3cFf3zERYtz85TJwBiQF8QQGqOPVcVmpLwlMTObhXrDhBAwGXWR8dQWZqZsDFyWq+YwJoSuKsVh4ZKDexLIE28Pe50YRi0EQO0U5JQJ7g9beBlnW8JkeKUXltvfz+dxlU2pmVVDoAQJF8D+RGVOH2gpj29k4wTqcLquu/Pui3xza0gQNwUZu8J+26yA==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MUS5OwTjuunzLOTGo50p5cCE7CInklhAuMwbBCUTIa8e28zFftEJvxTCMbolhQk/2YxoZsfbN8fOnOROfI25PUjzGCZGxFFKgHFTfZHdYfPGWEnTSAijVOTrt4bwqE30mIuJAZ4nGhywfKJJuZTcjCGx6pqebJTKrhE27V3mWqbga/C11gaMhjBxKa8sYZslqnHyqGv1pMH57l7TmH0MLCxQeQq1WgMizgN3SKdzN0FOZN0KjR9bL/lFeqvQzsLk7NboPGfbiebaiWvj87TZwV1awjFQuxS73Avqof0c1Nl3uFcD+G5EaW8CTAizw3Y/erLqJ+a1cj2H/Su+qSaFeg==
  • Authentication-results: spf=none (sender IP is ) smtp.mailfrom=poker@xxxxxxxxxxxx;
The files do appear to be on the Ncep ftp server, so it looks like a CONDUIT 
ingest problem.

Pete

Sent from my iPhone

On Nov 23, 2019, at 7:55 PM, Gilbert Sebenste <gilbert@xxxxxxxxxxxxxxxx> wrote:


Try stopping the LDM, remaking the queue, and starting again. Network issues 
can cause this to happen.

Gilbert

On Nov 23, 2019, at 7:49 PM, Pete Pokrandt via conduit 
<conduit@xxxxxxxxxxxxxxxx> wrote:


So I've had a chance to look at this now - the files that are coming through 
appear to be complete, but there are entire forecast hour files that are just 
missing.

For the 12 UTC 20191123 GFS 1 degree data, we are missing the 15h, 87h, 162h, 
168h, 303h and 354h forecasts completely. Everything else appears to be there.

For the 18 UTC 20191123 GFS 1 degree data, we are missing the 24h, 117h, 303h, 
330h, and 348h forecasts completely. Everything else is there.

The missing forecast hours for the 0.5 deg GFS are different than for the 1 
deg. for the 12 UTC 20191123 run I'm missing the 240h and 354h completely - 
everything else appears to be there.

For the 18 UTC 20191123 0.5 deg GFS run, I'm missing the 51h, 72h, 108h, 225h, 
249h, 267h, and 360h forecasts completely. everything else appears complete.

Definitely seems to be a problem on the ingest side.

Pete



<http://www.weather.com/tv/shows/wx-geeks/video/the-incredible-shrinking-cold-pool>-----
Pete Pokrandt - Systems Programmer
UW-Madison Dept of Atmospheric and Oceanic Sciences
608-262-3086  - poker@xxxxxxxxxxxx

________________________________
From: conduit <conduit-bounces@xxxxxxxxxxxxxxxx> on behalf of Pete Pokrandt via 
conduit <conduit@xxxxxxxxxxxxxxxx>
Sent: Saturday, November 23, 2019 12:59 PM
To: Tyle, Kevin R <ktyle@xxxxxxxxxx>; support@xxxxxxxxxxxxxxxx 
<support@xxxxxxxxxxxxxxxx>
Cc: GREGORY J TRIPOLI <tripoli@xxxxxxxxxxxx>; conduit@xxxxxxxxxxxxxxxx 
<conduit@xxxxxxxxxxxxxxxx>
Subject: Re: [conduit] 20191123: Spotty CONDUIT reception last 24 hours

Upon further review, we do seem to be having issues. Our 12 UTC GFS model 
updates stopped at 12 UTC, but I don't know why, the files look the right size 
and apparently were the size that was specified in the .status file, or else my 
grib2 to grib1 conversions would have stopped working, and they all worked fine.

I have to leave in a few minutes and can't look further into this until tonight.

Pete


<http://www.weather.com/tv/shows/wx-geeks/video/the-incredible-shrinking-cold-pool>-----
Pete Pokrandt - Systems Programmer
UW-Madison Dept of Atmospheric and Oceanic Sciences
608-262-3086  - poker@xxxxxxxxxxxx

________________________________
From: conduit <conduit-bounces@xxxxxxxxxxxxxxxx> on behalf of Tyle, Kevin R 
<ktyle@xxxxxxxxxx>
Sent: Saturday, November 23, 2019 12:06 PM
To: support@xxxxxxxxxxxxxxxx <support@xxxxxxxxxxxxxxxx>
Cc: conduit@xxxxxxxxxxxxxxxx <conduit@xxxxxxxxxxxxxxxx>
Subject: Re: [conduit] 20191123: Spotty CONDUIT reception last 24 hours

Thanks Tom ...

Hmm, this seems to contradict Pete's observation recently posted on the CONDUIT 
list. Also, to add to the conundrum, UW-MSN is our primary upstream CONDUIT 
feed ...

_____________________________________________
Kevin Tyle, M.S.; Manager of Departmental Computing
Dept. of Atmospheric & Environmental Sciences
University at Albany
Earth Science 228, 1400 Washington Avenue
Albany, NY 12222
Email: ktyle@xxxxxxxxxx
Phone: 518-442-4578
_____________________________________________
________________________________
From: Unidata User Support <support@xxxxxxxxxxxxxxxx>
Sent: Saturday, November 23, 2019 12:23 PM
To: Tyle, Kevin R <ktyle@xxxxxxxxxx>
Cc: support-conduit@xxxxxxxxxxxxxxxx <support-conduit@xxxxxxxxxxxxxxxx>
Subject: 20191123: [conduit] Spotty CONDUIT reception last 24 hours

Hi Kevin,

re:
>I've noticed a bunch of missed products since yesterday afternoon on our CO=
>NDUIT feed. Anyone else?

The real-time stats volumes for both Unidata (conduit.unidata.ucar.edu)
and UW/AOS (idd-agg.aos.wisc.edu) are measurably less than for the same
times yesterday.  FYI: both of these machines REQUESTs their CONDUIT
feed directly from NCEP.

Here are links to the CONDUIT volumes referred to above:

idd-agg.aos.wisc.edu
https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?CONDUIT+idd-agg.aos.wisc.edu

conduit.unidata.ucar.edu
https://rtstats.unidata.ucar.edu/cgi-bin/rtstats/siteindex?conduit.unidata.ucar.edu

My conclusion is that the products you note as being missed were never sent.

Cheers,

Tom
--
********************************************************************** <
Unidata User Support                              UCAR Unidata Program <
(303)497-8643                                            P.O. Box 3000 <
support@xxxxxxxxxxxxxxxx                             Boulder, CO 80307 <
---------------------------------------------------------------------- <
Unidata Web Support               http://www.unidata.ucar.edu/support  <
---------------------------------------------------------------------- <
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


conduit mailing list
conduit@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: 
https://www.unidata.ucar.edu/mailing_lists/
  • 2019 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the conduit archives: