Re: [conduit] CONDUIT Insert Tables Update

  • To: "Herzmann, Daryl E [AGRON]" <akrherz@xxxxxxxxxxx>, CONDUIT Users List <conduit@xxxxxxxxxxxxxxxx>, William Boll - NOAA Federal <william.boll@xxxxxxxx>
  • Subject: Re: [conduit] CONDUIT Insert Tables Update
  • From: Pete Pokrandt <poker@xxxxxxxxxxxx>
  • Date: Wed, 18 Oct 2023 15:42:10 +0000
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=aos.wisc.edu; dmarc=pass action=none header.from=aos.wisc.edu; dkim=pass header.d=aos.wisc.edu; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=X9Iq+B+GbcmbSUcDqgoDMENEY5sM4noAS+UiiPWHRKw=; b=aJyXge+S9K2M/N8K6ZYMX90STzbZBaeKl1W/vFwhCDR5US0fwbj4I2fKc/47RczqNlVAiwluV2jt8Eodw2sG0GKvvtqFmh/KPJVZQN8j1RYyOWGlsxOa1HVRavX8MaEdszvuhshAr7FDu25ZoI8xogEu17QwBBus4M7Q4pZQqgicctZ7WewLRU5hUEgdIPD5m2Msl/FZpOerp0vb9GPExlK9IPi42Y65Lkr4YgdAWRNJtWhOW2Yu9CUkSxnl1kiBIpvX07D5X/HyleClgOvkPgQIvq3OsfD7BHPe+r0lVaxFce9NO1tVpxWKG/41HQeX1U8+rHL2roNAZN7rdH5yuw==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bnbcJZErzJiZwRUljUG8V8i4Vf4OtyCjf4v/i8IueUKXx3+GNy6kQ6OG+tD4ylOUyoeMJRLCMOThar7H4Z+V377lsUpic+AS+bwBZfp2sCgOI/JZeiPD8nPta/Y4vHfqxO6j2FT2lSWMJXJ0uqARpSCZLZ+i0p7vfa5fFEsFw2RD18oa+dKLdXOMoVSDT8naHkYo+SHrJikdVwclHLopGIEpgrC0MKVbqLgrfUQX8UNS/eUhAtOuKR1DgpDWHUExNGyAbmAFfChW3yMe1O6krUXhynC2eXf9X2rt0MGPS+jnysDhQC+5Ii2V+bdUT2DXEEl2Ed+Dc4hW0sL3hETlcg==
  • Msip_labels:
Daryl,

I'm guessing he means to get the product info back into the metadata for most 
of the model data on the CONDUIT stream.

Prior to the fix, after the file name and exclamation point, there was info 
about the forecast hour/variable/level at the end, like

20210621T163257.947961Z pqutil[37223]               pqutil.c:display_watch:1186 
        INFO       75741 20210621163239.720822 CONDUIT 064  
data/nccf/com/gfs/prod/gfs.20210621/12/atmos/gfs.t12z.pgrb2.1p00.f225 
!grib2/ncep/GFS/#000/202106211200F225/HGHT/0 hPa PRES! 000064
20210621T163257.948000Z pqutil[37223]               pqutil.c:display_watch:1186 
        INFO       77396 20210621163239.820916 CONDUIT 065  
data/nccf/com/gfs/prod/gfs.20210621/12/atmos/gfs.t12z.pgrb2.1p00.f225 
!grib2/ncep/GFS/#000/202106211200F225/TMPK/0 hPa PRES! 000065
20210621T163257.948039Z pqutil[37223]               pqutil.c:display_watch:1186 
        INFO     1213997 20210621163241.523907 CONDUIT 047  
data/nccf/com/gfs/prod/gfs.20210621/12/atmos/gfs.t12z.pgrb2.0p25.f225 
!grib2/ncep/GFS/#000/202106211200F225/SPFH/0 hPa PRES! 000047
20210621T163257.948126Z pqutil[37223]               pqutil.c:display_watch:1186 
        INFO      170943 20210621163228.510736 CONDUIT 738  
data/nccf/com/gfs/prod/gfs.20210621/12/atmos/gfs.t12z.pgrb2.0p50.f222 
!grib2/ncep/GFS/#000/202106211200F222/VREL/-2 K*m**2*kg-1*s-1 POTV! 000738

Since the migration to RHEL8 and a newer version of the ldm, they have been 
coming through with generic default filler info for every product, like

20231015T041432.757539Z pqutil[27287]               pqutil.c:display_watch:1186 
        INFO       82796 20231015041414.643571 CONDUIT 659  
data/nccf/com/gfs/v16.3/gfs.20231015/00/atmos/gfs.t00z.pgrb2.1p00.f162 
!grib2/ncep/GFS/#000/FHRS//LVL! 000659
20231015T041432.757575Z pqutil[27287]               pqutil.c:display_watch:1186 
        INFO       67717 20231015041414.636494 CONDUIT 650  
data/nccf/com/gfs/v16.3/gfs.20231015/00/atmos/gfs.t00z.pgrb2.1p00.f162 
!grib2/ncep/GFS/#000/FHRS//LVL! 000650
20231015T041432.757608Z pqutil[27287]               pqutil.c:display_watch:1186 
        INFO       81162 20231015041414.644156 CONDUIT 660  
data/nccf/com/gfs/v16.3/gfs.20231015/00/atmos/gfs.t00z.pgrb2.1p00.f162 
!grib2/ncep/GFS/#000/FHRS//LVL! 000660

If you rely on the variable or level info after that exclamation point, as I do 
for some of my product sorting, that all broke with this change.

I would guess this change is to resolve that info. William, Mike, would you 
concur?

Pete

Pete Pokrandt - System Engineer IV
UW-Madison Dept of Atmospheric and Oceanic Sciences
608-262-3086  - poker@xxxxxxxxxxxx
________________________________
From: conduit <conduit-bounces@xxxxxxxxxxxxxxxx> on behalf of Herzmann, Daryl E 
[AGRON] <akrherz@xxxxxxxxxxx>
Sent: Wednesday, October 18, 2023 10:25 AM
To: CONDUIT Users List <conduit@xxxxxxxxxxxxxxxx>; William Boll - NOAA Federal 
<william.boll@xxxxxxxx>
Cc: _NWS NCEP NCO Dataflow <nco.dataflow@xxxxxxxx>
Subject: Re: [conduit] CONDUIT Insert Tables Update

Hi William,

Curiosity always kills my cat...  Could you kindly and briefly further describe 
what this change means?  "variables to be defined in the LDM queue" doesn't 
ring any bells for me as to what this change means.  Modifying product names 
can certainly be very impactful to downstream users.

daryl

________________________________________
From: conduit <conduit-bounces@xxxxxxxxxxxxxxxx> on behalf of William Boll - 
NOAA Federal via conduit <conduit@xxxxxxxxxxxxxxxx>
Sent: Wednesday, October 18, 2023 7:21 AM
To: CONDUIT Users List
Cc: _NWS NCEP NCO Dataflow
Subject: [conduit] CONDUIT Insert Tables Update

Attention CONDUIT Users,

Beginning on October 23, 2023 at 1500z, NCO Dataflow will make an update to the 
LDM insert tables on our CONDUIT systems. This will allow for individual 
variables to be defined in the LDM queue for the data products. We are not 
expecting any impacts to downstream users. If however you do encounter issues, 
please report them to NCO Dataflow 
(ncep.pmb.dataflow@xxxxxxxx<mailto:ncep.pmb.dataflow@xxxxxxxx>) as soon as 
possible.


Thank You,
William

--
Dataflow Team Lead (Acting)
NCEP Central Operations
College Park, Maryland 20740
(240)-736-8261

_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


conduit mailing list
conduit@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: 
https://www.unidata.ucar.edu/mailing_lists/
  • 2023 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the conduit archives: