Re: [ldm-users] NEXRAD ID changes??

Here is what I'm seeing:

> pqlist -v -i 2 -p N0BHTX -O
  282532 20220307/191247.959 NEXRAD3 73337671 SDUS54 KHUN 071912 /pN0BHTX
desi.unidata.ucar.edu_v_noaaport2.wright-weather.com
  286260 20220307/191605.178 NEXRAD3 73347001 SDUS54 KHUN 071915 /pN0BHTX
!nids/ desi.unidata.ucar.edu_v_noaaport2.wright-weather.com
  283571 20220307/191903.226 NEXRAD3 73355592 SDUS54 KHUN 071918 /pN0BHTX
!nids/ desi.unidata.ucar.edu_v_noaaport2.wright-weather.com
  288726 20220307/192204.649 NEXRAD3 73363877 SDUS54 KHUN 071921 /pN0BHTX
!nids/ desi.unidata.ucar.edu_v_noaaport2.wright-weather.com
  285988 20220307/192603.930 NEXRAD3 73374809 SDUS54 KHUN 071925 /pN0BHTX
!nids/ desi.unidata.ucar.edu_v_noaaport2.wright-weather.com

desi.unidata seems to be the source for all the products. But those that
are zlib compressed don't have a "!nids" on the end of the header,

I checked our feed at AWC which is coming straight from a dish in the back
40.

00000000  01 0d 0d 0a 30 30 30 0d  0d 0a 53 44 55 53 35 34
 |....000...SDUS54|
00000010  20 4b 48 55 4e 20 30 37  31 39 31 32 0d 0d 0a 4e  | KHUN
071912...N|
00000020  30 42 48 54 58 0d 0d 0a  *78 da* 85 57 7b 54 13 d7
 |0BHTX...x..W{T..|
00000030  ba 1f 1e 8a da 82 de ab  d6 16 8a da f6 54 c4 9e
 |.............T..|
00000040  4a 55 1e 0a 26 82 68 7d  b4 12 4f 11 53 c0 84 9e
 |JU..&.h}..O.S...|
00000050  52 e4 11 92 a8 90 cc 21  c3 58 d4 56 5b 2d a5 ad
 |R......!.X.V[-..|
00000060  0f 0a 31 80 72 25 2a 92  88 98 44 32 cc 5c 5b 7b
 |..1.r%*...D2.\[{|
00000070  44 94 80 8a 10 c2 30 33  ad 08 01 92 99 29 8f 64
 |D.....03.....).d|

It's coming off the dish that way.

Dan.

On Mon, Mar 7, 2022 at 1:18 PM Mike Zuranski <zuranski.wx@xxxxxxxxx> wrote:

> FWIW it's not just N.B products doing this...
>
> 20220307T191633.718220Z notifyme[70117]
> notifyme.c:notifymeprog_5:212       INFO       18370 20220307191632.821513
> NEXRAD3 73348475  SDUS26 KHNX 071911 /pNBBHNX !nids/
> 20220307T191633.718326Z notifyme[70117]
> notifyme.c:notifymeprog_5:212       INFO        6152 20220307191632.867833
> NEXRAD3 73348476  SDUS58 PAFC 071914 /pN0SAIH
> 20220307T191633.766274Z notifyme[70117]
> notifyme.c:notifymeprog_5:212       INFO       19778 20220307191632.870118
> NEXRAD3 73348477  SDUS28 PAFC 071910 /pN3BAKC !nids/
> 20220307T191633.766402Z notifyme[70117]
> notifyme.c:notifymeprog_5:212       INFO        2178 20220307191632.870344
> NEXRAD3 73348478  SDUS52 KMLB 071913 /pDPAMLB
> 20220307T191633.810195Z notifyme[70117]
> notifyme.c:notifymeprog_5:212       INFO       98306 20220307191632.886132
> NEXRAD3 73348479  SDUS21 KCTP 071913 /pN2BCCX !nids/
> 20220307T191633.810287Z notifyme[70117]
> notifyme.c:notifymeprog_5:212       INFO        1552 20220307191632.888075
> NEXRAD3 73348480  SDUS55 KSLC 071910 /pNCRSLC
> 20220307T191633.854166Z notifyme[70117]
> notifyme.c:notifymeprog_5:212       INFO       67908 20220307191633.029729
> NEXRAD3 73348481  SDUS52 KFFC 071915 /pTV0ATL !nids/
> 20220307T191633.854309Z notifyme[70117]
> notifyme.c:notifymeprog_5:212       INFO        4380 20220307191633.030216
> NEXRAD3 73348482  SDUS52 KMLB 071913 /pNTPMLB
> 20220307T191633.898156Z notifyme[70117]
> notifyme.c:notifymeprog_5:212       INFO         233 20220307191633.030298
> NEXRAD3 73348483  SDUS55 KSLC 071910 /pNVLSLC
> 20220307T191633.898265Z notifyme[70117]
> notifyme.c:notifymeprog_5:212       INFO        4050 20220307191633.032406
> NEXRAD3 73348484  SDUS52 KMLB 071913 /pDSPMLB !nids/
> 20220307T191633.946219Z notifyme[70117]
> notifyme.c:notifymeprog_5:212       INFO        4484 20220307191633.035764
> NEXRAD3 73348485  SDUS32 KMLB 071913 /pN1PMLB
> 20220307T191633.946316Z notifyme[70117]
> notifyme.c:notifymeprog_5:212       INFO       73239 20220307191633.047202
> NEXRAD3 73348486  SDUS55 KSLC 071915 /pNYGICX !nids/
> 20220307T191633.990205Z notifyme[70117]
> notifyme.c:notifymeprog_5:212       INFO       15915 20220307191633.049020
> NEXRAD3 73348487  SDUS26 KHNX 071911 /pNBUHNX !nids/
>
> I have not dug any deeper on those files however.
>
> -Mike
>
> ======================
> Mike Zuranski
> Meteorology Support Analyst
> College of DuPage - Nexlab
> Weather.cod.edu <http://weather.cod.edu/>
> ======================
>
>
> On Mon, Mar 7, 2022 at 1:11 PM Mike Zuranski <zuranski.wx@xxxxxxxxx>
> wrote:
>
>> I'm gonna go out on a limb and say it's a bug...
>>
>> An observation I made that correlates with yours:  Shortly after
>> RAX began issuing N.B products there was a period where they were coming
>> in... badly somehow.  The detail I noticed was the missing "!nids/" at the
>> end of nexrad products while watching the nexrad3 feed.  And like you
>> noticed that was temporary.
>>
>> I see the same thing this morning from a number of sites (LOT grabbed my
>> attention first).  I also see the number of sites doing this are
>> decreasing, so I'm wondering if this is being worked on somewhere.  But any
>> nexrad products that I see in LDM that do not contain "!nids/" at the end
>> do not jive with McIDAS nor Gempak while all others do.
>>
>> Brief example:
>> 20220307T185809.320700Z notifyme[10618]
>> notifyme.c:notifymeprog_5:212       INFO      123297 20220307185809.170361
>> NEXRAD3 73287741  SDUS58 PAFC 071856 /pN0BAKC !nids/
>> 20220307T185811.881051Z notifyme[10618]
>> notifyme.c:notifymeprog_5:212       INFO      314053 20220307185811.690988
>> NEXRAD3 73287963  SDUS51 KRLX 071857 /pN0BRLX
>> 20220307T185813.488004Z notifyme[10618]
>> notifyme.c:notifymeprog_5:212       INFO      191116 20220307185813.222289
>> NEXRAD3 73287985  SDUS53 KGRR 071856 /pN0BGRR !nids/
>>
>> Daryl:  I'm not sure there's a correlation to exengine4.fox.com, last
>> time it was data straight out of SBN doing this.  My Noaaport's been down a
>> while so I can't confirm that right now though.
>>
>> -Mike
>>
>> ======================
>> Mike Zuranski
>> Meteorology Support Analyst
>> College of DuPage - Nexlab
>> Weather.cod.edu <http://weather.cod.edu/>
>> ======================
>>
>>
>> On Mon, Mar 7, 2022 at 1:04 PM Daniel Vietor - NOAA Affiliate via
>> ldm-users <ldm-users@xxxxxxxxxxxxxxxx> wrote:
>>
>>> Is it just me or are people seeing a strange zlib compression of the new
>>> N0B products. When RAX first came out, it was OK and then for a couple of
>>> days I saw the front end of the product was zlib compressed like the old
>>> days of the nexrad products. It went away so I didn't think much about it.
>>>
>>> But then yesterday it came back. Several sites were intermittently zlib
>>> compressing the N0B product. I noted Topeka, Kansas City, Springfield, St
>>> Louis, Chicago, Evansville, Indianapolis and N Indiana were doing it. But
>>> it was intermittent. One scan would be zlib compressed and the next
>>> wouldn't be. It was like something in the product, like size, was
>>> triggering the zlib compression.
>>>
>>> I took a look at some of the data. Only the first 4000 bytes are zlib
>>> compressed. The radial payload is still bzip2 compressed. Since only the
>>> header, which is about 100 bytes, is uncompressed, the zlib compression is
>>> compressing over 3800 bytes of the radial payload.  This seems strange. Why
>>> compress data that are already compressed?
>>>
>>> After last night, it seems like this is a feature of the new N0B radar
>>> data and not a bug. So I resurrected the old ucnid.c program I wrote 24
>>> years ago to remove the zlib compression on the N0B products.
>>>
>>> So is this a bug or a feature?
>>>
>>> Dan.
>>>
>>> On Fri, Mar 4, 2022 at 10:36 AM Gilbert Sebenste <
>>> gilbert@xxxxxxxxxxxxxxxx> wrote:
>>>
>>>> Nothing beats a good Daryl LDM-users rant in the morning. He never lets
>>>> me down!
>>>>
>>>> Imagine my utter surprise when I did what Daryl said and I found
>>>> products I didn't know were being sent over NOAAport. Then, I got curious
>>>> and instead of using the "NEXRAD" feedtype, I used "ANY". That's when I was
>>>> shocked to find several products being sent under the HRS feed that were
>>>> completely undocumented by the NWS, and were needed! I alerted Steve
>>>> Emmerson, who put out a fix for it. 6.13.16 has it all sorted out now.
>>>>
>>>> I'll point out that this thinking goes for any product. If there's a
>>>> product you think should be there that isn't coming in, and you know you're
>>>> getting the entire feed, use a request of "ANY" on that header. Rarely,
>>>> it's an LDM bug, but like the example above, it can happen where a product
>>>> shows up on an unexpected feedtype.
>>>>
>>>> Gilbert
>>>>
>>>> > On Mar 4, 2022, at 10:17 AM, Herzmann, Daryl E [AGRON] <
>>>> akrherz@xxxxxxxxxxx> wrote:
>>>> >
>>>> > Good morning,
>>>> >
>>>> > I'd like to chime on the topic of "best practice for pqact entries".
>>>> For the WMO header found in products, there's a general form for the
>>>> entries found:
>>>> >
>>>> > TTAAII CCCC DDHHMI
>>>> >
>>>> > When there's a string found in the line below the WMO header that is
>>>> six or so characters, this "PIL" is appended to LDM product name in the
>>>> form like so:
>>>> >
>>>> > TTAAII CCCC DDHHMI /pPILXXX
>>>> >
>>>> > Summary: I *do not* trust the TTAAII value and only sparingly use it
>>>> within a pqact entry, but instead fully trust the /pPILXXX section.
>>>> >
>>>> > In December 2015, I meet with the NWS Data Management folks
>>>> expressing interest in quality controlling the CCCC portion of the WMO
>>>> header.  Can you imagine that it is sometimes wrong?  Well, it has been 6+
>>>> years now, hundreds of emails, and I'm still not done getting them to
>>>> correct issues with it.  As an example of how painful the rabbit hole is,
>>>> NWS Cheyenne KCYS was issuing TAFs for 3 sites using KOAX as the CCCC.
>>>> This took 8 months to fix and I am still unsure if it is fully fixed.
>>>> >
>>>> > I have not even attempted automated quality control of the TTAAII,
>>>> nor bugging NWS to fix it.  Another story.  I actually did attempt to get
>>>> NWS to fix the TTAAII in the case of a product that wasn't using 6
>>>> characters for the TTAAII, can you imagine that it is sometimes only 5?
>>>> That request was rejected.  Anyway,  this value used to be more rigorously
>>>> set, but is now more arbitrary than ever.  So in the original example 
>>>> below.
>>>> >
>>>> > NEXRAD ^SDUS[2357]. ....
>>>> ([0-3][0-9])([0-2][0-9])([0-6][0-9]).*/p(...)(...)
>>>> >
>>>> > I would only trust the SDUS and drop any checks on the other TTAAII
>>>> fields.  Instead, put your limiters into the /p section.
>>>> >
>>>> > NEXRAD ^SDUS.. ....
>>>> ([0-3][0-9])([0-2][0-9])([0-6][0-9]).*/p(N0Q|N0B|N0G|Nwhatever)(...)
>>>> >
>>>> > Oh, there are examples in the TTAAII, where the AA is wrong, but I
>>>> have ranted/whined enough already.
>>>> >
>>>> > daryl
>>>> >
>>>> > ________________________________________
>>>> > From: ldm-users <ldm-users-bounces@xxxxxxxxxxxxxxxx> on behalf of
>>>> Gilbert Sebenste <gilbert@xxxxxxxxxxxxxxxx>
>>>> > Sent: Friday, March 4, 2022 9:54 AM
>>>> > To: Daniel Vietor - NOAA Affiliate
>>>> > Cc: LDM-Users; Mike Voss
>>>> > Subject: Re: [ldm-users] NEXRAD ID changes??
>>>> >
>>>> > If you have LDM version 6.13.16, this shouldn't be happening. A bug
>>>> fix was applied to ensure that everything NEXRAD3 is supposed to be there.
>>>> >
>>>> > Gilbert
>>>> >
>>>> > On Mar 4, 2022, at 9:15 AM, Daniel Vietor - NOAA Affiliate via
>>>> ldm-users <ldm-users@xxxxxxxxxxxxxxxx> wrote:
>>>> >
>>>> > 
>>>> > I see a lot of them on the HDS feedtype. So I request both NEXRAD and
>>>> HDS.
>>>> >
>>>> > Dan.
>>>> >
>>>> > On Fri, Mar 4, 2022 at 9:04 AM Mike Voss <mike.voss@xxxxxxxx<mailto:
>>>> mike.voss@xxxxxxxx>> wrote:
>>>> > Hi All,
>>>> > For me at least, some of my NEXRAD products stopped filing correctly
>>>> yesterday at 18Z using this generic pattern match:
>>>> >
>>>> > NEXRAD ^SDUS[2357]. ....
>>>> ([0-3][0-9])([0-2][0-9])([0-6][0-9]).*/p(...)(...)
>>>> >       FILE    -overwrite      -close
>>>> nexrad/NIDS/\5/\4/\4_(\1:yyyy)(\1:mm)\1_\2\3
>>>> >
>>>> > as a test, when I changed to this:
>>>> > NEXRAD  ^SDUS66 ....
>>>> >
>>>> > I started getting my "N0Q" products for MUX station.
>>>> >
>>>> > Also, I stopped receiving the FNEXRAD NEXRCOMP products.  Is it
>>>> possible these things are related such that the NEXRCOMP depends on the
>>>> NIDS product ID's and somehow a recent change (yesterday) cause this to
>>>> break?
>>>> >
>>>> > thanks for any insights,
>>>> > -MIke
>>>> > _______________________________________________
>>>> > NOTE: All exchanges posted to Unidata maintained email lists are
>>>> > recorded in the Unidata inquiry tracking system and made publicly
>>>> > available through the web.  Users who post to any of the lists we
>>>> > maintain are reminded to remove any personal information that they
>>>> > do not want to be made public.
>>>> >
>>>> >
>>>> > ldm-users mailing list
>>>> > ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
>>>> > For list information or to unsubscribe,  visit:
>>>> https://www.unidata.ucar.edu/mailing_lists/
>>>> >
>>>> >
>>>> > --
>>>> > Dan Vietor
>>>> > Senior Research Meteorologist
>>>> > CIRA, Colorado State Univ
>>>> > Aviation Weather Center
>>>> > Kansas City, MO
>>>> > 816.584.7211
>>>> >
>>>> > _______________________________________________
>>>> > NOTE: All exchanges posted to Unidata maintained email lists are
>>>> > recorded in the Unidata inquiry tracking system and made publicly
>>>> > available through the web.  Users who post to any of the lists we
>>>> > maintain are reminded to remove any personal information that they
>>>> > do not want to be made public.
>>>> >
>>>> >
>>>> > ldm-users mailing list
>>>> > ldm-users@xxxxxxxxxxxxxxxx
>>>> > For list information or to unsubscribe,  visit:
>>>> https://www.unidata.ucar.edu/mailing_lists/
>>>>
>>>
>>>
>>> --
>>> *Dan Vietor*
>>> *Senior Research Meteorologist*
>>> CIRA, Colorado State Univ
>>> Aviation Weather Center
>>> Kansas City, MO
>>> 816.584.7211
>>>
>>> _______________________________________________
>>> NOTE: All exchanges posted to Unidata maintained email lists are
>>> recorded in the Unidata inquiry tracking system and made publicly
>>> available through the web.  Users who post to any of the lists we
>>> maintain are reminded to remove any personal information that they
>>> do not want to be made public.
>>>
>>>
>>> ldm-users mailing list
>>> ldm-users@xxxxxxxxxxxxxxxx
>>> For list information or to unsubscribe,  visit:
>>> https://www.unidata.ucar.edu/mailing_lists/
>>>
>>

-- 
*Dan Vietor*
*Senior Research Meteorologist*
CIRA, Colorado State Univ
Aviation Weather Center
Kansas City, MO
816.584.7211
  • 2022 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: