I'm gonna go out on a limb and say it's a bug...
An observation I made that correlates with yours: Shortly after RAX began
issuing N.B products there was a period where they were coming in... badly
somehow. The detail I noticed was the missing "!nids/" at the end of
nexrad products while watching the nexrad3 feed. And like you noticed that
was temporary.
I see the same thing this morning from a number of sites (LOT grabbed my
attention first). I also see the number of sites doing this are
decreasing, so I'm wondering if this is being worked on somewhere. But any
nexrad products that I see in LDM that do not contain "!nids/" at the end
do not jive with McIDAS nor Gempak while all others do.
Brief example:
20220307T185809.320700Z notifyme[10618]
notifyme.c:notifymeprog_5:212 INFO 123297 20220307185809.170361
NEXRAD3 73287741 SDUS58 PAFC 071856 /pN0BAKC !nids/
20220307T185811.881051Z notifyme[10618]
notifyme.c:notifymeprog_5:212 INFO 314053 20220307185811.690988
NEXRAD3 73287963 SDUS51 KRLX 071857 /pN0BRLX
20220307T185813.488004Z notifyme[10618]
notifyme.c:notifymeprog_5:212 INFO 191116 20220307185813.222289
NEXRAD3 73287985 SDUS53 KGRR 071856 /pN0BGRR !nids/
Daryl: I'm not sure there's a correlation to exengine4.fox.com, last time
it was data straight out of SBN doing this. My Noaaport's been down a
while so I can't confirm that right now though.
-Mike
======================
Mike Zuranski
Meteorology Support Analyst
College of DuPage - Nexlab
Weather.cod.edu <http://weather.cod.edu/>
======================
On Mon, Mar 7, 2022 at 1:04 PM Daniel Vietor - NOAA Affiliate via ldm-users
<ldm-users@xxxxxxxxxxxxxxxx> wrote:
> Is it just me or are people seeing a strange zlib compression of the new
> N0B products. When RAX first came out, it was OK and then for a couple of
> days I saw the front end of the product was zlib compressed like the old
> days of the nexrad products. It went away so I didn't think much about it .
>
> But then yesterday it came back. Several sites were intermittently zlib
> compressing the N0B product. I noted Topeka, Kansas City, Springfield, St
> Louis, Chicago, Evansville, Indianapolis and N Indiana were doing it. But
> it was intermittent. One scan would be zlib compressed and the next
> wouldn't be. It was like something in the product, like size, was
> triggering the zlib compression.
>
> I took a look at some of the data. Only the first 4000 bytes are zlib
> compressed. The radial payload is still bzip2 compressed. Since only the
> header, which is about 100 bytes, is uncompressed, the zlib compression is
> compressing over 3800 bytes of the radial payload. This seems strange. Why
> compress data that are already compressed?
>
> After last night, it seems like this is a feature of the new N0B radar
> data and not a bug. So I resurrected the old ucnid.c program I wrote 24
> years ago to remove the zlib compression on the N0B products.
>
> So is this a bug or a feature?
>
> Dan.
>
> On Fri, Mar 4, 2022 at 10:36 AM Gilbert Sebenste <gilbert@xxxxxxxxxxxxxxxx>
> wrote:
>
>> Nothing beats a good Daryl LDM-users rant in the morning. He never lets
>> me down!
>>
>> Imagine my utter surprise when I did what Daryl said and I found products
>> I didn't know were being sent over NOAAport. Then, I got curious and
>> instead of using the "NEXRAD" feedtype, I used "ANY". That's when I was
>> shocked to find several products being sent under the HRS feed that were
>> completely undocumented by the NWS, and were needed! I alerted Steve
>> Emmerson, who put out a fix for it. 6.13.16 has it all sorted out now.
>>
>> I'll point out that this thinking goes for any product. If there's a
>> product you think should be there that isn't coming in, and you know you're
>> getting the entire feed, use a request of "ANY" on that header. Rarely,
>> it's an LDM bug, but like the example above, it can happen where a product
>> shows up on an unexpected feedtype.
>>
>> Gilbert
>>
>> > On Mar 4, 2022, at 10:17 AM, Herzmann, Daryl E [AGRON] <
>> akrherz@xxxxxxxxxxx> wrote:
>> >
>> > Good morning,
>> >
>> > I'd like to chime on the topic of "best practice for pqact entries".
>> For the WMO header found in products, there's a general form for the
>> entries found:
>> >
>> > TTAAII CCCC DDHHMI
>> >
>> > When there's a string found in the line below the WMO header that is
>> six or so characters, this "PIL" is appended to LDM product name in the
>> form like so:
>> >
>> > TTAAII CCCC DDHHMI /pPILXXX
>> >
>> > Summary: I *do not* trust the TTAAII value and only sparingly use it
>> within a pqact entry, but instead fully trust the /pPILXXX section.
>> >
>> > In December 2015, I meet with the NWS Data Management folks expressing
>> interest in quality controlling the CCCC portion of the WMO header. Can
>> you imagine that it is sometimes wrong? Well, it has been 6+ years now,
>> hundreds of emails, and I'm still not done getting them to correct issues
>> with it. As an example of how painful the rabbit hole is, NWS Cheyenne
>> KCYS was issuing TAFs for 3 sites using KOAX as the CCCC. This took 8
>> months to fix and I am still unsure if it is fully fixed.
>> >
>> > I have not even attempted automated quality control of the TTAAII, nor
>> bugging NWS to fix it. Another story. I actually did attempt to get NWS
>> to fix the TTAAII in the case of a product that wasn't using 6 characters
>> for the TTAAII, can you imagine that it is sometimes only 5? That request
>> was rejected. Anyway, this value used to be more rigorously set, but is
>> now more arbitrary than ever. So in the original example below.
>> >
>> > NEXRAD ^SDUS[2357]. ....
>> ([0-3][0-9])([0-2][0-9])([0-6][0-9]).*/p(...)(...)
>> >
>> > I would only trust the SDUS and drop any checks on the other TTAAII
>> fields. Instead, put your limiters into the /p section.
>> >
>> > NEXRAD ^SDUS.. ....
>> ([0-3][0-9])([0-2][0-9])([0-6][0-9]).*/p(N0Q|N0B|N0G|Nwhatever)(...)
>> >
>> > Oh, there are examples in the TTAAII, where the AA is wrong, but I have
>> ranted/whined enough already.
>> >
>> > daryl
>> >
>> > ________________________________________
>> > From: ldm-users <ldm-users-bounces@xxxxxxxxxxxxxxxx> on behalf of
>> Gilbert Sebenste <gilbert@xxxxxxxxxxxxxxxx>
>> > Sent: Friday, March 4, 2022 9:54 AM
>> > To: Daniel Vietor - NOAA Affiliate
>> > Cc: LDM-Users; Mike Voss
>> > Subject: Re: [ldm-users] NEXRAD ID changes??
>> >
>> > If you have LDM version 6.13.16, this shouldn't be happening. A bug fix
>> was applied to ensure that everything NEXRAD3 is supposed to be there.
>> >
>> > Gilbert
>> >
>> > On Mar 4, 2022, at 9:15 AM, Daniel Vietor - NOAA Affiliate via
>> ldm-users <ldm-users@xxxxxxxxxxxxxxxx> wrote:
>> >
>> >
>> > I see a lot of them on the HDS feedtype. So I request both NEXRAD and
>> HDS.
>> >
>> > Dan.
>> >
>> > On Fri, Mar 4, 2022 at 9:04 AM Mike Voss <mike.voss@xxxxxxxx<mailto:
>> mike.voss@xxxxxxxx>> wrote:
>> > Hi All,
>> > For me at least, some of my NEXRAD products stopped filing correctly
>> yesterday at 18Z using this generic pattern match:
>> >
>> > NEXRAD ^SDUS[2357]. ....
>> ([0-3][0-9])([0-2][0-9])([0-6][0-9]).*/p(...)(...)
>> > FILE -overwrite -close
>> nexrad/NIDS/\5/\4/\4_(\1:yyyy)(\1:mm)\1_\2\3
>> >
>> > as a test, when I changed to this:
>> > NEXRAD ^SDUS66 ....
>> >
>> > I started getting my "N0Q" products for MUX station.
>> >
>> > Also, I stopped receiving the FNEXRAD NEXRCOMP products. Is it
>> possible these things are related such that the NEXRCOMP depends on the
>> NIDS product ID's and somehow a recent change (yesterday) cause this to
>> break?
>> >
>> > thanks for any insights,
>> > -MIke
>> > _______________________________________________
>> > NOTE: All exchanges posted to Unidata maintained email lists are
>> > recorded in the Unidata inquiry tracking system and made publicly
>> > available through the web. Users who post to any of the lists we
>> > maintain are reminded to remove any personal information that they
>> > do not want to be made public.
>> >
>> >
>> > ldm-users mailing list
>> > ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
>> > For list information or to unsubscribe, visit:
>> https://www.unidata.ucar.edu/mailing_lists/
>> >
>> >
>> > --
>> > Dan Vietor
>> > Senior Research Meteorologist
>> > CIRA, Colorado State Univ
>> > Aviation Weather Center
>> > Kansas City, MO
>> > 816.584.7211
>> >
>> > _______________________________________________
>> > NOTE: All exchanges posted to Unidata maintained email lists are
>> > recorded in the Unidata inquiry tracking system and made publicly
>> > available through the web. Users who post to any of the lists we
>> > maintain are reminded to remove any personal information that they
>> > do not want to be made public.
>> >
>> >
>> > ldm-users mailing list
>> > ldm-users@xxxxxxxxxxxxxxxx
>> > For list information or to unsubscribe, visit:
>> https://www.unidata.ucar.edu/mailing_lists/
>>
>
>
> --
> *Dan Vietor*
> *Senior Research Meteorologist*
> CIRA, Colorado State Univ
> Aviation Weather Center
> Kansas City, MO
> 816.584.7211
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/
>
