We have a socket feed of products from the gateway to act as a backup if
NOAAPort goes down (eclipse season) or to add in products available on the
gateway that aren't on NOAAPort (fax charts, specialized GRIB data,
international data). The products are formatted just different enough that
they all end up as duplicates vs the NOAAPort feed from LDM. Because we
don't control the code that ingests the data from the socket feed into the
LDM, we changed the productMaker code in the LDM. There are 3 changes:
1) zero out the sequence number "000" (not sure this is needed or not but
all products from socket have sequence number of "000")
2) remove a space after the sequence number
3) only append a CR-CR-LF at the end of the product if there isn't one in
the data (as in GRIB or BUFR)
Without this, the GRIB products were 1 byte longer than what we got from
the socket feed and the text products were 4 bytes longer. So most of the
products from the socket feed ended up going in twice.
At some point, we'll be switching over to an LDM feed of data from the
gateway. I don't know if that will change this situation or not.
Dan.
On Fri, Mar 4, 2022 at 1:04 PM Herzmann, Daryl E [AGRON] <
akrherz@xxxxxxxxxxx> wrote:
> Greetings,
>
> The issue has been found with a weatherwire ingest into the IDD, hopefully
> it gets removed before tomorrow :) In the interim, look for the \017
> character and drop those products / dedup them against the noaaport
> content. That doesn't catch them all, but most of it.
>
> daryl
>
> ________________________________________
> From: ldm-users <ldm-users-bounces@xxxxxxxxxxxxxxxx> on behalf of Gilbert
> Sebenste <gilbert@xxxxxxxxxxxxxxxx>
> Sent: Friday, March 4, 2022 12:58 PM
> To: Wojtowicz, David P Jr
> Cc: ldm-users@xxxxxxxxxxxxxxxx
> Subject: Re: [ldm-users] Receipt of "duplicated" products over the IDD
> with some debugging string attached to the end of the product
>
> True, but they may have been doing additional testing related to this.
> When they did the testing last week, they had some significant issues…
>
> Gilbert
>
> On Mar 4, 2022, at 12:42 PM, Wojtowicz, David P Jr <davidw@xxxxxxxxxxxx>
> wrote:
>
>
> Changing the underlying network shouldn't cause duplicated products. (For
> example, the network equipment knows nothing about the data headers and
> would have no possible way of producing the extra strings)
>
> From: ldm-users <ldm-users-bounces@xxxxxxxxxxxxxxxx> On Behalf Of Gilbert
> Sebenste
> Sent: Friday, March 4, 2022 11:59 AM
> To: Mike Zuranski <zuranski.wx@xxxxxxxxx>
> Cc: ldm-users@xxxxxxxxxxxxxxxx
> Subject: Re: [ldm-users] Receipt of "duplicated" products over the IDD
> with some debugging string attached to the end of the product
>
> NCEP was testing their network routers and related activities this morning.
>
> NOUS42 KWNO 041141
> ADASDM
>
> SENIOR DUTY METEOROLOGIST NWS ALERT ADMINISTRATIVE MESSAGE
> NWS NCEP CENTRAL OPERATIONS COLLEGE PARK MD
> 1141Z FRI MAR 04 2022
>
> ..SCHEDULED NCWCP NETWORK MAINTENANCE TODAY
>
> WHAT: NCO WILL BE PERFORMING A FAILOVER TEST ON THE ROUTERS
> LOCATED AT COLLEGE PARK/NCWCP.
>
> WHEN: FRIDAY, MARCH 4, 2022 BETWEEN 1200Z AND 1400Z.
>
> IMPACTS: WFOS/RFCS/RHQS/CWSUS/NWR/CENTERS/WCOSS-I&II MAY
> EXPERIENCE TWO BRIEF (3-5) MINUTES OUTAGE AT THE START AND END OF
> THE MAINTENANCE WINDOW WHILE TRAFFIC GETS AUTO REROUTED BACK AND
> FORTH BETWEEN PRIMARY AND SECONDARY PATHS DURING THIS FAILOVER
> TEST.
>
> PUBLIC ACCESS TO NWS WEBSITES INCLUDING IDP AND NIDS WILL NOT BE
> IMPACTED.
>
> LIDDICK/SDM/NCO/NCEP
>
>
> Gilbert
>
> On Fri, Mar 4, 2022 at 11:18 AM Mike Zuranski <zuranski.wx@xxxxxxxxx
> <mailto:zuranski.wx@xxxxxxxxx>> wrote:
> I can confirm that double product looks the same for me, complete with
> LOX-846094 at the end of the second. I'm not sure which site was feeding
> that at the time though.
>
> -Mike
>
> ======================
> Mike Zuranski
> Meteorology Support Analyst
> College of DuPage - Nexlab
> Weather.cod.edu<
> https://urldefense.com/v3/__http:/weather.cod.edu/__;!!DZ3fjg!o1bK4FJaMKZ5_t1qEkELlilHYevjZtBWwmCexGZCYVOlLAXVgqyWlw_XQSV1LFAZhg$
> >
> ======================
>
>
> On Fri, Mar 4, 2022 at 11:05 AM Herzmann, Daryl E [AGRON] <
> akrherz@xxxxxxxxxxx<mailto:akrherz@xxxxxxxxxxx>> wrote:
> Greetings,
>
> Over the past couple of hours, I have noticed a problem with product text
> over IDD including what appears to be a debug tag at the end of the
> product. For example:
>
> WWUS76 KLOX 041650
>
> Came at 2022-03-04T16:50:36Z with 2026 bytes and again at
> 2022-03-04T16:50:38Z at 2038 bytes. The second product had the following
> at the end of the text:
>
> &&
>
> $$
> LOX-846094
>
> That "LOX-846094" appears to be some debugging string with the
> transmission of the product? Looking at the my IDD logging, I see:
>
> wxengine4.fox.com<
> https://urldefense.com/v3/__http:/wxengine4.fox.com__;!!DZ3fjg!o1bK4FJaMKZ5_t1qEkELlilHYevjZtBWwmCexGZCYVOlLAXVgqyWlw_XQSVerSLGBg$
> >
>
> as the ingest source of this product. Anybody know anything about this?
>
> daryl
>
>
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/<
> https://urldefense.com/v3/__https:/www.unidata.ucar.edu/mailing_lists/__;!!DZ3fjg!o1bK4FJaMKZ5_t1qEkELlilHYevjZtBWwmCexGZCYVOlLAXVgqyWlw_XQSXzKCu6Bw$
> >
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/<
> https://urldefense.com/v3/__https:/www.unidata.ucar.edu/mailing_lists/__;!!DZ3fjg!o1bK4FJaMKZ5_t1qEkELlilHYevjZtBWwmCexGZCYVOlLAXVgqyWlw_XQSXzKCu6Bw$
> >
>
>
> --
> ----
>
> Gilbert Sebenste
> Consulting Meteorologist
> AllisonHouse, LLC
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/
>
--
*Dan Vietor*
*Senior Research Meteorologist*
CIRA, Colorado State Univ
Aviation Weather Center
Kansas City, MO
816.584.7211
