Mike, et al,
Thanks for you and the UNIDATA team’s work on this. When NOAAport goes down,
*usually* only EMWIN stays up for external users since it uses a socket
connection that bypasses the problem we saw last week.
For those who don’t know, the text portion of the EMWIN feed carries METARS,
text forecasts, and all watches, advisories and warnings, as well as climate
summaries and what you normally see on IDS|DDPLUS (though not everything). It’s
much better than nothing. As an aside, NOAA Weather Wire Service (NWWS) is
derived directly from NOAAport, so if it goes down, so does NWWS.
Gilbert
Gilbert Sebenste
Meteorology Support Analyst
[cid:image001.png@01D96627.3E2DD8D0]
From: noaaport <noaaport-bounces@xxxxxxxxxxxxxxxx> On Behalf Of Mike Zuranski
Sent: Monday, April 3, 2023 11:40 AM
To: LDM <ldm-users@xxxxxxxxxxxxxxxx>; NOAAPORT <noaaport@xxxxxxxxxxxxxxxx>
Subject: [External] [noaaport] The SBN Outage, Supplemental IDD Data and
Duplicate Products
CAUTION: This email originated from outside of COD’s system. Do not click
links, open attachments, or respond with sensitive information unless you
recognize the sender and know the content is safe.
Greetings everyone,
Last Wednesday a NOAAPort outage sparked a conversation on these lists
regarding the insertion of products into the IDD, specifically the EMWIN feed
in this case, and the prevention of duplicate products being inserted. We had
an internal discussion afterwards last week and we want to give you all an
update. Below is a brief rundown on what happened and what we are proposing
moving forward.
After data stopped flowing over NOAAPort, products from the EMWIN feed were
inserted into the IDD as a stop-gap measure to restore at least some of the
IDD|DDPLUS feed. However, the script used was rather old and didn’t fully
handle the differences between EMWIN and NOAAPort data that exist today. As a
result, some duplicate products were inserted into the IDD as data began to
flow over NOAAPort again.
In the discussions that followed we believe the technical details in improving
that EMWIN insertion script aren’t too involved; trim some trailing whitespaces
which may or may not be present, include key parts of the WMO header while
ignoring others (e.g. the three-digit product sequence number), and the
duplicate product rejection logic in LDM ought to handle this situation moving
forward. We are working on proofing these changes now.
We are also investigating how we can best make this switchover automatic here
at Unidata. In the event of another NOAAPort outage, we are proposing to
insert supplemental data from sources such as EMWIN from our end to compensate.
With the improved handling mentioned above we should be able to ensure no
duplicate products will be entered into the IDD, and with this operation
running out of the Unidata Program Center we’ll be able to quickly address any
unexpected issues as we deploy this capability. This may take some time to
implement, no ETA at this time but we believe this is the best path forward.
We’d again like to thank everyone for contributing their input last Wednesday,
it was very beneficial to help drive our internal discussion. We’ll continue
to update as we move forward on this, but please feel free to reach out with
any questions or concerns.
Best,
-Mike
Mike Zuranski
Data Engineer II
Unidata Program Center
University Corporation for Atmospheric Research
