Greetings everyone,
Last Wednesday a NOAAPort outage sparked a conversation on these lists
regarding the insertion of products into the IDD, specifically the EMWIN
feed in this case, and the prevention of duplicate products being
inserted. We had an internal discussion afterwards last week and we want
to give you all an update. Below is a brief rundown on what happened and
what we are proposing moving forward.
After data stopped flowing over NOAAPort, products from the EMWIN feed were
inserted into the IDD as a stop-gap measure to restore at least some of the
IDD|DDPLUS feed. However, the script used was rather old and didn’t fully
handle the differences between EMWIN and NOAAPort data that exist today.
As a result, some duplicate products were inserted into the IDD as data
began to flow over NOAAPort again.
In the discussions that followed we believe the technical details in
improving that EMWIN insertion script aren’t too involved; trim some
trailing whitespaces which may or may not be present, include key parts of
the WMO header while ignoring others (e.g. the three-digit product sequence
number), and the duplicate product rejection logic in LDM ought to handle
this situation moving forward. We are working on proofing these changes
now.
We are also investigating how we can best make this switchover automatic
here at Unidata. In the event of another NOAAPort outage, we are proposing
to insert supplemental data from sources such as EMWIN from our end to
compensate. With the improved handling mentioned above we should be able
to ensure no duplicate products will be entered into the IDD, and with this
operation running out of the Unidata Program Center we’ll be able to
quickly address any unexpected issues as we deploy this capability. This
may take some time to implement, no ETA at this time but we believe this is
the best path forward.
We’d again like to thank everyone for contributing their input last
Wednesday, it was very beneficial to help drive our internal discussion.
We’ll continue to update as we move forward on this, but please feel free
to reach out with any questions or concerns.
Best,
-Mike
*Mike Zuranski*
Data Engineer II
Unidata Program Center
University Corporation for Atmospheric Research
