[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [noaaport] [External] The SBN Outage, Supplemental IDD Data and Duplicate Products

Mike, et al,

 

Thanks for you and the UNIDATA team’s work on this. When NOAAport goes down, *usually* only EMWIN stays up for external users since it uses a socket connection that bypasses the problem we saw last week.

For those who don’t know, the text portion of the EMWIN feed carries METARS, text forecasts, and all watches, advisories and warnings, as well as climate summaries and what you normally see on IDS|DDPLUS (though not everything). It’s much better than nothing. As an aside, NOAA Weather Wire Service (NWWS) is derived directly from NOAAport, so if it goes down, so does NWWS.

 

Gilbert

 

Gilbert Sebenste

Meteorology Support Analyst

 

From: noaaport <address@hidden> On Behalf Of Mike Zuranski
Sent: Monday, April 3, 2023 11:40 AM
To: LDM <address@hidden>; NOAAPORT <address@hidden>
Subject: [External] [noaaport] The SBN Outage, Supplemental IDD Data and Duplicate Products

 

CAUTION: This email originated from outside of COD’s system. Do not click links, open attachments, or respond with sensitive information unless you recognize the sender and know the content is safe.

 

Greetings everyone,
 
Last Wednesday a NOAAPort outage sparked a conversation on these lists regarding the insertion of products into the IDD, specifically the EMWIN feed in this case, and the prevention of duplicate products being inserted.  We had an internal discussion afterwards last week and we want to give you all an update.  Below is a brief rundown on what happened and what we are proposing moving forward.
 
After data stopped flowing over NOAAPort, products from the EMWIN feed were inserted into the IDD as a stop-gap measure to restore at least some of the IDD|DDPLUS feed.  However, the script used was rather old and didn’t fully handle the differences between EMWIN and NOAAPort data that exist today.  As a result, some duplicate products were inserted into the IDD as data began to flow over NOAAPort again.
 
In the discussions that followed we believe the technical details in improving that EMWIN insertion script aren’t too involved; trim some trailing whitespaces which may or may not be present, include key parts of the WMO header while ignoring others (e.g. the three-digit product sequence number), and the duplicate product rejection logic in LDM ought to handle this situation moving forward.  We are working on proofing these changes now.
 
We are also investigating how we can best make this switchover automatic here at Unidata.  In the event of another NOAAPort outage, we are proposing to insert supplemental data from sources such as EMWIN from our end to compensate.  With the improved handling mentioned above we should be able to ensure no duplicate products will be entered into the IDD, and with this operation running out of the Unidata Program Center we’ll be able to quickly address any unexpected issues as we deploy this capability.  This may take some time to implement, no ETA at this time but we believe this is the best path forward.
 
We’d again like to thank everyone for contributing their input last Wednesday, it was very beneficial to help drive our internal discussion.  We’ll continue to update as we move forward on this, but please feel free to reach out with any questions or concerns.
 
Best,
-Mike

 

 

Mike Zuranski

Data Engineer II

Unidata Program Center

University Corporation for Atmospheric Research

_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


noaaport mailing list
address@hidden
For list information or to unsubscribe, visit: 
https://www.unidata.ucar.edu/mailing_lists/