Re: [## 1441 ##] SNOTEL site TUGA2 temperature data issue Jan 03

Hello Leon and All,

Following up from the Synoptic side here.  Just to make sure we understand the current procedure:

1) MADIS generates the SNOTEL SHEF files at https://madis-data.ncep.noaa.gov/madisPublic1/data/snotel/hourly/

2) Synoptic Data/MesoWest then ingest the SNOTEL data from those files.

3) MADIS does not use the SHEF files directly to ingest the SNOTEL data, but instead relies upon the public IOOS-CSV format generated by Synoptic Data PBC (mesowest_csvplus.dat.gz) to ingest the SNOTEL data instead of the SHEF files?

If we have that correct, then I believe the issues do reside originally with the issue of the SHEF files not always having all of the data during the overnight hours.  We provided an example back on December 12 illustrating what we had found with the SHEF files during the time period of UTC midnight to midnight local time (which would match the 00-09Z timeframe in the original request).  Let me know if resending that document might be helpful.

The mesowest_csvplus.dat.gz file we transmit has a temporal limit on how far back it queries for data, in order to keep the volume/file size down in the 5-minute sending to MADIS, so backfilled SNOTEL temperature data would not be in that file being transmitted if outages in the SNOTEL SHEF files are lasting several hours each night.

Happy to answer any other questions as well.

Regards

-Alex

---- on Thu, 04 Jan 2024 12:36:44 -0500 "Leon Benjamin - NOAA Affiliate"<address@hidden> wrote ----

Dale,

Sorry for you getting involved in this.

 

This is an Alaskan SNOTEL site.  It needs SYNOPTIC support not the SYNOPTIC subproviders support.

-- Leon

On Thu, Jan 4, 2024 at 10:32 AM Dale Kirmer [KDOT] <address@hidden> wrote:

Hi,

I am assuming this site description is looking for RWIS data from Kansas?

We had something interrupt our RWIS data from January 4, 2024 at 12:00 am till 9:35am.

We should be back up and running.

 

Best Regards,

Dale Kirmer

 

Dale Kirmer, P.E. | Staff Engineer, Bureau of Maintenance

O: 785.296-6355

address@hidden

 

 

From: Leon Benjamin - NOAA Affiliate <address@hidden>
Sent: Thursday, January 04, 2024 11:27 AM
To: MITCH <address@hidden>
Cc: Support User <address@hidden>; _NWS NCEP MADIS-data <address@hidden>
Subject: Re: SNOTEL site TUGA2 temperature data issue Jan 03

 

EXTERNAL: This email originated from outside of the organization. Do not click any links or open any attachments unless you trust the sender and know the content is safe.

Mitch,

This is a site that SYNOPTIC pulls the data from SHEF files at: https://madis-data.ncep.noaa.gov/madisPublic1/data/snotel/hourly/

 

SYNOPTIC processes those files and includes data from them in the IOOS file that it ldms to MADIS.

 

Who understands that data feed and process? 

 

-- Leon

 

On Thu, Jan 4, 2024 at 10:22 AM MITCH <address@hidden> wrote:

Leon,

       Unsure, and not my site.  However,  I did notice yesterday that my site in Calcium NY was not reporting out.  I assumed it was a firewall issue on my end. 

Get BlueMail for Android

On Jan 4, 2024, at 11:37 AM, Leon Benjamin - NOAA Affiliate <address@hidden> wrote:

All,

 

Can you look through your records and see if you sent MADIS TUGA2 temperature data for each hour from Jan 03 00z to 09z.

 

We had a question from the field and I'm trying to track down where we had a problem.

 

Unfortunately, the SNOTEL SHEF file doesn't go back enough for me to check, but I do see data in the logs for the SHEF file. 

 

Thanks,

-- Leon

 

Leon Benjamin

MADIS Tier 3 support

MADIS help email: address@hidden

MADIS Web page: https://madis.ncep.noaa.gov/

 

 

--

Leon Benjamin

MADIS Tier 3 support

MADIS help email: address@hidden

MADIS Web page: https://madis.ncep.noaa.gov/

 

--

Leon Benjamin

MADIS Tier 3 support

MADIS help email: address@hidden

MADIS Web page: https://madis.ncep.noaa.gov/