Re: [ldm-users] Processing METAR into a database

• To: "Herbster, Christopher G." <herbstec@xxxxxxxx>
• Subject: Re: [ldm-users] Processing METAR into a database
• From: Mike Zuranski <zuranski.wx@xxxxxxxxx>
• Date: Sat, 14 Dec 2024 14:37:10 -0600

Hi Chris,

Yes this is possible.  What I've done in the past was to implement Postgres
databases and wrote Python scripts to ingest the data.  In my case all I
was really doing was logging the product IDs coming over the IDD, but the
core concept is the same.  I genuinely can't speak for the status of LDM's
ability to send info into a DB directly, it has never come up since I've
been using the LDM.  IMO, I wouldn't worry about the decoders and instead
write something yourself; the operations you're describing seem
"relatively" straight forward.

I have some code that will likely help you out, I'll be back once I dig it
out from my backups.

Best,
-Mike


On Sat, Dec 14, 2024 at 12:09 PM Herbster, Christopher G. <herbstec@xxxxxxxx>
wrote:

> Hi all,
>
> I have a number of desires for having processed METAR data in a database
> like format. We would like to do some forecast verifications for an
> internal effort, as well as possibly create charts of recent/past
> conditions like: https://www.weather.gov/wrh/timeseries?site=KDAB
>
> I suspect that some of you have significant experience doing this, and I
> would really appreciate any sagely advice you can provide.
>
> I was expecting to implement something that would happen as a pqact entry
> when data arrive, though I have not ruled out making hourly files that get
> processed next hour. Data would be either from NOAAPort, IDD or (perhaps?)
> local archives. Does it matter that the Unidata (supported) decoders look
> like they are unchanged for many years? An Internet search showed many
> GitHub options are out there. (Yikes!)
>
> I was also going to create a database entry of the images that flow
> through our system as a way to query/load/delete files that live under our
> web pages. Again, I would certainly benefit from any input folks would like
> to provide.
>
> I am not yet married to a database on the backend, so input there is
> welcome as well. We do have an Apache Hive system that is a candidate for
> this, but I have tinkered with MariaDB in the past. (I am a rookie at
> whatever I will use.)
>
> Is this already being done in EDEX? We had, and will have again, and EDEX
> server running internally, so perhaps I can exploit that.
>
> Thanks in advance for any thoughts, suggestions, and/or tools.
>
> Chris
>
> *Christopher Herbster, Ph.D.*
> *Associate Professor of Meteorology*
> *Applied Aviation Science*
> Daytona Beach Campus
> College of Aviation Rm. 338
>
> 1 Aerospace Boulevard
> Daytona Beach, FL 32114
> 386.226.6444
> *christopher.herbster@xxxxxxxx <christopher.herbster@xxxxxxxx>*
>
> *Embry-Riddle Aeronautical University*
> Florida | Arizona | Worldwide
>
>
> 386.226.6446 Weather Center
> http://wx.erau.edu/
>
> Schedule at:  http://wx.erau.edu/faculty/herbster/Schedules/
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> https://www.unidata.ucar.edu/mailing_lists/
>


-- 
- Mike Zuranski

• Follow-Ups:
  • Re: [ldm-users] Processing METAR into a database
    • From: Mike Zuranski

• References:
  • [ldm-users] Processing METAR into a database
    • From: Herbster, Christopher G.