[ldm-users] New Real Time Private EMWIN data feed...

Hi everyone.


Back last year Gilbert Sebenste over at Northern Illinois University created
a dedicated EMWIN server for backup should NOAAPORT go down and you need a
backup source of watches/warnings and other test products.   
I've been a user of EMWIN for 7 years now, and come to rely on it for
watches/warnings.  However, just like Noaaport, sometimes it goes down.   So
I started using the combination of two different EMWIN downlinks (goes and
Ku) with the IDS/DDPLUS for 2 years now.  Special thanks to Gerry Creager at
Texas A&M for providing me with the feed of data back then.
It essentially means that in the past two years I've never missed a
watch/warning for my SMS/SNPP alerting system and website
Gilbert and I were discussing EMWIN the other day, as he was gracious enough
to provide me an alternate back feed for IDS/DDPLUS to compliment the Texas
A&M feed, and I started thinking about how he's doing it.  He's currently
using FTP on his dedicated box to pull in EMWIN every 2 minutes from
WALLOPS, the GOES/EMWIN uplink facility.    
I've worked with Santos Rodriguez for years with EMWIN dissemination of
these products, and work heavily with our local WFO with Skywarn
participation.   At any rate, I got thinking, it can't be that hard to get
the data from the Windows EMWIN app and run pqinsert to get it into an
experimental feed on my LDM box.  
My system here (remember, I'm a serious weather hobbyist) is an 8 core box
running VMWare ESX (I own an ISP business here in Tampa).  WxMesg (EMWIN
software http://www.wxmesg.com) runs Windows 2003 on one VM, and CentOS run
a different VM for LDM.  (Along with others for my local personal weather
station, etc)   So using Samba/SMB file sharing, I setup WxMesg to
additionally drop the products from the satellite ingest software into a
directory that is viewable over on the LDM server.
I wrote a quick dirty wrapper as a service that is always watching that
directory, and calls up PQinsert when it finds a new file.  Getting the WMO
headers 100% right looked to be something of a long process though.    But
then Gilbert provided me with a copy of the GNU script that he is using to
do EMWIN via FTP so I can see how he's doing it.  That script was written by
Stonie Cooper from Planetary Data, Inc and I have to give him huge credit
here on this part of the project.   A huge percentage of that script was
devoted to the FTP process.  But since I didn't need that, I took just the
file parsing portion of his script, and married it with my wrapper daemon.
When a file is detected, it spawns off the portion of his script that
processes the files and inserts them into the EXP dataset on my LDM box.
Data inserted from both of my EMWIN downlink dishes comes onto the LDM
looking as the following (from ldmadmin watch) in real time:

Mar 12 17:15:50 pqutil INFO:    16384 20080312171550.070     EXP 999  EMWIN

Mar 12 17:16:35 pqutil INFO:     1293 20080312171635.752     EXP 999  EMWIN

Mar 12 17:16:35 pqutil INFO:     1289 20080312171635.868     EXP 999  EMWIN

Mar 12 17:17:02 pqutil INFO:      205 20080312171702.576     EXP 999  EMWIN

Mar 12 17:17:03 pqutil INFO:    55296 20080312171702.679     EXP 999  EMWIN


Tom Yoksas was the one who suggested that I put "EMWIN " in the front of the
product, making it easier to work with in pqact.     Full details of all the
products offered on the EMWIN stream are available at
http://www.weather.gov/emwin/windat.htm .
I have plenty of spare bandwidth, and am multihomed on the net with multiple
carriers, and also have native IPv6 on my network.  I'm offering this feed
to anyone who would like to pick it up as my way of saying thanks to the
Unidata community for helping me out with the products you guys provide me
that assists with what I do with Skywarn here in West Central Florida.
This data has less than a 1 second delay from when the file is written off
the satellite receiver till it's in the queue, its VERY real time.  Normal
disclaimer applies; don't rely on this data for safety of life information.
I do my best to keep this box up and running, as several hundred rely on it
here in Tampa, but as with any volunteer service, I cannot guarantee 100%
uptime.  Now having said that, last year I only had 30 minutes of downtime
total, proving it's VERY stable.    The feed, only having a 9600BPS stream
of data is -very- low bandwidth usage.   The next generation of EMWIN which
will be coming will support 38400bps of data.    How you use the data is up
to you.
If you're interested in picking up this feed, drop me an e-mail with your
hostname(s), and I'll add you to the allow on my LDM box.   The hostname is
ldm.aprsfl.net and is running LDM 6.6.5.   The name of the feed will be
If you have any questions, please let me know!
Dave Anderson


This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

  • 2008 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: