Hi all,
I recently grabbed a copy of the RSL library from Github to see if it had been
updated to read files coming from Build 19 and, indeed, it had. So I thought it
wouldn’t be too difficult to update extlib/rsl in gempak to use the updates.
After a couple of trips down some rabbit holes in my “spare time” over the
holidays , I realized yesterday that I was trying too hard and just copied the
new RSL routins over to my extlib/rsl/src and made a couple of rather small
additions to handle reading radar locations from $GEMTBL files and, somewhat
amazingly, I now am able to use gpnexr2 and nexr2rhi to read and display Build
19 files.
I’m working on writing up what I did and am looking for any volunteers who
would like to see if my directions make sense and are usable.
Best,
Clint
[https://ucomm.unl.edu/images/email-signature/Nebraska_N_RGB_small_on_white.gif]
Clinton Rowe
Professor & Chair
University of Nebraska–Lincoln
Earth and Atmospheric Sciences<https://eas.unl.edu/>
402.472.1946<tel:4024721946>
From: ldm-users <ldm-users-bounces@xxxxxxxxxxxxxxxx> On Behalf Of Ryan May
Sent: Tuesday, September 29, 2020 1:43 PM
To: Ryan Hickman <ryan@xxxxxxxxxxxxxxxx>
Cc: LDM <ldm-users@xxxxxxxxxxxxxxxx>
Subject: Re: [ldm-users] level2 via LDM
Hi,
If you're doing any parsing of the data, it's important to note that Build 19
has added a new moment (CFP-clutter filter power removed) and changed ZDR to
use 16-bit values to store individual values. Any decoder relying on hard-coded
offsets/sizes is quite likely to choke in some fashion on these modifications.
If that's the cause, I'm not sure why that's only now showing up though--KDDC
started shipping this data back in May.
Ryan
On Tue, Sep 29, 2020 at 12:23 PM Ryan Hickman
<ryan@xxxxxxxxxxxxxxxx<mailto:ryan@xxxxxxxxxxxxxxxx>> wrote:
We process Level 2 files and are seeing confirmation of this. Random sites at
random times which are even crashing the ORPG reader for LDM files.
- Ryan
On Sep 29, 2020, at 12:07 PM, Karen Cooper - NOAA Affiliate via ldm-users
<ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>> wrote:
And I just saw my typo -- VCP 35 sails 1.
On Tue, Sep 29, 2020 at 1:03 PM Karen Cooper - NOAA Affiliate
<karen.cooper@xxxxxxxx<mailto:karen.cooper@xxxxxxxx>> wrote:
I probably should have stated -- I was specifically looking at KDVN and KESX --
both in VCP32 Sails1.
On Tue, Sep 29, 2020 at 12:57 PM Karen Cooper - NOAA Affiliate
<karen.cooper@xxxxxxxx<mailto:karen.cooper@xxxxxxxx>> wrote:
I'm just going to throw my 2 cents in.
I'm seeing the same number of packets in ldm data from radars in Build 18.2 and
Build 19. 67 packets from S -> E, but the packets from the Build 19 radar are
larger.
Also, I'm only seeing one time in the identifier from the radars and the
complete ldm files are about the same length of time apart -- 8-9 minutes.
We do our own reading of the data, and at first glance, things seem fine - with
multiple 00.50 cuts, but not other cuts?
On Tue, Sep 29, 2020 at 12:13 PM Victor Gensini
<vgensini@xxxxxxx<mailto:vgensini@xxxxxxx>> wrote:
Ohhhh..the fun continues. I have been troubleshooting offline with Daryl.
So, from what I can tell, the level2 files for some radars (for whatever
reason) are transmitting two volume scans per file. The radars in question
appear random, but it might be related to a recent upgrade in ROC software or
something. EDIT: the files that blow things up are SAILS_1!!!
The file sizes are double because (surprise!) there are two timesteps due to
this SAILS_1.
Copy of my pqact here, but I assume this is going to create somewhat of a
headache for others as well.
https://atlas.niu.edu/vgensini/temp/pqact.split.radar<https://urldefense.proofpoint.com/v2/url?u=https-3A__atlas.niu.edu_vgensini_temp_pqact.split.radar&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=ubRAum_abJ_VeyWwkZyVzQ&m=wqW2WLuyzyZU0t0LOCAuj0KF3bVKIE-fp2Is49kDI1s&s=h-QDaGbqc6qJ7tB3DK-0zu8M-k-rw0vDaHCBL3uXpmI&e=>
-----------------------------------------------
Vittorio (Victor) A. Gensini, Ph.D., CCM
Associate Professor
Department of Geographic and Atmospheric Sciences
Northern Illinois University
DeKalb, IL 60115
https://atlas.niu.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__atlas.niu.edu&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=ubRAum_abJ_VeyWwkZyVzQ&m=wqW2WLuyzyZU0t0LOCAuj0KF3bVKIE-fp2Is49kDI1s&s=1ul48zZaK4YDu3f8FTXb6odgGM3gHeanIRA2L4wWTk4&e=>
https://wcs.niu.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__wcs.niu.edu_&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=ubRAum_abJ_VeyWwkZyVzQ&m=wqW2WLuyzyZU0t0LOCAuj0KF3bVKIE-fp2Is49kDI1s&s=TaVGS_2Jr8INZjDrZ5BRwLkSYlpfiqudyFGoD4FeJqQ&e=>
________________________________
From: Herzmann, Daryl E [AGRON]
<akrherz@xxxxxxxxxxx<mailto:akrherz@xxxxxxxxxxx>>
Sent: Tuesday, September 29, 2020 11:38 AM
To: Victor Gensini <vgensini@xxxxxxx<mailto:vgensini@xxxxxxx>>;
ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
<ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>>
Subject: Re: level2 via LDM
Well Howdy,
This sounds like a queue that is not holding an hour's worth of data and thus
getting duplicated / triplicated product transmisions. My questions:
1) Is your LDM queue holding an hour's worth of data `pqmon`? Do you know if
your upstream sources are holding an hour's worth of data? Is your LDM logging
anything exciting about switching upstreams etc ?
2) Do the file write timestamps on the files make sense? Such that the write
time coincides with the end of the VCP transmission?
3) Do your file byte sizes match what is found on the files at AWS / my
realtime site?
https://mesonet-nexrad.agron.iastate.edu/level2/raw/KDVN/<https://urldefense.proofpoint.com/v2/url?u=https-3A__mesonet-2Dnexrad.agron.iastate.edu_level2_raw_KDVN_&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=ubRAum_abJ_VeyWwkZyVzQ&m=wqW2WLuyzyZU0t0LOCAuj0KF3bVKIE-fp2Is49kDI1s&s=Lb4ugxnNf9grIrRBrVNUKxxSETYlIdm_CjL3tqZ__4s&e=>
Do you have a recent example (today / yesterday) of a 2x / 3x file? Can you
share that file for others to interrogate!
daryl
--
/**
* daryl herzmann
* Systems Analyst III -- Iowa Environmental Mesonet
*
https://mesonet.agron.iastate.edu<https://urldefense.proofpoint.com/v2/url?u=https-3A__mesonet.agron.iastate.edu&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=ubRAum_abJ_VeyWwkZyVzQ&m=wqW2WLuyzyZU0t0LOCAuj0KF3bVKIE-fp2Is49kDI1s&s=4JZGzyWA3mHX5BoxTy2-2_EcQnl0oPy076S5-bJgF1I&e=>
*/
________________________________________
From: ldm-users
<ldm-users-bounces@xxxxxxxxxxxxxxxx<mailto:ldm-users-bounces@xxxxxxxxxxxxxxxx>>
on behalf of Victor Gensini <vgensini@xxxxxxx<mailto:vgensini@xxxxxxx>>
Sent: Tuesday, September 29, 2020 11:30 AM
To: ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
Subject: [ldm-users] level2 via LDM
Tom/Steve and LDM junkies,
Have there been any recent changes to the format of LEVEL2 radar files coming
over LDM that I perhaps missed?
A couple of weeks ago, I had a couple scripts start to break that are using
gpnexr2_gf with:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.
Backtrace for this error:
#0 0x7FD1DF3C1697
#1 0x7FD1DF3C1CDE
#2 0x7FD1DE8BC3AF
#3 0x7FD1DE9F0B54
#4 0x4BCFCB
#5 0x4B8E59
#6 0x4B93C7
#7 0x42A510
#8 0x4288FE
#9 0x40F5A7
#10 0x40903D
#11 0x40554A
#12 0x405588
#13 0x7FD1DE8A8504
#14 0x4046D7
#15 0xFFFFFFFFFFFFFFFF
Segmentation fault (core dumped)
A quick peek of the file size for these stations (relative to others) indicate
nearly a doubling (in some cases tripling) of the file size versus other
stations that are still working. As an example, compare your file sizes today
for KDVN and KILX, both of which are in VCP 35.
I'm using the standard hhmmssRadarII perl script to create the volumes off of
LDM. Again, no issues for some sites, and then this memory error is popping up
for others. I sense a change in file contents or format...
Any/all ideas welcome.
Victor
-----------------------------------------------
Vittorio (Victor) A. Gensini, Ph.D., CCM
Associate Professor
Department of Geographic and Atmospheric Sciences
Northern Illinois University
DeKalb, IL 60115
https://atlas.niu.edu<http://atlas.niu.edu<https://atlas.niu.edu%3chttp:/atlas.niu.edu>>
https://wcs.niu.edu<https://urldefense.proofpoint.com/v2/url?u=https-3A__wcs.niu.edu&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=ubRAum_abJ_VeyWwkZyVzQ&m=wqW2WLuyzyZU0t0LOCAuj0KF3bVKIE-fp2Is49kDI1s&s=3UeHETCywCPtjdWWaMv4457dxR92HRCYXk5nBtl-PP0&e=><http://wcs.niu.edu/<https://urldefense.proofpoint.com/v2/url?u=http-3A__wcs.niu.edu_&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=ubRAum_abJ_VeyWwkZyVzQ&m=wqW2WLuyzyZU0t0LOCAuj0KF3bVKIE-fp2Is49kDI1s&s=TaVGS_2Jr8INZjDrZ5BRwLkSYlpfiqudyFGoD4FeJqQ&e=>>
<http://a><http://atlas.niu.edu>
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web. Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe, visit:
https://www.unidata.ucar.edu/mailing_lists/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.unidata.ucar.edu_mailing-5Flists_&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=ubRAum_abJ_VeyWwkZyVzQ&m=wqW2WLuyzyZU0t0LOCAuj0KF3bVKIE-fp2Is49kDI1s&s=qMxi5fxGY3epRGz3tZz8wkkx2-vRIJs_-EzUQ_udN4o&e=>
--
"Outside of a dog, a book is a man's best friend. Inside of a dog, it's too
dark to read."
--Groucho Marx
-------------------------------------------
Karen.Cooper@xxxxxxxx<mailto:Karen.Cooper@xxxxxxxx>
Phone#: 405-325-6456
Cell: 405-834-8559
National Severe Storms Laboratory
--
"Outside of a dog, a book is a man's best friend. Inside of a dog, it's too
dark to read."
--Groucho Marx
-------------------------------------------
Karen.Cooper@xxxxxxxx<mailto:Karen.Cooper@xxxxxxxx>
Phone#: 405-325-6456
Cell: 405-834-8559
National Severe Storms Laboratory
--
"Outside of a dog, a book is a man's best friend. Inside of a dog, it's too
dark to read."
--Groucho Marx
-------------------------------------------
Karen.Cooper@xxxxxxxx<mailto:Karen.Cooper@xxxxxxxx>
Phone#: 405-325-6456
Cell: 405-834-8559
National Severe Storms Laboratory
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web. Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe, visit:
https://www.unidata.ucar.edu/mailing_lists/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.unidata.ucar.edu_mailing-5Flists_&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=ubRAum_abJ_VeyWwkZyVzQ&m=wqW2WLuyzyZU0t0LOCAuj0KF3bVKIE-fp2Is49kDI1s&s=qMxi5fxGY3epRGz3tZz8wkkx2-vRIJs_-EzUQ_udN4o&e=>
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web. Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx<mailto:ldm-users@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe, visit:
https://www.unidata.ucar.edu/mailing_lists/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.unidata.ucar.edu_mailing-5Flists_&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=ubRAum_abJ_VeyWwkZyVzQ&m=wqW2WLuyzyZU0t0LOCAuj0KF3bVKIE-fp2Is49kDI1s&s=qMxi5fxGY3epRGz3tZz8wkkx2-vRIJs_-EzUQ_udN4o&e=>
--
Ryan May, Ph.D.
Software Engineer
UCAR/Unidata
Boulder, CO
