Hi,
If you're doing any parsing of the data, it's important to note that Build
19 has added a new moment (CFP-clutter filter power removed) and changed
ZDR to use 16-bit values to store individual values. Any decoder relying on
hard-coded offsets/sizes is quite likely to choke in some fashion on these
modifications. If that's the cause, I'm not sure why that's only now
showing up though--KDDC started shipping this data back in May.
Ryan
On Tue, Sep 29, 2020 at 12:23 PM Ryan Hickman <ryan@xxxxxxxxxxxxxxxx> wrote:
> We process Level 2 files and are seeing confirmation of this. Random sites
> at random times which are even crashing the ORPG reader for LDM files.
>
> - Ryan
>
> On Sep 29, 2020, at 12:07 PM, Karen Cooper - NOAA Affiliate via ldm-users <
> ldm-users@xxxxxxxxxxxxxxxx> wrote:
>
>
> And I just saw my typo -- VCP 35 sails 1.
>
> On Tue, Sep 29, 2020 at 1:03 PM Karen Cooper - NOAA Affiliate <
> karen.cooper@xxxxxxxx> wrote:
>
>> I probably should have stated -- I was specifically looking at KDVN and
>> KESX -- both in VCP32 Sails1.
>>
>> On Tue, Sep 29, 2020 at 12:57 PM Karen Cooper - NOAA Affiliate <
>> karen.cooper@xxxxxxxx> wrote:
>>
>>> I'm just going to throw my 2 cents in.
>>>
>>> I'm seeing the same number of packets in ldm data from radars in Build
>>> 18.2 and Build 19. 67 packets from S -> E, but the packets from the Build
>>> 19 radar are larger.
>>>
>>> Also, I'm only seeing one time in the identifier from the radars and the
>>> complete ldm files are about the same length of time apart -- 8-9 minutes.
>>>
>>> We do our own reading of the data, and at first glance, things seem fine
>>> - with multiple 00.50 cuts, but not other cuts?
>>>
>>>
>>>
>>> On Tue, Sep 29, 2020 at 12:13 PM Victor Gensini <vgensini@xxxxxxx>
>>> wrote:
>>>
>>>> Ohhhh..the fun continues. I have been troubleshooting offline with
>>>> Daryl.
>>>>
>>>> So, from what I can tell, the level2 files for some radars (for
>>>> whatever reason) are transmitting two volume scans per file. The radars in
>>>> question appear random, but it might be related to a recent upgrade in ROC
>>>> software or something. *EDIT: the files that blow things up are
>>>> SAILS_1!!!*
>>>>
>>>> The file sizes are double because (surprise!) there are two timesteps
>>>> due to this SAILS_1.
>>>>
>>>> Copy of my pqact here, but I assume this is going to create somewhat of
>>>> a headache for others as well.
>>>>
>>>> https://atlas.niu.edu/vgensini/temp/pqact.split.radar
>>>>
>>>> -----------------------------------------------
>>>>
>>>> Vittorio (Victor) A. Gensini, Ph.D., CCM
>>>>
>>>> Associate Professor
>>>>
>>>> Department of Geographic and Atmospheric Sciences
>>>>
>>>> Northern Illinois University
>>>>
>>>> DeKalb, IL 60115
>>>>
>>>> https://atlas.niu.edu <http://atlas.niu.edu>
>>>> https://wcs.niu.edu <http://wcs.niu.edu/>
>>>>
>>>> <http://a> <http://atlas.niu.edu>
>>>> ------------------------------
>>>> *From:* Herzmann, Daryl E [AGRON] <akrherz@xxxxxxxxxxx>
>>>> *Sent:* Tuesday, September 29, 2020 11:38 AM
>>>> *To:* Victor Gensini <vgensini@xxxxxxx>; ldm-users@xxxxxxxxxxxxxxxx <
>>>> ldm-users@xxxxxxxxxxxxxxxx>
>>>> *Subject:* Re: level2 via LDM
>>>>
>>>> Well Howdy,
>>>>
>>>> This sounds like a queue that is not holding an hour's worth of data
>>>> and thus getting duplicated / triplicated product transmisions. My
>>>> questions:
>>>>
>>>> 1) Is your LDM queue holding an hour's worth of data `pqmon`? Do you
>>>> know if your upstream sources are holding an hour's worth of data? Is your
>>>> LDM logging anything exciting about switching upstreams etc ?
>>>>
>>>> 2) Do the file write timestamps on the files make sense? Such that the
>>>> write time coincides with the end of the VCP transmission?
>>>>
>>>> 3) Do your file byte sizes match what is found on the files at AWS / my
>>>> realtime site?
>>>>
>>>> https://mesonet-nexrad.agron.iastate.edu/level2/raw/KDVN/
>>>>
>>>> Do you have a recent example (today / yesterday) of a 2x / 3x file?
>>>> Can you share that file for others to interrogate!
>>>>
>>>> daryl
>>>>
>>>> --
>>>> /**
>>>> * daryl herzmann
>>>> * Systems Analyst III -- Iowa Environmental Mesonet
>>>> * https://mesonet.agron.iastate.edu
>>>> */
>>>>
>>>> ________________________________________
>>>> From: ldm-users <ldm-users-bounces@xxxxxxxxxxxxxxxx> on behalf of
>>>> Victor Gensini <vgensini@xxxxxxx>
>>>> Sent: Tuesday, September 29, 2020 11:30 AM
>>>> To: ldm-users@xxxxxxxxxxxxxxxx
>>>> Subject: [ldm-users] level2 via LDM
>>>>
>>>> Tom/Steve and LDM junkies,
>>>>
>>>> Have there been any recent changes to the format of LEVEL2 radar files
>>>> coming over LDM that I perhaps missed?
>>>>
>>>> A couple of weeks ago, I had a couple scripts start to break that are
>>>> using gpnexr2_gf with:
>>>>
>>>> Program received signal SIGSEGV: Segmentation fault - invalid memory
>>>> reference.
>>>>
>>>> Backtrace for this error:
>>>> #0 0x7FD1DF3C1697
>>>> #1 0x7FD1DF3C1CDE
>>>> #2 0x7FD1DE8BC3AF
>>>> #3 0x7FD1DE9F0B54
>>>> #4 0x4BCFCB
>>>> #5 0x4B8E59
>>>> #6 0x4B93C7
>>>> #7 0x42A510
>>>> #8 0x4288FE
>>>> #9 0x40F5A7
>>>> #10 0x40903D
>>>> #11 0x40554A
>>>> #12 0x405588
>>>> #13 0x7FD1DE8A8504
>>>> #14 0x4046D7
>>>> #15 0xFFFFFFFFFFFFFFFF
>>>> Segmentation fault (core dumped)
>>>>
>>>> A quick peek of the file size for these stations (relative to others)
>>>> indicate nearly a doubling (in some cases tripling) of the file size versus
>>>> other stations that are still working. As an example, compare your file
>>>> sizes today for KDVN and KILX, both of which are in VCP 35.
>>>>
>>>> I'm using the standard hhmmssRadarII perl script to create the volumes
>>>> off of LDM. Again, no issues for some sites, and then this memory error is
>>>> popping up for others. I sense a change in file contents or format...
>>>>
>>>> Any/all ideas welcome.
>>>>
>>>> Victor
>>>>
>>>>
>>>> -----------------------------------------------
>>>>
>>>> Vittorio (Victor) A. Gensini, Ph.D., CCM
>>>>
>>>> Associate Professor
>>>>
>>>> Department of Geographic and Atmospheric Sciences
>>>>
>>>> Northern Illinois University
>>>>
>>>> DeKalb, IL 60115
>>>>
>>>> https://atlas.niu.edu<http://atlas.niu.edu>
>>>>
>>>> https://wcs.niu.edu<http://wcs.niu.edu/>
>>>>
>>>> <http://a><http://atlas.niu.edu>
>>>> _______________________________________________
>>>> NOTE: All exchanges posted to Unidata maintained email lists are
>>>> recorded in the Unidata inquiry tracking system and made publicly
>>>> available through the web. Users who post to any of the lists we
>>>> maintain are reminded to remove any personal information that they
>>>> do not want to be made public.
>>>>
>>>>
>>>> ldm-users mailing list
>>>> ldm-users@xxxxxxxxxxxxxxxx
>>>> For list information or to unsubscribe, visit:
>>>> https://www.unidata.ucar.edu/mailing_lists/
>>>>
>>>
>>>
>>> --
>>> *"Outside of a dog, a book is a man's best friend. Inside of a dog,
>>> it's too dark to read."*
>>> *--Groucho Marx*
>>>
>>> -------------------------------------------
>>> Karen.Cooper@xxxxxxxx
>>>
>>> Phone#: 405-325-6456
>>> Cell: 405-834-8559
>>> National Severe Storms Laboratory
>>>
>>>
>>
>> --
>> *"Outside of a dog, a book is a man's best friend. Inside of a dog, it's
>> too dark to read."*
>> *--Groucho Marx*
>>
>> -------------------------------------------
>> Karen.Cooper@xxxxxxxx
>>
>> Phone#: 405-325-6456
>> Cell: 405-834-8559
>> National Severe Storms Laboratory
>>
>>
>
> --
> *"Outside of a dog, a book is a man's best friend. Inside of a dog, it's
> too dark to read."*
> *--Groucho Marx*
>
> -------------------------------------------
> Karen.Cooper@xxxxxxxx
>
> Phone#: 405-325-6456
> Cell: 405-834-8559
> National Severe Storms Laboratory
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> https://www.unidata.ucar.edu/mailing_lists/
>
--
Ryan May, Ph.D.
Software Engineer
UCAR/Unidata
Boulder, CO
