Re: [ldm-users] Missing/Duplicated RTMA grids on noaaport NGRID feed

  • To: "Herzmann, Daryl E [AGRON]" <akrherz@xxxxxxxxxxx>
  • Subject: Re: [ldm-users] Missing/Duplicated RTMA grids on noaaport NGRID feed
  • From: Gilbert Sebenste <gilbert@xxxxxxxxxxxxxxxx>
  • Date: Tue, 12 Feb 2019 07:10:12 -0600
Hey Daryl,

I can tell you that with LDM 6.13.7, there were major updates to the GEMPAK 
tables. There are also major fixes to it as well, as during beta testing, 
multiple issues were discovered.

If you are running a NOAAport ingest site, you should upgrade immediately for 
multiple reasons, including:

1. Completely updated GEMPAK grids; but more than that, it was discovered 
during beta testing that the files were too long under LDM 6.13.6 and previous, 
and thus weren’t being properly read. Then, other issues were discovered 
because of that. All of those are detailed in the CHANGE_LOG file.

2. As a result of #1, it also negatively affected LDM NOAAport ingest and relay 
stability.

3. Last but certainly not least, memory leaks and bugs which caused queue 
corruption on both the direct ingest and relay sides of the LDM, and a few 
security holes were patched as well (now do I have your attention? :-)  ), 
including the potential to write over the binary, and the elimination of the 
use of the portmapper.

Check out the CHANGE_LOG for the details, but no matter who you are, 6.13.7 is 
a significant bugfix that you should install ASAP after testing. There are a 
number of bugs that have been around for years that have been swatted, 
including ldmadmin stop really does stop the LDM, and the stopping of the LDM 
now takes generally a much shorter amount of time to do (on one of the College 
of DuPage older NOAAport ingest machines, it used to take 10-45 minutes for the 
LDM to stop, if it did at all; it now takes 20 seconds to stop after the 
upgrade!). Additionally, random queue corruption has been fixed as well for all 
users. It really runs a lot smoother under CentOS 6/7 and Ubuntu 18. 

And if you go to the RTSTATS page, you will see all of UNIDATA and several 
universities have already upgraded to it.

I give it a huge thumbs up after testing it extensively since last April, when 
I was between jobs. I really wanted to put some of these bugs to bed, and in a 
good way, I opened a Pandora’s box in doing so. A lot of latent issues have 
been fixed. Again, check out the CHANGE_LOG in 6.13.7 for details.

Gilbert

> On Feb 11, 2019, at 11:14 PM, Herzmann, Daryl E [AGRON] <akrherz@xxxxxxxxxxx> 
> wrote:
> 
> Good evening all,
> 
> I continue to pull what hair is left on my head out trying to resolve why my 
> NGRID ingest is being so inconsistent lately.  As I boggle the RUC2/#130 
> products, I notice strange things like this:
> 
> LTDZ98 KWBG 111600 !grib2/ncep/RUC2/#130/201902111600F020/TMPK/2 m HGHT
> LTDZ98 KWBG 111500 !grib2/ncep/RUC2/#130/201902111500F020/TMP/2 m HGHT
> 
> Could there be upstream NGRID ingests running different versions of LDM and 
> thus getting different variable names translated?  ie TMPK vs TMP ?  Of 
> course, I have my fancy pants id3b archiving of products and their origins 
> and it denotes that:
> 
> TMPK -> np1.ssec.wisc.edu_v_192.168.1.10
> TMP -> np2.ssec.wisc.edu_v_192.168.1.10
> 
> Hmmm, but this example difference is not consistent as I look at other grids 
> from this model, so the goose chase continues!  Anybody know what is going on?
> 
> daryl
> 
> /**
> * daryl herzmann
> * Systems Analyst III -- Iowa Environmental Mesonet
> * https://mesonet.agron.iastate.edu
> */
> 
> ________________________________________
> From: ldm-users-bounces@xxxxxxxxxxxxxxxx <ldm-users-bounces@xxxxxxxxxxxxxxxx> 
> on behalf of Herzmann, Daryl E [AGRON] <akrherz@xxxxxxxxxxx>
> Sent: Tuesday, February 5, 2019 10:16:56 AM
> To: LDM Users
> Subject: [ldm-users] Missing/Duplicated RTMA grids on noaaport NGRID feed
> 
> Hi all,
> 
> I've noticed a recent uptick in the number of RTMA grids seemingly not being 
> sent over noaaport and thus available on the NGRID LDM feedtype.  For 
> example, the UREL field today is missing for the following hours 1, 2, 7, 11, 
> 13
> 
> An example UREL field at 15z today
> 
> LUIA98 KWBR 051500 !grib2/ncep/RTMA/#255/201902051500F000/UREL/10 m HGHT
> 
> Is anybody else noticing this or perhaps I have some processing latency on my 
> end causing grief?  As I review motherlode,
> 
> https://motherlode.ucar.edu/native/grid/NCEP/RTMA/CONUS_2p5km/
> 
> The hourly files all appear to be about the same size 79M, but the files only 
> have the UGRD field, not UREL.
> 
> My ignorance is always bliss and I feel like I may be missing something here.
> 
> Another exampe, the 13z UREL file did come on the 3rd
> 
> LUIA98 KWBR 031300 !grib2/ncep/RTMA/#255/201902031300F000/UREL/10 m HGHT
> 
> Puzzled and thanks!
> daryl
> 
> /**
> * daryl herzmann
> * Systems Analyst III -- Iowa Environmental Mesonet
> * https://mesonet.agron.iastate.edu
> */
> 
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
> 
> 
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/
> 
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
> 
> 
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/mailing_lists/ 



  • 2019 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: