Hi folks,
We have tried to make a larger queue many times. Whenever we then restart the
EDEX services we find that the queue has reverted back to the original LDM
queue size (as shown by regutil entry and actual queue file size). Has anyone
been able to make this change “stick?”
We have installed the 15.1.2 release. Here is a transcript of an attempt to
change the queue size that we just performed.
=================================
[root@hurricane ~]# edex stop
Stopping EDEX Camel (request):
Stopping EDEX Camel (ingest):
Stopping EDEX Camel (ingestGrib):
EDEX ingestGrib shutdown
EDEX request shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
Waiting for EDEX ingest to shutdown
EDEX ingest shutdown
Stopping QPID
Session terminated, killing shell... ...killed.
/awips2/tools/bin/edex: line 338: 32642 Terminated su -c "service
qpidd stop"
Stopping httpd:
Stopping logging service: [ OK ]
Stopping EDEX PostgreSQL:
Stopping AWIPS II LDM:Stopping the LDM server...
Waiting for the LDM server to terminate...
[edex status]
postgres :: not running
pypies :: not running
qpid :: not running
EDEXingest :: not running
EDEXgrib :: not running
EDEXrequest :: not running
ldmadmin :: not running
[root@hurricane ~]# regutil /queue/size
1500M
[root@hurricane ~]# regutil -s "4096M" /queue/size
[root@hurricane ~]# regutil /queue/size
4096M
[root@hurricane ~]# edex start
Starting EDEX PostgreSQL:
Starting logging service: [ OK ]
Starting httpd: nohup: redirecting stderr to stdout
[ OK ]
Starting QPID
QPID is running (PID 988)
Starting EDEX Camel (request):
Starting EDEX Camel (ingest):
Starting EDEX Camel (ingestGrib):
EDEX Camel (request) is running (wrapper PID 1240)
EDEX Camel (request) is running (java PID 1422)
EDEX Camel (ingestGrib) is running (wrapper PID 1238)
EDEX Camel (ingestGrib) is running (java PID 1328)
EDEX Camel (ingest) is running (wrapper PID 1239)
EDEX Camel (ingest) is running (java PID 1499)
Cleaning LDM: [ OK ]
Creating the ldm queue: [ OK ]
Starting AWIPS II LDM:The product-queue is OK.
Checking pqact(1) configuration-file(s)...
/awips2/ldm/etc/pqact.conf: syntactically correct
Checking LDM configuration-file (/awips2/ldm/etc/ldmd.conf)...
Starting the LDM server...
[root@hurricane ~]# regutil /queue/size
1500M
[root@hurricane ~]#
As you can see, we used regutil to adjust the size, queried to confirm the
change, started the edex services and checked the queue size, only to find it
back to the 1500M default. Where does this come from?
Any help is welcome!
Chris Herbster
--
Dr. Christopher G. Herbster
Associate Professor
Director of Science and Technology
for the ERAU Weather Center
Applied Aviation Sciences
Embry-Riddle Aeronautical Univ.
600 S. Clyde Morris Blvd.
Daytona Beach, FL 32114-3900
386.226.6444 Office
386.226.6446 Weather Center
http://wx.erau.edu/
Schedule at: http://wx.erau.edu/faculty/herbster/Schedules/
From: awips2-users-bounces@xxxxxxxxxxxxxxxx
[mailto:awips2-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Michael James
Sent: Tuesday, March 22, 2016 4:29 PM
To: Brian Bernard <bbernard@xxxxxxxxxxxx>
Cc: awips2-users@xxxxxxxxxxxxxxxx
Subject: Re: [awips2-users] Optimizing AWIPS 2 and LDM
Any performance improvements with a larger LDM queue?
The grid log will show you which files are crashing the JVM, I suggest
disabling those models in pqact.conf. I've seen this happen with the NDFD and
I'm not sure why. Which other models cause core dumps?
With 15.1.3 there are new pqact.conf entries for a number of models, with a
default set enabled and most regional, ensemble, and duplicate models disabled.
The master set of grids to ingest to EDEX (including most regional models) is
https://raw.githubusercontent.com/Unidata/awips2/unidata_15.1.1/rpms/awips2.upc/Installer.ldm/patch/etc/pqact.grids
I'm able to process this entire set on a server with 16 cores, 24 GB memory,
and 1 TB SDD disk. But if I turn on other feeds, especially NEXRAD3, I have to
pare down the number grids I'm ingesting or everything backs up. Hence the
default set in 15.1.3 pqact.conf.
.
Michael James
Unidata Program Center
Boulder, CO
On Tue, Mar 22, 2016 at 3:17 PM, Brian Bernard
<bbernard@xxxxxxxxxxxx<mailto:bbernard@xxxxxxxxxxxx>> wrote:
Hello,
Does anyone have any tips on optimizing the AWIPS II EDEX server and LDM?
I mean optimizing to minimize data loss, and so data (particularly gridded
data) is processed quickly and with reduced latency.
At the present time, I’m using a Dell T3610 workstation for my demo AWIPS II
Edex server. It has 64GB RAM, four 512GB SSD and an Intel Xeon E5-1620
processor with eight cores.
I find that with some of the data, the server slows down and in fact it will
stop ingesting grib data after a couple of days. When I check the logs, I
notice a number of core dumps that have been generated.
So, if anyone can give any suggestions, I would greatly appreciate it.
Regards,
Brian Bernard
_______________________________________________
awips2-users mailing list
awips2-users@xxxxxxxxxxxxxxxx<mailto:awips2-users@xxxxxxxxxxxxxxxx>
For list information, to unsubscribe, or change your membership options, visit:
http://www.unidata.ucar.edu/mailing_lists/
