[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[LDM #WSJ-190258]: queue size question
- Subject: [LDM #WSJ-190258]: queue size question
- Date: Fri, 22 Mar 2019 15:05:39 -0600
John,
> We are in the process of moving to a virtual (vmware) linux system for
> down loading and creating our regional satellite/radar images and we've
> had it quit twice in two days after running normally for approximately
> 3 months since its creation.
What is "it"? The VM? The LDM?
> I noticed that pqmon shows a low max age number (well under 3600 seconds)
> of ~130:
>
> 20190322T150506.531572Z pqmon[6148] NOTE pqmon.c:358:main() nprods nfree
> nempty nbytes maxprods maxfree minempty maxext age
> 20190322T150506.531618Z pqmon[6148] NOTE pqmon.c:466:main() 11155 1
> 12682 1215750360 23837 275 0 3880 131
>
> I tried increasing the product queue but when I tried to increase it to 5G
> it reverts to 700M.
How did you try? By modifying the relevant registry parameter then deleting and
remaking the queue?
> I can set it to 3.5G, but the max age only increased t
> o around 300:
Then you'll need to make it much larger.
> 20190322T155936.243586Z pqmon[27704] NOTE pqmon.c:358:main() nprods nfree
> nempty nbytes maxprods maxfree minempty maxext age
> 20190322T155936.243646Z pqmon[27704] NOTE pqmon.c:466:main() 31748 1
> 36878 3499797704 39305 2 29321 205624 296
>
> Here's the command I used to increase the product queue. I was unable to
> use 3.5G and instead had to use this: regutil -u 3500000000 /queue/size
Ah! The command "regutil -s 3.5G /queue/size" could also have been used.
> The memory of the virtual linux server is 24G.
The queue should be large enough so that the minimum residence time is at least
one hour. Obviously, the necessary size will depend on your data inflow.
Periodically obtaining metrics and plotting them as described in the LDM
documentation is the best way to settle on a size.
> Result of ldmadmin config:
> hostname: mammatus.ttu.edu
> os: Linux
> release: 4.1.12-124.26.3.el7uek.x86_64
> ldmhome: /home/ldm
> LDM version: 6.13.6
> PATH:
> /home/ldm/ldm-6.13.6/bin:/home/ldm/decoders:/home/ldm/util:/home/ldm/bin:/home/ldm/decoders:/home/ldm/util:/home/ldm/bin:/home/ldm/perl5/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/ldm/.local/bin:/home/ldm/bin:/home/gempak/GEMPAK6.7.0/os/linux64/bin:/home/gempak/GEMPAK6.7.0/bin
> LDM conf file: /home/ldm/etc/ldmd.conf
> pqact(1) conf file: /home/ldm/etc/pqact.conf
> scour(1) conf file: /home/ldm/etc/scour.conf
> product queue: /home/ldm/var/queues/ldm.pq
> queue size: 3500000000 bytes
> queue slots: default
> reconciliation mode: increase queue
> pqsurf(1) path: /home/ldm/var/queues/pqsurf.pq
> pqsurf(1) size: 2M
> IP address: 0.0.0.0
> port: 388
> PID file: /home/ldm/ldmd.pid
> Lock file: /home/ldm/.ldmadmin.lck
> maximum clients: 512
> maximum latency: 3600
> time offset: 3600
> log file: /home/ldm/var/logs/ldmd.log
> numlogs: 7
> log_rotate: 1
> netstat: /bin/netstat -A inet -t -n
> top: /bin/top -b -n 1
> metrics file: /home/ldm/var/logs/metrics.txt
> metrics files: /home/ldm/var/logs/metrics.txt*
> num_metrics: 4
> check time: 1
> delete info files: 0
> ntpdate(1): /usr/sbin/ntpdate
> ntpdate(1) timeout: 5
> time servers: ntp.ucsd.edu ntp1.cs.wisc.edu ntppub.tamu.edu
> otc1.psu.edu timeserver.unidata.ucar.edu
> time-offset limit: 10
Regards,
Steve Emmerson
Ticket Details
===================
Ticket ID: WSJ-190258
Department: Support LDM
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata
inquiry tracking system and then made publicly available through the web. If
you do not want to have your interactions made available in this way, you must
let us know in each email you send to us.
