Good morning Joseph,
Your pqmon output shows that your queue only has ~560 seconds worth of
data within it. Typically, you want to see this value at least 3600
seconds (1 hour). The default settings of LDM and LDM's feed logic like
to see queue sizes that hold at least an hour of data. So linear
extrapolation would suggest that you want to expand the size by ~6x.
Here are some links regarding expanding the queue size
https://www.unidata.ucar.edu/software/ldm/ldm-current/basics/tips.html
https://www.unidata.ucar.edu/software/ldm/ldm-current/basics/LDM-registry.html
You should check your LDM logs for messages like 'processing last message
in queue', this is another sign that your system can not process messages
fast enough for your queue size. You should expand your queue size or
look at splitting up the pqact processing load into multiple pqact
processes. This is tricky as you get into other issues like how much
capacity your system has and how pqact jobs are configured.
As far as the limit, if you are on 64bit OS, I am unsure what that limit
is. It is likely larger than you need for this usage case :)
daryl
On Wed, 16 Jan 2019, Joseph Spenner wrote:
> Hello, I'm running an LDM and processing a lot of data from Unisys. I'm
> missing some data, and suspecting my product queue might not be big enough.
> I saw this page:
> https://www.unidata.ucar.edu/software/ldm/ldm-current/troubleshooting/pqmon.html
>
>
> And when I run that command, the age is well below 3600:
>
> $ pqmon -i 5
> Jan 16 21:40:45 pqmon NOTE: Starting Up (2603)
> Jan 16 21:40:45 pqmon NOTE: nprods nfree nempty nbytes maxprods
> maxfree minempty maxext age
> Jan 16 21:40:45 pqmon NOTE: 40061 94 1 807272424 40155
> 305 0 1240468552 560
> Jan 16 21:40:50 pqmon NOTE: 40084 71 1 815131664 40155
> 305 0 1232653232 565
> Jan 16 21:40:55 pqmon NOTE: 40085 71 0 816728704 40155
> 305 0 1231043728 569
> Jan 16 21:41:00 pqmon NOTE: 40078 78 0 820314760 40155
> 305 0 1227516928 572
> Jan 16 21:41:05 pqmon NOTE: 40100 55 1 821967680 40155
> 305 0 1225949200 576
>
> My queue file is about 2G:
> -rw-rw-r-- 1 ldm ldm 2052718592 Jan 16 21:54 ldm.pq
>
> I was looking to delete and recreate it, but not sure how to make it bigger.
> What is the limit? And how do I specify the size?
> Thanks!
> Regards,Joseph Spenner
--
/**
* daryl herzmann
* Systems Analyst III -- Iowa Environmental Mesonet
* https://mesonet.agron.iastate.edu
*/
