[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Support #BLB-756234]: LDM data lagging when writing to disk


> We are having and LDM issue that I'm hoping you might be able to help us
> with.  We are placing data on the LDM queue for a machine and that
> machine writes the data out to files.  What we are seeing is a lag of
> 8-10 minutes or more from when the data is received on the LDM queue to
> when it actually gets written out to disk.  Would you happen to know
> what might be causing this lag?  The machine does receive a lot of data
> on the LDM queue, but it seems to work fine some times and then take
> 8-10 minutes to write the files out at other times.  We are stumped on
> this one.  Have you seen this behavior before?  Do you know what might
> be causing it?  Any help you can provide would be greatly appreciated.

Assuming that the data-products are being written to files using the FILE 
action of pqact(1), then the reason for the delay could very well be caused by 
the throughput through the LDM product-queue. If you're receiving a lot of 
data-products (e.g., all of CONDUIT) and doing a lot of processing of that 
data, then when the data arrives at a high rate (which CONDUIT does 
periodically) the system will be so busy that it might take some minutes to 
FILE a just-added data-product.

For this reason, we recommend that LDMs that request a lot of data don't 
process that data locally but, instead, pass the data on to other, requesting 
LDMs that process subsets of the data.

Hope this helps.

> Thank you,
> Mr. Stacey Simmons
> WeahterTAP.com

Steve Emmerson

Ticket Details
Ticket ID: BLB-756234
Department: Support LDM
Priority: Normal
Status: Closed

NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.