[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #AZV-974943]: LDM 6.12.6 Question

Mike,

pqact(1) uses the signature (i.e., MD5 checksum) of the last, 
successfully-processed data-product from a "*.state" file to determine where to 
start processing data-products from the product-queue. If the corresponding 
product exists, then pqact(1) will start just after that product; otherwise, 
pqact(1) will start with the product whose age in the product-queue in seconds 
is given by the argument of the "-o" option. The default is zero, i.e., start 
with the most recent product.

On another issue, specifying a time-offset of 24 hours for the LDM system will 
only be effective if the product-queue of the upstream LDM holds at least 24 
hours worth of data.

> Hello Steve,
> 
> I was wondering if you could help me clear up one more point of
> understanding with LDM.
> 
> The way UNAVCO treats data is that it's timeless and needs to be collected
> regardless of delays.  I know this is different than the LDM model,
> but I think I have the tuning set to work for us.  After an outage I was
> using the fields time-offset and max-latency in the registry.xml file to
> have the downstream LDM request old queued files for up to 24 hours past
> current time.  This however was not producing the results I expected.
> I found the *.info files in the ldm home directory and tried to clear
> those so that there were no indications of the last file processed by
> the downstream LDM.
> 
> This still didn't help.  Next I cleared the downstream product queue and
> still had unexpected results.  What I finally discovered is that in the
> ldmd.conf file where pqact gets forked, needed to have add the -o flag
> set with a value to match the max-latency value.  Now that I have all the
> settings I need I believe I understand how the LDM process was working.
> The downstream was requesting the data from upstream correctly but then
> pqact was considering the data too old because the default offset for
> pqact was only around 10 minutes.  When pqact saw old data it sent a
> 'reclass' message I believe to tune the data request with another
> feedme request to the upstream LDM starting at the current time minus
> ten minutes.  This is what I gathered from the entries in the log file.
> 
> I think I now have everything appropriately tuned so that after an
> outage our downstream LDM should go back in time up to 24 hours to
> retrieve products starting with the last file processed as logged in the
> .info file.  Performance is good under this configuration so I'm happy
> with the version of LDM I am running.  What I was most curious about is
> could you confirm my understanding of the behavior of LDM and can you
> give me a reason why it's advantageous for pqact to run on it's own flags
> in the ldmd.conf file and not take it's offset from the registry.xml file.
> 
> Thanks
> 
> -Mike


Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: AZV-974943
Department: Support LDM
Priority: Normal
Status: Closed