Hi Daryl,
On 7/3/23 09:05, Herzmann, Daryl E [AGRON] wrote:
I'm attempting to get straight in my head some relay latency between a LDM
server and client. Both are running the current release, 6.14.5. My simple
test is as follows, on the LDM upstream
pqinsert something every two seconds
on the LDM downstream, monitor receipt of these inserted products via `ldmadmin
watch` and also putting the LDM client connection into debug mode to see what
the RPC is doing. What I observe appears to LDM batching the RPC calls every
30 seconds instead of a more timely relay of inserted products to downstreams.
Your observation is mostly correct in the following sense:
If 'pqinsert' processes are run outside of the LDM process group, then
the LDM routines that send product to downstreams will not be informed
that there is a new product to send, but they will check every 30
seconts.. The default action for an upstreamLDM is to send products
when notified that there are products to send, or wait 30 seconds and
then check to see if there are new products to send. The net effect of
this for 'pqinsert' processes run outside of the LDM process group is
latencies that will range from 0 to 30 seconds with an average around 15
seconds.
There are two solutions for the "30 second delay":
- run the 'pqinsert' instances a part of the LDM process group
I.e., run them from an EXEC entry in the LDM configuration file.
- have the insertion process (e.g., BASH, Python, etc. script) send a
CONT signal to the _negative_ of the process ID of the lead LDM
server running on the machine (the process ID for the lead server is
stored in the ~ldm/ldmd.pid file)
re:
My typical naive and likely wrong understanding of LDM's code is that this is
controlled by:
src/registry/globals.c
/*
* Time to sleep in pq_suspend() and before retrying connects.
*/
unsigned int interval = 30;
So my questions:
1) Is this latency a function of my usage of `pqinsert` on the server side
instead of some proper LDM forked child process?
2) Are there side effects / significant performance concerns of setting the
`interval` value above to some lower threshold? I am thinking about setting it
to 1 or 5 seconds. Do I need to be careful to change this on all my LDM
servers and clients in my local topology?
3) Is there some other setting that controls this batching of RPC from the
server to the client?
Please let us know if the explanation above does not answer this
question.
Cheers,
Tom
--
+----------------------------------------------------------------------+
* Tom Yoksas UCAR Unidata Program *
* (303) 497-8642 (last resort) P.O. Box 3000 *
* yoksas@xxxxxxxx Boulder, CO 80307 *
* Unidata WWW Service http://www.unidata.ucar.edu/ *
+----------------------------------------------------------------------+
