LDM 6.1.0 Release Notes
Changes from LDM 6.0.15
- Modified the product-queue module (pq):
-
Made available 4 data-product slots that were, otherwise,
unvailable. This means that the minimum number of empty
slots can now go to 0.
-
Added a writer-counter variable to the product-queue file.
Opening the product-queue for writing increments the on-disk
value and closing the product-queue decrements it. This
allows for a fast determination of whether the product-queue
was properly closed and, consequently, self-consistent. As
a consequence, the script for starting the LDM system at
boot-time has changed significantly (see the pqcheck(1)
utility). Furthermore, ALL INGESTERS MUST BE RELINKED
AGAINST THE NEW LDM LIBRARY FOR THE NEW CONSISTENCY-TEST TO
BE RELIABLE.
-
Creation of a product-queue file now fails if the requested
size of the file (which is contained in an "off_t") is too
large to be represented by variables of type "size_t".
- Modified the LDM server (rpc.ldmd):
-
Modified setting of "from" time in downstream LDM for the
data-product selection criteria. Now, it will not ever
request any products that are older than the "time offset"
argument of the "-o" option if it was specified or the
"maximum latency" argument of the the "-m" option (default 1
hour) otherwise. Before this change, a downstream LDM could
request data-products that were much older than this if the
upstream LDM was unavailable for a long time.
-
Added a 1 second pause before an upstream LDM first starts
sending data-products or notifications to a downstream
LDM. This prevents, for example, a HEREIS packet arriving
contiguously with a FEEDME reply and works around the poor
RPC implementations of some operating systems.
- Modified the pqact(1) utility:
-
Added a work-around for the non-conformance of OSF/1
regarding the SA_RESTART option of sigaction(). On such
an operating-system, a write to a pipe by pqact(1) could
be terminated by a signal (e.g., SIGCHLD) causing the
data-product to be incompletely processed.
-
Increased the maximum number of file descriptors that can
be simultaneously open from "32" to something based on the
output of "getconf OPEN_MAX".
-
Corrected examples in configuration-file (pqact.conf).
- Modified the pqcreate(1) utility: it now interprets the
specification of the size of the product-queue by pqcreate(1)
differently. Previously, a "K", "M", or "G" suffix would mean
"kibi", "mibi", or "gibi" (powers of two nearest 1e3, 1e6, or 1e9);
now it means "kilo", "mega", or "giga". This now makes "4000M"
equal to "4G", whereas before they were different.
- Modified the ldmadmin(1) script:
-
Added the command "restart".
-
Added the options "-m maxLatency" and "-o offset" to the
"start" command (the "restart" command also has these
options).
-
Added the variable $pq_slots. This variable sets the number
of data-product slots in the LDM product-queue.
-
Modified the interpretation of the specification of the size
of the product-queue to be consistent with the pqcreate(1)
modification, above.
-
Greatly increased the performance of the "ldmadmin
queuecheck" command by switching from use of pqcat(1) to use
of pqcheck(1).
-
Removed the "ps" command because there are just too many
ps(1) implementations out there that differ from the UNIX
standard and from each other (and the same computer can have
incompatible versions).
-
The "watch" and "queuecheck" commands now first check that
the LDM system is running.
-
Added printing of $numlogs and $log_rotate to the "ldmadmin
config" command.
- Modified the scour(1) utility:
-
Greatly improved performance -- mostly by using xargs(1).
-
Removed package-specificity from examples in
configuration-file (scour.conf).
- Modified the ulog(3) module: it no longer appends an ASCII NUL to
messages when logging to a file.
- Changed the feedtype-string that is printed for NEXRAD Level
II data-products from CRAFT to NEXRAD2. The strings CRAFT,
NEXRD2, and NEXRAD2 can all be used to specify this feedtype in
configuration-files, however.
- Increased the maximum number of statistical-reporting bins in the
"binstats" module (which is used by rtstats(1) and pqbinstats(1))
from 96 to 4000 to eliminate underreporting problem now that more
than 96 CRAFT radars exist.
- Corrected manual pages. Removed all references to the
LDM 5 "Site Manager's Guide". Replaced references
to "http://www.unidata.ucar.edu" with references to
"/content/software/ldm/"
Changes from LDM 6.0.14
-
Added code to work-around TCP bug introduced by patch APAR
IY38541 for the AIX operating-system. This patch is included
in maintenance-level 11 of AIX 4.3 and maintenance-level 5 of
AIX 5.1. The bug makes TCP connection non-blocking, causing
upstream LDM to exit due to "resource temporarily unavailable".
With this work-around, LDM6-s running on the above AIX systems
will be able to send data-products to requesting downstream
LDM6-s. They will not, however, be able to satisfy requests
by downstream LDM5-s. Also, the same O/S upgrades introduced
an, apparently, related bug in the TCP layer that causes the
connection from a downstream LDM running on the upgraded O/S to
an upstream LDM to episodically dissapear. The upstream LDM
reports a "broken pipe" and the downstream LDM reports that the
connection to the upstream LDM "closed". This problem will be
reported to IBM.
-
The above bug also causes a connection from a downstream LDM
running on the upgraded AIX system to episodically close for no
apparent reason. Consequently, the reconnection strategy of a
downstream LDM-6 has been modified to be more aggressive (i.e.,
to try to reconnect earlier) but with an exponential backoff to
the default maximum of 30 seconds.
Known Problems
The default value for the variable $surf_size in the
ldmadmin script is incorrect: instead of "2M", it should be
"2000000" (with the quotes). The symptom of this problem is that the
command ldmadmin mksurfqueue hangs. The solution is to
edit the installed ldmadmin script, changing the value as
mentioned.
Reporting Problems
If you encounter bugs or problems, please contact support@unidata.ucar.edu.
Include in the email all relevant items mentioned in the instructions.
