LDM 6.0.14 Release Notes

Changes from LDM 6.0.13

• Ported package to HP-UX B.11.11 U.

• Fixed bug in downstream LDM 6 introduced in version 6.0.4: under some circumstances, a failed FEEDME or NOTIFY request wouldn't cause the connection to be closed and, consequently, the socket and client-handle resources to be released -- leading to a blocked, upstream LDM process with a read-lock on a product.

• Improved robustness of the rtstats(1) utility in the face of bad clock times (e.g., CRAFT data). This should prevent rtstats(1) from opening multiple connections in the same minute. THIS IS A BIG DEAL.

• Fixed memory-leak in receiving LDM process. When receiving from a sending LDM 6, the memory-usage of the receiving process would sometimes jump by about 250 kilobytes if the connection was broken and then re-established.

• Changed the effect of a SIGTERM to the top-level LDM process: it now sends a SIGINT to all child processes in order to exit in a more timely manner.

• Made the IS_ALIVE check on the upstream LDM more tolerant of connection failures. This should reduce the number of unnecessary reconnections.

• Modified LDM 6 product-sending code so that it checks the connection if it hasn't sent anything for 30 seconds. This should help prevent the buildup of upstream processes.

• Fixed latent bug in module "up6". Too few arguments passed to udebug(3) when a product-queue lock is encountered.

• Fixed error-reporting bug. Receiving LDM-s will no longer attempt to unregister from the portmapper -- eliminating the possibility of disconcerting (but harmless) "pmap_unset()" failure messages.

• Modified the "ldmadmin start" command to check the syntactic validity of the pqact(1) configuration-file if it exists. This should help the user who changes that configuration-file without checking it.

• Improved error-logging of LDM 6 connection attempts. If an attempt fails, a error-message will be logged at the ERROR level. The reason or reasons behind the failures of the port 388 and portmapper attempts are logged at the INFO level.

• Modified the LDM so that a SIGUSR1 causes the pseudo-random number generator in the product-queue module to be reset. This feature is for testing only, shouldn't be used, and will probably go away.

• Eliminated use of the word "class" as a variable name to accomodate C++.

• Added observance of "done" flag to module ldm_clnt -- resulting in more responsive termination of an LDM system that is attempting to connect to an upstream LDM. Modified module acl accordingly.

Changes from LDM 6.0.12

• Fixed a small memory-leak bug in the utility rtstats(1). If the destination LDM was version 5, then it would leak about 37 bytes per transmission.

• Modified the file-existance tests in ldmadmin script so that they will work on systems (e.g., Linux) that would, otherwise, require a perl utility with compiled-in support for large-files.

Changes from LDM 6.0.11

• Modified strategy for determining which program version is used when connecting to an upstream LDM (version 5 or 6): a downstream LDM will now connect using version 5 if and only if a previous version 6 connection attempt receives a program version mismatch indication (RPC_PROGVERSMISMATCH). This should eliminate the possibility of a downstream LDM-6 connecting to an upstream LDM-6 using LDM-5 protocols during the "window of vulnerability" that exists when the upstream LDM system is started.

• Changed downstream LDM code so that it doesn't sleep before reconnecting to the upstream LDM when the connection is closed due to a timeout. This will reduce the magnitude of product latencies due to connection timeouts at the cost of more log messages.

• Synchronous checks have been added in several places to make the LDM more responsive to a termination request (such as from an "ldmadmin stop").

• To work around a bug in Unisys ingest systems, the program pqing(1) has been modified to allow an extraneous newline between the end-of-product and beginning-of-product delimiters.

Changes from LDM 6.0.10

• A downtstream, requesting LDM process will no longer terminate if it can't connect to the upstream host in any fashion or if the upstream host isn't running an LDM. Instead, it will continue to retry.

Changes from LDM 6.0.3

• Elimiated bug in ldmsend and rtstats that caused them to crash when sending to an LDM 6.

• Eliminated bug in downstream LDM 6 that, under certain circumstances, caused an assertion failure upon receipt of a RECLASS reply from an upstream LDM.

• Eliminated bug in downstream LDM 6 that caused "wanted" and "allowed" product-class specifications in diagnostic RECLASS message to be identical.

• Eliminated bug in upstream LDM 6 that caused wanted" "and "allowed" product-class specifications in diagnostic "Restricting request" message to be identical.

• Demoted logging-level of rtstats(1) "couldn't connect" message from FAILURE, to WARNING to lessen user anxiety and because failure of rtstats to connect affects statistics reporting but not data-product transmission.

• Fixed small memory leak that occurred in ldmsend and rtstats everytime an ERROR: message was printed.

• Improved rtstats(1) connection algorithm, resulting in elimination of crashing due to segmentation violation and fewer error messages.

Changes from LDM 6.0.2

• Improved some diagnostic messages.

• Made ldmadmin pqactcheck return a failure exit-status if the configuration file contains a syntax error.

• Generalized receiving LDM: now accepts multiple BLKDATA messages for a product.

• A downstream LDM no longer adjusts the time-interval of acceptible data products when replying to a HIYA. Duplicate product detection is still in effect. This change prevents rejection of data products sent using ldmsend by a host with an inaccurate clock. It also places the burden of knowing what to send on the upstream HIYA- initiating LDM.

• Fixed bug that prevented the maximum hereis size field of ACCEPT entries in the LDM configuration-file from having an effect.

• Ported the ldmsend and rtstats utilities to LDM-6 protocols: they can now send to a version 6 LDM. Improved support for BSD-derived systems in the ldmadmin script.

Changes from LDM 5

Platform Support

The LDM package has been ported to FreeBSD 4.5 and OSF/1 V5.1

Installation

• The directory $LDMHOME/etc is created during installation if it doesn't exist.

• The LDM configuration-file, ldmd.conf, is copied to $LDMHOME/etc if it's not already there.

rpc.ldmd(1)

• The performance of the LDM has been greatly improved:

  • The LDM now uses batched RPC calls whenever possible.

  • The LDM no longer consolidates FEEDME requests to the same upstream LDM into one request (and consequent receiving LDM): a receiving LDM is now spawned for every upstream FEEDME request in the LDM configuration file. This will likely result in many more rpc.ldmd child processes on both the upstream and downstream hosts. This modification is also a prerequisite for assigning priorities to products,

  • The LDM can now receive BLKDATA data-products of arbitrary size: the dependency on DBUFMAX has been eliminated.

  • The FEEDME request and HIYA reply have been enhanced with a parameter that specifies the maximum size, in bytes, that the upstream LDM should use in its HEREIS messages. This parameter is called maximum HEREIS size.

• The timeliness with which child FEEDME and NOTIFYME processes terminate if they haven't yet connected to the upstream LDM has been greatly improved.

• A maximum HEREIS size field has been added to the REQUEST and ACCEPT entries of the LDM configuration-file.

• Log messages are clearer.

• Receiving LDM processes are more aggressive in their attempts to connect to an upstream LDM.

• Sending LDM processes are far less likely to timeout.

• Receiving LDM processes are now allowed to receive products of the requested product-class regardless of any ALLOW entries in the configuration-file.

• It is no longer necessary to run the portmapper daemon on the LDM's host system.

• An IS_ALIVE RPC message has been added so that a downstream LDM process can inquire about the status of its upstream LDM process.

• Support for protocol version 4 has been removed.

pqing(1)

The "pqing" utility now uses the function scan_wmo_binary when the feedtype is IDS|DDPLUS for SDI ingest. We are guaranteed that these products will end in the 4 byte FOS trailing sequence. The previously-used, less complete function scan_wmo terminated a product if a single CTRL-C is found within the product. This would cause check-sum differences with products inserted without using pqing.

ldmadmin(1)

• The utility sets the current working directory to the LDM's home directory before taking any action to constrain the location of LDM core files.

• The ldmadmin stop command no longer terminates successfully even though the LDM is still running. Additionally, the command now prints a status message if it needs to wait for the LDM to stop.

• It is no longer necessary to create an initial, empty logfile to successfully start the LDM. The newlog utility (which is invoked by the ldmadmin start command) now ensures that the logfile exists.

• The ldmadmin start command is now faster and will terminate with a bad status if the ldmd.pid file exists because this indicates, at a minimum, a bad shutdown of a previously running LDM.

• The ldmadmin queuecheck command now uses pqcat -s to perform a more rigorous check of the product queue. For good performance, this should only be used when the LDM is stopped.

• The script wlll now abort if the name of teh host computer is not a fully-qualified domain name.

rtstats(1)

newlog(1)

ulog(3)

• Log messages to the syslogd(8) daemon will no longer be truncated unless the operating system requires it.

• The logging function vulog(3) is now publicly-accessible.

Development Support

• The target binary.tar.Z has been added to the top-level makefile template Makefile.in.

• The following items have been made configurable at build-time via the configure script: the port and RPC program number used by the LDM (--with-alt-port), the logging facility used by the LDM system (--with-local1), and the type of product queue (--with-faux-pq). These features are intended to be used only by the Unidata Program Center for testing purposes.

Known Problems

There are no known problems at this time.

Reporting Problems