LDM 6.0.13 Release Notes

Changes from LDM 6.0.12

• Fixed a small memory-leak bug in the utility rtstats(1). If the destination LDM was version 5, then it would leak about 37 bytes per transmission.

• Modified the file-existance tests in ldmadmin script so that they will work on systems (e.g., Linux) that would, otherwise, require a perl utility with compiled-in support for large-files.

Changes from LDM 6.0.11

• Modified strategy for determining which program version is used when connecting to an upstream LDM (version 5 or 6): a downstream LDM will now connect using version 5 if and only if a previous version 6 connection attempt receives a program version mismatch indication (RPC_PROGVERSMISMATCH). This should eliminate the possibility of a downstream LDM-6 connecting to an upstream LDM-6 using LDM-5 protocols during the "window of vulnerability" that exists when the upstream LDM system is started.

• Changed downstream LDM code so that it doesn't sleep before reconnecting to the upstream LDM when the connection is closed due to a timeout. This will reduce the magnitude of product latencies due to connection timeouts at the cost of more log messages.

• Synchronous checks have been added in several places to make the LDM more responsive to a termination request (such as from an "ldmadmin stop").

• To work around a bug in Unisys ingest systems, the program pqing(1) has been modified to allow an extraneous newline between the end-of-product and beginning-of-product delimiters.

Changes from LDM 6.0.10

• A downtstream, requesting LDM process will no longer terminate if it can't connect to the upstream host in any fashion or if the upstream host isn't running an LDM. Instead, it will continue to retry.

Changes from LDM 6.0.3

• Elimiated bug in ldmsend and rtstats that caused them to crash when sending to an LDM 6.

• Eliminated bug in downstream LDM 6 that, under certain circumstances, caused an assertion failure upon receipt of a RECLASS reply from an upstream LDM.

• Eliminated bug in downstream LDM 6 that caused "wanted" and "allowed" product-class specifications in diagnostic RECLASS message to be identical.

• Eliminated bug in upstream LDM 6 that caused wanted" "and "allowed" product-class specifications in diagnostic "Restricting request" message to be identical.

• Demoted logging-level of rtstats(1) "couldn't connect" message from FAILURE, to WARNING to lessen user anxiety and because failure of rtstats to connect affects statistics reporting but not data-product transmission.

• Fixed small memory leak that occurred in ldmsend and rtstats everytime an ERROR: message was printed.

• Improved rtstats(1) connection algorithm, resulting in elimination of crashing due to segmentation violation and fewer error messages.

Changes from LDM 6.0.2

• Improved some diagnostic messages.

• Made ldmadmin pqactcheck return a failure exit-status if the configuration file contains a syntax error.

• Generalized receiving LDM: now accepts multiple BLKDATA messages for a product.

• A downstream LDM no longer adjusts the time-interval of acceptible data products when replying to a HIYA. Duplicate product detection is still in effect. This change prevents rejection of data products sent using ldmsend by a host with an inaccurate clock. It also places the burden of knowing what to send on the upstream HIYA- initiating LDM.

• Fixed bug that prevented the maximum hereis size field of ACCEPT entries in the LDM configuration-file from having an effect.

• Ported the ldmsend and rtstats utilities to LDM-6 protocols: they can now send to a version 6 LDM. Improved support for BSD-derived systems in the ldmadmin script.

Changes from LDM 5

Platform Support

The LDM package has been ported to FreeBSD 4.5 and OSF/1 V5.1

Installation

• The directory $LDMHOME/etc is created during installation if it doesn't exist.

• The LDM configuration-file, ldmd.conf, is copied to $LDMHOME/etc if it's not already there.

rpc.ldmd(1)

• The performance of the LDM has been greatly improved:

  • The LDM now uses batched RPC calls whenever possible.

  • The LDM no longer consolidates FEEDME requests to the same upstream LDM into one request (and consequent receiving LDM): a receiving LDM is now spawned for every upstream FEEDME request in the LDM configuration file. This will likely result in many more rpc.ldmd child processes on both the upstream and downstream hosts. This modification is also a prerequisite for assigning priorities to products,

  • The LDM can now receive BLKDATA data-products of arbitrary size: the dependency on DBUFMAX has been eliminated.

  • The FEEDME request and HIYA reply have been enhanced with a parameter that specifies the maximum size, in bytes, that the upstream LDM should use in its HEREIS messages. This parameter is called maximum HEREIS size.

• The timeliness with which child FEEDME and NOTIFYME processes terminate if they haven't yet connected to the upstream LDM has been greatly improved.

• A maximum HEREIS size field has been added to the REQUEST and ACCEPT entries of the LDM configuration-file.

• Log messages are clearer.

• Receiving LDM processes are more aggressive in their attempts to connect to an upstream LDM.

• Sending LDM processes are far less likely to timeout.

• Receiving LDM processes are now allowed to receive products of the requested product-class regardless of any ALLOW entries in the configuration-file.

• It is no longer necessary to run the portmapper daemon on the LDM's host system.

• An IS_ALIVE RPC message has been added so that a downstream LDM process can inquire about the status of its upstream LDM process.

• Support for protocol version 4 has been removed.

pqing(1)

The "pqing" utility now uses the function scan_wmo_binary when the feedtype is IDS|DDPLUS for SDI ingest. We are guaranteed that these products will end in the 4 byte FOS trailing sequence. The previously-used, less complete function scan_wmo terminated a product if a single CTRL-C is found within the product. This would cause check-sum differences with products inserted without using pqing.

ldmadmin(1)

• The utility sets the current working directory to the LDM's home directory before taking any action to constrain the location of LDM core files.

• The ldmadmin stop command no longer terminates successfully even though the LDM is still running. Additionally, the command now prints a status message if it needs to wait for the LDM to stop.

• It is no longer necessary to create an initial, empty logfile to successfully start the LDM. The newlog utility (which is invoked by the ldmadmin start command) now ensures that the logfile exists.

• The ldmadmin start command is now faster and will terminate with a bad status if the ldmd.pid file exists because this indicates, at a minimum, a bad shutdown of a previously running LDM.

• The ldmadmin queuecheck command now uses pqcat -s to perform a more rigorous check of the product queue. For good performance, this should only be used when the LDM is stopped.

• The script wlll now abort if the name of teh host computer is not a fully-qualified domain name.

rtstats(1)

newlog(1)

ulog(3)

• Log messages to the syslogd(8) daemon will no longer be truncated unless the operating system requires it.

• The logging function vulog(3) is now publicly-accessible.

Development Support

• The target binary.tar.Z has been added to the top-level makefile template Makefile.in.

• The following items have been made configurable at build-time via the configure script: the port and RPC program number used by the LDM (--with-alt-port), the logging facility used by the LDM system (--with-local1), and the type of product queue (--with-faux-pq). These features are intended to be used only by the Unidata Program Center for testing purposes.

Known Problems

Linux, large product-queues, and ldmadmin

In order for the ldmadmin script to correctly test for the existance of a large product-queue (approximately 2 gigabytes to 4 gigabytes) on a Linux system, the perl utility must have been built with support for large files. If the perl utility doesn't support large files and the product-queue lies within the above range, then the ldmadmin script will not work correctly.

Solutions include the following: