LDM 6.1.0 Release Notes

• Changes from LDM 6.0.15

• Changes from LDM 6.0.14

• Known Problems

• Modified the product-queue module (pq):

  • Made available 4 data-product slots that were, otherwise, unvailable. This means that the minimum number of empty slots can now go to 0.

  • Added a writer-counter variable to the product-queue file. Opening the product-queue for writing increments the on-disk value and closing the product-queue decrements it. This allows for a fast determination of whether the product-queue was properly closed and, consequently, self-consistent. As a consequence, the script for starting the LDM system at boot-time has changed significantly (see the pqcheck(1) utility). Furthermore, ALL INGESTERS MUST BE RELINKED AGAINST THE NEW LDM LIBRARY FOR THE NEW CONSISTENCY-TEST TO BE RELIABLE.

  • Creation of a product-queue file now fails if the requested size of the file (which is contained in an "off_t") is too large to be represented by variables of type "size_t".

• Modified the LDM server (rpc.ldmd):

  • Modified setting of "from" time in downstream LDM for the data-product selection criteria. Now, it will not ever request any products that are older than the "time offset" argument of the "-o" option if it was specified or the "maximum latency" argument of the the "-m" option (default 1 hour) otherwise. Before this change, a downstream LDM could request data-products that were much older than this if the upstream LDM was unavailable for a long time.

  • Added a 1 second pause before an upstream LDM first starts sending data-products or notifications to a downstream LDM. This prevents, for example, a HEREIS packet arriving contiguously with a FEEDME reply and works around the poor RPC implementations of some operating systems.

• Modified the pqact(1) utility:

  • Added a work-around for the non-conformance of OSF/1 regarding the SA_RESTART option of sigaction(). On such an operating-system, a write to a pipe by pqact(1) could be terminated by a signal (e.g., SIGCHLD) causing the data-product to be incompletely processed.

  • Increased the maximum number of file descriptors that can be simultaneously open from "32" to something based on the output of "getconf OPEN_MAX".

  • Corrected examples in configuration-file (pqact.conf).

• Modified the pqcreate(1) utility: it now interprets the specification of the size of the product-queue by pqcreate(1) differently. Previously, a "K", "M", or "G" suffix would mean "kibi", "mibi", or "gibi" (powers of two nearest 1e3, 1e6, or 1e9); now it means "kilo", "mega", or "giga". This now makes "4000M" equal to "4G", whereas before they were different.

• Modified the ldmadmin(1) script:

  • Added the command "restart".

  • Added the options "-m maxLatency" and "-o offset" to the "start" command (the "restart" command also has these options).

  • Added the variable $pq_slots. This variable sets the number of data-product slots in the LDM product-queue.

  • Modified the interpretation of the specification of the size of the product-queue to be consistent with the pqcreate(1) modification, above.

  • Greatly increased the performance of the "ldmadmin queuecheck" command by switching from use of pqcat(1) to use of pqcheck(1).

  • Removed the "ps" command because there are just too many ps(1) implementations out there that differ from the UNIX standard and from each other (and the same computer can have incompatible versions).

  • The "watch" and "queuecheck" commands now first check that the LDM system is running.

  • Added printing of $numlogs and $log_rotate to the "ldmadmin config" command.

• Modified the scour(1) utility:

  • Greatly improved performance -- mostly by using xargs(1).

  • Removed package-specificity from examples in configuration-file (scour.conf).

• Modified the ulog(3) module: it no longer appends an ASCII NUL to messages when logging to a file.

• Changed the feedtype-string that is printed for NEXRAD Level II data-products from CRAFT to NEXRAD2. The strings CRAFT, NEXRD2, and NEXRAD2 can all be used to specify this feedtype in configuration-files, however.

• Increased the maximum number of statistical-reporting bins in the "binstats" module (which is used by rtstats(1) and pqbinstats(1)) from 96 to 4000 to eliminate underreporting problem now that more than 96 CRAFT radars exist.

• Corrected manual pages. Removed all references to the LDM 5 "Site Manager's Guide". Replaced references to "http://www.unidata.ucar.edu" with references to "/software/ldm/"

• Added code to work-around TCP bug introduced by patch APAR IY38541 for the AIX operating-system. This patch is included in maintenance-level 11 of AIX 4.3 and maintenance-level 5 of AIX 5.1. The bug makes TCP connection non-blocking, causing upstream LDM to exit due to "resource temporarily unavailable". With this work-around, LDM6-s running on the above AIX systems will be able to send data-products to requesting downstream LDM6-s. They will not, however, be able to satisfy requests by downstream LDM5-s. Also, the same O/S upgrades introduced an, apparently, related bug in the TCP layer that causes the connection from a downstream LDM running on the upgraded O/S to an upstream LDM to episodically dissapear. The upstream LDM reports a "broken pipe" and the downstream LDM reports that the connection to the upstream LDM "closed". This problem will be reported to IBM.

• The above bug also causes a connection from a downstream LDM running on the upgraded AIX system to episodically close for no apparent reason. Consequently, the reconnection strategy of a downstream LDM-6 has been modified to be more aggressive (i.e., to try to reconnect earlier) but with an exponential backoff to the default maximum of 30 seconds.

The default value for the variable $surf_size in the ldmadmin script is incorrect: instead of "2M", it should be "2000000" (with the quotes). The symptom of this problem is that the command ldmadmin mksurfqueue hangs. The solution is to edit the installed ldmadmin script, changing the value as mentioned.

Reporting Problems

If you encounter bugs or problems, please contact support@unidata.ucar.edu. Include in the email all relevant items mentioned in the instructions.