LDM version 5.1.2 available
A new LDM, version 5.1.2, is now available from the LDM
ftp directory. Source code and binaries are now available. This version
has been tested on all of our supported platforms. HPUX users please note: We
have verified that version 5.1.2 runs on HPUX V10.20. The HPUX V10.20 binary
also runs on HPUX V11.0 but one cannot compile/run the ldm-5.1.2 version on
the HPUX V11.0.
Thank you to everyone who tested the beta release and reported problems. Thanks
especially to Harry Edmon and Gilbert Sebenste, who not only reported bugs but
served as guinea pigs to test our fixes.
The main changes from the last released version (5.0.9), as detailed in the
RELEASE_NOTES for 5.1.2, are to the
product queue data structures and algorithms to speed up the insertion, deletion,
and access to products. A tradeoff for the increased speed is that the product
queue can no longer grow dynamically as needed, so a fixed size queue of adequate
size must be created prior to running the LDM. Greater speedups over the previous
LDM are seen as the number of products in the queue increases.
Note: The product queue data structures are different from versions
5.1.1 or earlier. If you are currently using one of those versions you cannot
use an existing product queue when you switch to the new version. To use the
new version you must make a new queue. (You may want to keep the old queue
around temporarily until you're sure your installation is successful. To save
the old queue, simply rename it with a new file name.)
Other changes include:
- It is no longer necessary to run the pqexpire program
to delete old products out of the queue. The new version deletes old products
as needed to make space for new products. If you already have an LDM installation,
you should probably comment out the "exec pqexpire" entry in your ldmd.conf
file before starting the new LDM, to ensure pqexpire will not be started.
(New installations will get an ldmd.conf file with that line already commented
out.) It's still possible to run pqexpire for fine tuning, for example keeping
old large products out of the queue so there is more room for small products,
but we haven't seen much need to run it with the new LDM.
- The default queue creation has been modified to initialize the queue completely,
which is slower but safer than the previous behavior of delaying disk allocation
for the queue until needed. Previously, the queue was created very quickly
as a "sparse file", which means most of the disk space was not yet allocated.
This could cause mysterious crashes later, when disk space needed for the
queue exceeded available space.
However, an option to build the queue the old, faster way still exists.
This is invoked with a -f option to pqcreate, or via 'ldmadmin mkqueue -f'.
You can use this option if you are positive you have enough disk space for
the queue, for example, if you've been running with a queue of that size
for some time.
- The new release also includes a new program, pqmon, to monitor
the state of a product queue including the number of products in the queue,
the number of free regions, the number of bytes in-use, the maximum extent
of any free region, and the age of the oldest product in the queue.
- New feed types and synonyms for existing feed types were added to handle
planned additions and changes to the IDD data streams
- The 'restart' option to ldmadmin was deprecated. Although many people were
using this option, it often did not work properly. Now we recommend using
'ldmadmin stop' and 'ldmadmin start'. (After stopping, make sure all LDM processes
are dead before restarting.)
- Another feature available with the new version is the possibility of creating
and using product queues larger than 2 Gbytes. On 64-bit platforms such as
Sparc-v9, Alphas, and IRIX64 systems, it is possible to build the LDM programs
to support huge queues that use 64-bit offsets instead of 32-bit offsets,
so product queues can be as large as the amount of local disk space you have
available. If you feel adventuresome and have lots of disk space on a 64-bit
Unix system, you could try building a 64-bit version of the LDM programs.
We are developing a web page
with more details about this.
More detailed information on the improvements in the new LDM is available here.
Please see source code installation
for directions on how to install from a source code distribution.
Please see binary code installation
for directions on how to install from a binary code distribution.
Important Points For Version 5.1.2
- Optional: save the old product queue if you have room and may need to switch
between the running version and the new version in less than an hour.
- If you are currently running version 5.1.1 or earlier, then make a new
product queue before starting the LDM.
- If you install from a binary distribution, comment out the pqexpire line
in your existing ldmd.conf before starting the LDM.
- When your LDM is running, try invoking "pqmon -i 1" to see your queue filling
up with products.
- Send feedback or questions to support@unidata.ucar.edu.
This document is maintained by
Anne Wilson <anne@unidata.ucar.edu>
