Unidata - To provide the data services, tools, and cyberinfrastructure leadership that advance Earth system science, enhance educational opportunities, and broaden participation.

DataToolsCommunityDownloadsSupportProjectsAbout Us

•

LDM Reference Glossary and Index

This is a hyperlinked glossary and index of the terms, and phrases used to explain the fundamentals of the Unidata LDM.

A B C D E F G H I K L M N O P R S T U W Y

A

ACCEPT entry: An entry in the LDM configuration-file ldmd.conf that allows an upstream LDM to send unrequested data-products. Decription of entry in ldmd.conf.
ALLOW entry: An entry in the LDM configuration-file ldmd.conf that allows a downstream LDM to connect and indicates what data-products it is allowed to receive.
asynchronous: Describes something that does not cause the initiator to wait until the thing is finished. For example, an asynchronous RPC message does not cause the sender to wait for a reply.
autoshift: Describes the capability of a downstream LDM 6 to change the transfer-mode of its connection depending on its success-rate of inserting data-products into the product-queue. LDM 6.4 was the first version to have this capability.

B

BLKDATA: A message from an upstream LDM to a downstream LDM containing data-product data. LDM 6 protocol message.

C

child process: A UNIX^Ž process created by a parent process. Official description.
CONDUIT: A feedtype comprising a huge volume of model output.
creation-time, data-product: See data-product creation-time.
crontab utility: A UNIX^Ž utility for scheduling the periodic execution of commands. Official description.
COMINGSOON: An RPC message from an upstream LDM to a downstream LDM containing data-product metadata asking if the downstream LDM wants the corresponding data-product. LDM 6 protocol message.

D

data-product: The fundamental quantum of data acted upon by an LDM. Structure of a data-product.
data-product data: The data portion of a data-product. Further information.
data-product creation-time: The time when the data-product was created by the relevant LDM ingester.
data-product identifier: A character-string that uniquely identifies a data-product. Product identifiers are created by LDM ingesters using the policy of the administrative entity responsible for the relevant feedtype.
data-product latency: The difference between when a data-product is received by a downstream LDM and the data-product's creation-time.
data-product metadata: The metadata portion of a data-product. Further information.
data-product origin: A character-string that identifies the computer on which the LDM ingester was running when it created the data-product. Typically, this is the fully-qualified hostname of the computer.
data-product selection-criteria: The criteria used for selecting data-products. The criteria is composed of a start-time and a stop-time that must bracket the data-product creation-time of matching products, and a list of pairs of feedtype and data-product identifier ERE-s.
data-product signature: An array of 16 unsigned bytes that is unique to a data-product. Signatures are computed by LDM ingesters using the MD5 checksum algorithm. An MD5 checksum webpage.
data-product size: The number of bytes in the data section of a data-product. The value must be less than or equal to 0xFFFFFFFF (4294967295).
decoder: A feedtype-specific program that disposes of data-products that it receives from the pqact process. Further information.
deprecated: This item or feature might go away in a future release. Users are cautioned against using or depending on it.
downstream host: The computer on which a downstream LDM is running.
downstream LDM: Typically, a forked, child process of rpc.ldmd that sends a FEEDME request to an upstream LDM and, consequently, receives data-products, which it then writes into the product-queue. In atypical situations, this phrase can also refer to a notifyme process.
downstream LDM 5: A downstream LDM that uses the LDM 5 protocol. For backward compatibility, LDM 6 is capable of acting as a downstream LDM 5.
downstream LDM 6: A downstream LDM that uses the LDM 6 protocol. Behavior.

E

EXEC entry: An entry in the LDM configuration-file ldmd.conf that tells the top-level rpc.ldmd to asynchronously execute a program as a child process on startup.
environment variable: A generally mutable object with a name and a character-string value that can be set by a user via the shell or by a program. Copies of environment variables are inherited by child processes. Official description.
execvp() function: A UNIX^Ž function that executes a command-string. Official description.
ERE: Extended Regular Expression. ERE-s provide a mechanism to select specific strings from a set of character strings. ERE webpage.

F

FEEDME: A message from a downstream LDM to an upstream LDM requesting data-products. LDM 6 protocol message.
feedtype: A class of data-products coming from a common source (e.g., WMO, NPORT). Feedtypes may be hierarchically arranged. A feedtype may be represented as an integral value (e.g., 1) or as a name (e.g., PPS). Feedtype webpage.
fork: A UNIX^Ž system-call that duplicates the invoking parent process to create a new child process. Official description.

G

gateway LDM: A site's LDM that receives data from offsite or that sends data to an offsite LDM
gdbm database: A database based on the GNU dbm database routines.
GEMPAK: A UPC-supported package for analyzing and displaying primarily meteorological data (with emphasis on gridded model output). GEMPAK homepage.
grep utility: The UNIX^Ž utility for searching text files. Official description.

H

HEREIS: A message from an upstream LDM to a downstream LDM that contains a data-product.
HIYA: An unsolicited message from an upstream LDM to a downstream LDM specifying what data-products are about to be sent. LDM 6 protocol message.

I

IDD: The Unidata Internet Data Distribution system. The IDD system is a network of LDMs and their associated institutions that cooperate to distribute meteorological and related data-products for research and education under the auspices of the Unidata program. IDD homepage.
IDD rtstats webpages: A set of webpages for displaying the statistics of non-gateway LDMs on the Unidata IDD that are reported via the rtstats utility. Top-level webpage. Use in monitoring the LDM.
identifier, data-product: See data-product identifier.
ingester: A feedtype-specific program that creates data-products and inserts them into the product-queue. Further information.
insertion-time: The time at which a data-product was inserted into the local product-queue.
iostat utility: A commonly-available UNIX^Ž utility for displaying system I/O statistics.
IS_ALIVE: A message from a downstream LDM to the top-level rpc.ldmd process on an upstream host inquiring whether or not the relevant upstream LDM process is still running. LDM 6 protocol message.

K

kill utility: The UNIX^Ž utility for sending signals to processes. Official description.

L

LDM: Local Data Manager. A system of software for the distribution and processing of data-products in near real-time via the Internet. LDM homepage.
localn: The facility used by the LDM for logging messages via the syslog daemon. By default, the local0 facility is used. Configuring the syslog.conf file. Setting the facility via the configure script.
LDM, downstream: See downstream LDM.
LDM logfile: The file used to log diagnostic messages from LDM programs. Format and interpretation. Rotation and purging.
LDM, receiving: See downstream LDM.
LDM, sending: See upstream LDM.
LDM, upstream: See upstream LDM.
LDM 4: An LDM that uses the LDM 4 protocol.
LDM 4 protocol: The LDM protocol used by version 4 of the LDM.
LDM 5: An LDM that uses the LDM 5 protocol.
LDM 5 protocol: The LDM protocol used by version 5 of the LDM.
LDM 6: An LDM that uses the LDM 6 protocol.
LDM 6 protocol: The LDM protocol used by version 6 of the LDM. LDM 6 protocol webpage.
LDM protocol: The set of RPC messages used by a version of the LDM to communicate with another LDM of the same version.
LDM user: The user on a computer system that owns the local LDM installation. Typically, the user's name is "ldm". LDM user webpage. Creating user-account for. Becoming the LDM user.
ldmadmin: Perl script for managing an operational LDM system. Further information.
ldmadmin-pl.conf: Perl script for configuring the ldmadmin script. Further information.
ldmping: A utility that temporarily connects to an upstream LDM to test its availability.
ldmd.conf: The configuration-file for the top-level rpc.ldmd process. ldmd.conf webpage.
lex utility: A UNIX^Ž utility for generating a lexical scanner. Official description.
logfile, LDM: See LDM logfile.

M

make utility: A UNIX^Ž utility for maintaining, updating, and regenerating groups of programs. Official description.
man utility: A UNIX^Ž utility for displaying reference-documentation on UNIX^Ž utilities. Official description. User-support example.
MANPATH: The environment variable that determines what directories are searched for manual pages and in what order. Setting in preinstallation step. User-support reference. The man(1) utility.
maximum HEREIS size: The threshold, in bytes, above which an upstream LDM 6 uses a COMINGSOON/BLKDATA message-pair to send a data-product rather than a single HEREIS message.
McIDAS: A UPC-supported package for analyzing and displaying primarily meteorological data (with emphasis on satellite imagery). McIDAS homepage.
metadata, data-product: See data-product metadata.

N

NOTIFICATION: A message from an upstream LDM to an downstream LDM containing data-product metadata. LDM 6 protocol message.
NOTIFYME: An RPC message from a downstream LDM to an upstream LDM requesting data-product metadata. LDM 6 protocol message.
notifying upstream LDM 5: An upstream LDM 5 that sends NOTIFICATION messages to a downstream LDM 5 rather than data-products.
notifyme utility: A utility that makes a NOTIFYME request to an upstream LDM and, optionally, writes the data-product metadata that it receives to an LDM logfile.
ntpd: The Network Time Protocol Daemon. This freely-available program ensures that a computer's system clock is accurate, which is one of the prerequisites for correct operation of the LDM. ntp.org home page.
NULLPROC: A message used only to flush or verify a connection.

O

origin, data-product: See data-product origin.

P

parent process

The UNIX^Ž process that created a child process. Official description.

pathological regular-expression

An extended regular-expression (ERE) that is longer than two characters and has a ".*" prefix.

pathname

A character-string that identifies a file. Pathnames may be absolute or relative to the current working directory. Official description.

perl utility

A freely-available scripting utility. Official webpage.

PIPE action

See pqact PIPE action.

portmapper

A program that maps RPC program numbers and versions to Internet port numbers. The utility rpcbind is a portmapper.

pqact

A program that reads data-products from the product-queue and disposes of them according to user-configurable instructions in a pqact configuration-file. Further information.

pqact EXEC action

An entry in a pqact configuration-file that tells the associated pqact utility to execute the associated program as a child process when a matching data-product is encountered. Configuration-file entry.

pqact PIPE action

An entry in a pqact configuration-file that tells the associated pqact utility to pipe the data-product data from the matching data-product to a program that the pqact utility starts as a child process. Configuration-file entry.

pqact configuration-file

A file that tells the associated pqact utility how to dispose of data-products. Further information.

pqcat

A program that reads data-products from the product-queue and writes them to a file or to standard output. Further information.

pqcheck

A program that checks the writer-counter in the product-queue.

pqing

An LDM ingester that reads meteorological data in WMO or AFOS format from a file or serial port.

pqinsert

An LDM ingester that inserts files.

pqmon

A program that prints a summary of the state of the product-queue. Use in monitoring the product-queue.

pqsend

A program that sends selected data-products from the product-queue to an LDM using HIYA messages and then terminates.

pqsurf

A program that reads data-products that are composite WMO surface bulletins from the product-queue, splits them into individual bulletins, inserts them as data-products into a separate product-queue, and processes the individual bulletins similar to pqact. Deprecated.

pqutil

An interactive program that reads from or writes to the product-queue.

process

The address space and system resources in which a program executes on a UNIX^Ž system. Official description. Child process.

process group

A collection of processes that can be signaled as a group. Official description.

process group leader

The top-level process of a process group. Official description.

product creation-time

See data-product creation-time.

product identifier

See data-product identifier.

product metadata

See data-product metadata.

product origin

See data-product origin.

product-queue

The memory-mapped disk-file used by the LDM system to cache data-products. Typically, data-products are written to the product-queue by LDM ingesters and read from the product-queue by upstream LDM processes.

product signature

See data-product signature.

product size

See data-product size.

R

transfer-mode: One of two modes in which a downstream LDM can receive data-products. In the primary transfer-mode, data-products are received via asynchronous HEREIS messages. In the alternate transfer-mode, data-products are received via synchronous COMINGSOON and BLKDATA messages.
receiving LDM: See downstream LDM.
RECLASS: An RPC. reply that specifies a new data-product selection-criteria. It may be sent from an upstream LDM to a downstream LDM in response to a FEEDME request or from a downstream LDM 5 to an upstream LDM 5 in response to a synchronous LDM 5 HEREIS or COMINGSOON message.
regex utility: A program for validating ERE-s and for testing their efficiency.
REQUEST entry: An entry in the LDM configuration-file, ldmd.conf, that causes a request to be made to an upstream LDM for data-products. ldmd.conf webpage.
RPC: Remote Procedure Call. A system by which a function-call on one computer is turned into a function-invocation on another computer. An RPC webpage.
rpcgen utility: A program for generating RPC-using C code from an input file.
rpc.ldmd: The main program of the LDM system. An LDM 6 program. Behavior.
rtstats: A utility that monitors the product-queue and periodically sends summary statistics to a downstream LDM, which is, typically, at the Unidata Program Center. The utility is usually started by the top-level rpc.ldmd process due to an entry in the LDM configuration-file, ldmd.conf. IDD rtstats webpages.
runtime symbolic link: A symbolic link in the LDM user's home directory that references (i.e., points to) the version-specific LDM subdirectory that is the currently operational one. Referenced in preinstallation steps. Removal of. Creation of.

S

scour

A crontab-driven utility for deleting old data-files based on pathname patterns and retention-times.

sending LDM

See upstream LDM.

set-uid-root

If a program is set-uid-root, then it is owned by user "root" and the set-user-ID bit of the file is enabled.

set-user-ID

If a program's set-user-ID bit is on, then, when executed, the user-ID of the resulting process will be that of the owner of the program rather than that of the user who executed the program.

SIGCONT

The UNIX^Ž "continue" signal. This signal is used by LDM data-product ingesters and downstream LDMs to notify the LDM process group that a data-product has just been added to the product-queue.

SIGINT

The UNIX^Ž interrupt signal. The default action is to terminate a process. Official description.

signature, data-product

See data-product signature.

SIGTERM

The UNIX^Ž termination signal. The default action is to terminate a process. Official description.

SIGUSR2

The UNIX^Ž, user-defined signal #2. The canonical action taken by an LDM process upon receipt of this signal is to "rotate" the logging-level: notice -> information -> debug -> notice -> ...

size, data-product

See data-product size.

standard error

The diagnostic print-stream of a UNIX^Ž process.

standard input

The standard input-stream of a UNIX^Ž process.

standard output

The standard print-stream of a UNIX^Ž process.

su utility

A non-standard but common utility for becoming another user.

synchronous

Describes something that causes the initiator to wait until the thing is finished. For example, a synchronous RPC message causes the sender to wait for a reply.

syslog daemon

A configurable UNIX^Ž daemon that disposes of log messages sent to it by other processes.

system group

A group of UNIX^Ž system users. Users within a group often have write access to the same files and directories. Official description.

T

TCP: Transport Control Protocol. An low-level Internet protocol for reliably transferring a stream of bytes from one computer to another. TCP tutorial (including TCP State Transition Diagram). Original RFC 793.
time utility: A UNIX^Ž utility for timing a simple command. Official description.
top utility: A freely-available, 3rd-party UNIX^Ž utility for displaying system resources and utlization. GNU distribution.

U

Unidata Program Center (UPC): The UCAR office that coordinates the Unidata program. UPC homepage.
upstream filter: The data-product filter that an upstream LDM 6 uses to decide whether or not to send a data-product to a downstream LDM 6. The filter has two parts: 1) an "OK" filter that data-products must match in order to be sent; and 2) an optional "not" filter that data-products must not match in order to be sent. Such filters are specified in the ALLOW entries of the LDM configuration-file, ldmd.conf.
upstream host: The computer on which a upstream LDM is running.
upstream LDM: Typically, a forked, child process of rpc.ldmd that reads data-products from the product-queue and sends the data-products or the data-product metadata to a downstream LDM as a result of a FEEDME or NOTIFYME request. In atypical situations, this phrase can also refer to a pqsend process.
upstream LDM 5: An upstream LDM that uses the LDM 5 protocol. For backward compatibility, LDM 6 is capable of acting as an upstream LDM 5.
upstream LDM 6: An upstream LDM that uses the LDM 6 protocol. Behavior.
uptime utility: A freely-available, 3rd-party UNIX^Ž utility for displaying system CPU utlization. GNU distribution.
user-account: The account on a UNIX^Ž system for a user of the system. Official description.

W

writer-counter: A field in the product-queue whose value is the number of times the product-queue has been opened for writing without an intervening close. Further information.

Y

yacc utility: A UNIX^Ž utility for generating a gramatical parser. Official description.

				Unidata is a member of the UCAR Office of Programs, is managed by the University Corporation for Atmospheric Research, and is sponsored by the National Science Foundation.
				P.O. Box 3000 • Boulder, CO 80307-3000 USA • Tel: 303-497-8643 • Fax: 303-497-8690