LDM Reference Glossary and Index
This is a hyperlinked glossary and index of the terms, and
phrases used to explain the fundamentals of the Unidata LDM.
A
B
C
D
E
F
G
H
I
K
L
M
N
O
P
R
S
T
U
W
V
Y
A
- ACCEPT entry
- An entry in the LDM
configuration-file
ldmd.conf that allows an
upstream LDM to send
unrequested
data-products.
Decription of entry in
ldmd.conf.
- ALLOW entry
- An entry in the LDM
configuration-file
ldmd.conf that allows a
downstream LDM to connect
and indicates what
data-products it
is allowed to receive.
- anti-DOS feature
- A feature of the LDM that mitigates deliberate or accidental
denial-of-service attacks by ensuring non-overlapping feeds to every
downstream host. This feature is enabled by default but can be disabled
through the enable-anti-DOS
parameter of the LDM registry. See the
description of the behavior of an
upstream LDM process for details.
- asynchronous
- Describes something that does not
cause the initiator to wait until the thing is finished. For
example, an asynchronous RPC message does
not cause the sender to wait for a reply.
- autoshift
- Describes the capability of a
downstream LDM 6 to change the
transfer-mode
of its connection depending on its success-rate of inserting
data-products
into the
product-queue.
LDM 6.4 was the first version to have this capability.
B
- BLKDATA
- A message from an
upstream LDM
to a downstream LDM
containing
data-product data.
LDM 6 protocol message.
C
- child process
- A UNIX� process created by a
parent process.
Official description.
- CONDUIT
- A feedtype comprising a huge
volume of model output.
- creation-time, data-product
- See data-product
creation-time.
- crontab utility
- A UNIX� utility for scheduling the periodic
execution of commands.
Official description. See
suggested entries for the LDM
user's crontab(1) file.
- COMINGSOON
- An
RPC message from an
upstream LDM
to a downstream LDM
containing data-product metadata
asking if the
downstream LDM wants the
corresponding
data-product.
LDM 6 protocol message.
D
- data-product
- The fundamental quantum of data acted upon by an LDM. Structure of a data-product.
- data-product data
- The data portion of a data-product.
Further information.
- data-product creation-time
- The time when the data-product was created
by the relevant LDM
ingester.
- data-product identifier
- A character-string that uniquely identifies a data-product. Product
identifiers are created by LDM
ingesters using the policy of the administrative entity
responsible for the relevant feedtype.
- data-product latency
- The difference between when a
data-product is received
by a
downstream LDM and the
data-product's creation-time.
- data-product metadata
- The metadata portion of a data-product.
Further information.
- data-product origin
- A character-string that identifies the computer on which the
LDM
ingester
was running when it created the data-product. Typically,
this is the fully-qualified hostname of the computer.
- data-product selection-criteria
- The criteria used for selecting
data-products.
The criteria is composed of a start-time and a stop-time that must
bracket the data-product
creation-time of matching products, and a list of
pairs of feedtype and
data-product identifier
ERE-s.
- data-product signature
- An array of 16 unsigned bytes that is unique to a data-product. Signatures
are computed by LDM
ingesters
using the
MD5 checksum algorithm.
An MD5 checksum webpage.
- data-product size
- The number of bytes in the data section of a data-product. The value must
be less than or equal to 0xFFFFFFFF (4294967295).
- decoder
- A feedtype-specific program
that disposes of data-products
that it receives from the pqact
process.
Further information.
- deprecated
- This item or feature might go away in a future release.
Users are cautioned against using or depending on it.
- downstream host
- The computer on which a
downstream LDM is
running.
- downstream LDM
- Typically, a forked,
child process of ldmd that sends a
FEEDME request to an
upstream LDM and,
consequently, receives
data-products, which it
then writes into the product-queue.
In atypical situations, this phrase can also refer to a notifyme process.
- downstream LDM 5
- A downstream LDM that
uses the LDM 5 protocol.
For backward compatibility,
LDM 6 is capable of acting as a
downstream LDM 5.
- downstream LDM 6
- A downstream LDM that
uses the LDM 6 protocol.
Behavior.
E
- EXEC entry
- An entry in the LDM
configuration-file
ldmd.conf that tells the
top-level ldmd to
asynchronously execute a program as a
child process on startup.
- environment variable
- A generally mutable object with a name and a character-string value
that can be set by
a user via the shell or by a program. Copies of environment
variables are inherited by
child processes.
Official description.
- execvp() function
- A UNIX� function that executes a command-string.
Official description.
- ERE
- Extended Regular Expression. ERE-s provide a mechanism to select
specific strings from a set of character strings.
ERE webpage.
F
- FEEDME
- A message from a
downstream LDM
to an upstream LDM
requesting data-products.
LDM 6 protocol message.
- feedtype
- A class of data-products
coming from a common source (e.g., WMO, NPORT). Feedtypes may
be hierarchically arranged. A feedtype may be represented as an
integral value (e.g., 1) or as a name (e.g., PPS).
Feedtype webpage.
- fork
- A UNIX� system-call that duplicates the
invoking
parent process to create a new
child process.
Official description.
G
- gateway LDM
- A site's
LDM
that receives data from offsite or that sends data to an offsite
LDM
- gdbm database
- A database based on the
GNU dbm database routines.
- GEMPAK
- A UPC-supported package for
analyzing and displaying primarily meteorological data
(with emphasis on gridded model output).
GEMPAK homepage.
- gnuplot utility
- A freely-available, 3rd-party UNIX� utility for
graphing data.
GNU distribution.
- grep utility
- The UNIX� utility for searching text files.
Official description.
H
- HEREIS
- A message from an
upstream LDM
to a downstream LDM
that contains a data-product.
- HIYA
- An unsolicited message from an
upstream LDM
to a downstream LDM
specifying what data-products
are about to be sent.
LDM 6 protocol message.
I
- IDD
- The Unidata Internet Data Distribution system. The IDD system is a
network of LDMs and their
associated institutions that cooperate to distribute
meteorological and related data-products for research
and education under the auspices of the Unidata program.
IDD homepage.
- IDD rtstats webpages
- A set of webpages for displaying the statistics of
non-gateway LDMs on the Unidata
IDD that are reported via the
rtstats utility.
Top-level webpage.
Use in monitoring the LDM.
- identifier, data-product
- See data-product identifier.
- ingester
- A feedtype-specific program
that creates data-products and
inserts them into the product-queue. Further information.
- insertion-time
- The time at which a
data-product
was inserted into the local
product-queue.
- iostat utility
- A commonly-available UNIX� utility for
displaying system I/O statistics.
- IS_ALIVE
- A message from a
downstream LDM
to the top-level
ldmd process on
an upstream host
inquiring whether or not the relevant
upstream LDM process is
still running.
LDM 6 protocol message.
K
- kill utility
- The UNIX� utility for sending signals to processes.
Official description.
L
- LDM
- Local Data Manager. A system of software for the distribution
and processing of
data-products in near
real-time via the Internet.
LDM homepage.
- LDM, downstream
- See downstream LDM.
- LDM, receiving
- See downstream LDM.
- LDM, sending
- See upstream LDM.
- LDM, upstream
- See upstream LDM.
- LDM 4
- An LDM that uses the
LDM 4 protocol.
- LDM 4 protocol
- The LDM protocol used by version 4
of the LDM.
- LDM 5
- An LDM that uses the
LDM 5 protocol.
- LDM 5 protocol
- The LDM protocol used by version 5
of the LDM.
- LDM 6
- An LDM that uses the
LDM 6 protocol.
- LDM 6 protocol
- The LDM protocol used by version 6
of the LDM.
LDM 6 protocol webpage.
- LDM log file
- The file used to log diagnostic messages from
LDM programs.
Format and interpretation.
Rotation and purging.
- LDM protocol
- The set of RPC messages used by a version of the
LDM to
communicate with another LDM of the
same version.
- LDM registry
- An LDM-specific database of configuration parameters.
LDM registry webpage.
- LDM user
- The user on a computer system that owns the local LDM installation. Typically, the
user's name is "ldm". LDM user webpage.
Creating user-account for.
Becoming the LDM user.
- ldmadmin
- Perl script for managing an operational
LDM system.
Further information.
- ldmd
- The top-level LDM server.
- ldmping
- A utility that temporarily connects to an
upstream LDM to
test its availability.
- ldmd.conf
- The configuration-file for the top-level ldmd process.
ldmd.conf webpage.
- lex utility
- A UNIX� utility for generating a lexical scanner.
Official description.
- localn
- The facility used by the LDM for logging messages via the
syslog daemon.
By default, the local0 facility is used.
Configuring the
syslog.conf file.
Setting the facility via the
configure
script.
- logfile, LDM
- See LDM logfile.
M
- make utility
- A UNIX� utility for maintaining, updating, and
regenerating groups of programs.
Official description.
- man utility
- A UNIX� utility for displaying reference-documentation on
UNIX� utilities.
Official description.
User-support example.
- MANPATH
- The environment variable that
determines what directories are searched for
manual pages and in what order.
Setting in preinstallation step.
User-support reference.
The man(1) utility.
- maximum HEREIS size
- The threshold, in bytes, above which an
upstream LDM 6 uses
a
COMINGSOON/BLKDATA
message-pair to send a
data-product rather than a
single
HEREIS message.
- McIDAS
- A UPC-supported package for
analyzing and displaying primarily meteorological data
(with emphasis on satellite imagery).
McIDAS homepage.
- metadata, data-product
- See data-product metadata.
- minimum virtual residence
time
- The minimum, observed
virtual residence time
since this parameter was initialized (either by creation of the
product-queue or via the
pqutil utility.
If this interval is not greater than the maximum
data-product latency configuration
parameter, then a
downstream LDM
might not reject a duplicate incoming
data-product.
N
- netstat utility
- A utility for displaying network connections.
- NOAAPORT
- A satellite broadcast network run by NOAA to diseminate weather data to
field offices. The
LDM
can be
built
to ingest data-products from the NOAAPORT broadcast.
- NOTIFICATION
- A message from an
upstream LDM to an
downstream LDM
containing data-product metadata.
LDM 6 protocol message.
- NOTIFYME
- An RPC message from a
downstream LDM to an
upstream LDM
requesting data-product metadata.
LDM 6 protocol message.
- notifying upstream LDM 5
- An upstream LDM 5 that
sends NOTIFICATION
messages to a
downstream LDM 5
rather than data-products.
- notifyme utility
- A utility that makes a NOTIFYME request to an upstream LDM and,
optionally, writes the data-product metadata that it receives to an
LDM logfile.
- NTP
- The Network Time Protocol. This protocol is used to obtain time
information from a time-server.
See also ntpd.
- ntpd
- The Network Time Protocol Daemon. This freely-available program
ensures that a computer's system clock is accurate, which is one of
the prerequisites for correct operation
of the LDM.
ntp.org home page.
- ntpdate
- A utility that uses the
Network Time Protocol (NTP) to either check the accuracy of the system
clock or to set the system clock.
See also ntpd.
- NULLPROC
- A message used only to flush or verify a connection.
O
- origin, data-product
- See data-product origin.
P
- parent process
- The UNIX� process that created
a child process.
Official description.
- pathological regular-expression
- An extended regular-expression (ERE) that is
longer than two characters and has a ".*" prefix.
- pathname
- A character-string that identifies a file. Pathnames may be absolute
or relative to the current working directory.
Official description.
- perl utility
- A freely-available scripting utility.
Official webpage.
- plotMetrics utility
- An
LDM
utility for plotting time-series of performance metrics.
Uses the gnuplot utility.
- PIPE action
- See pqact PIPE action.
- portmapper
- A program that maps RPC program numbers and
versions to Internet port numbers. The utility rpcbind is
a portmapper.
- pqact
- A program that reads data-products from the product-queue and disposes
of them according to user-configurable instructions in a
pqact configuration-file.
Further information.
- pqact EXEC action
- An entry in a pqact
configuration-file that tells the associated
pqact utility
to execute the associated program as a
child process when a matching
data-product is
encountered.
Configuration-file entry.
- pqact PIPE action
- An entry in a pqact
configuration-file that tells the associated
pqact utility
to pipe the data-product
data from the matching
data-product
to a program that the pqact utility starts as a
child process.
Configuration-file entry.
- pqact configuration-file
- A file that tells the associated pqact utility how to dispose of data-products.
Further information.
- pqcat
- A program that reads data-products from the product-queue and writes
them to a file or to
standard output.
Further information.
- pqcheck
- A program that checks the
writer-counter
in the
product-queue.
- pqing
- An LDM
ingester
that reads meteorological data in WMO or AFOS format from a file
or serial port.
- pqinsert
- An LDM
ingester
that inserts files.
- pqmon
- A program that prints a summary of the state of the
product-queue.
Use in monitoring the product-queue.
- pqsend
- A program that sends selected data-products from the product-queue
to an LDM using HIYA messages and then terminates.
- pqsurf
- A program that reads
data-products
that are composite WMO surface bulletins from the
product-queue,
splits them into individual bulletins, inserts them as data-products into a
separate product-queue, and
processes the individual bulletins similar to pqact.
Deprecated.
- pqutil
- A utility for interacting with the product-queue.
- process
- The address space and system resources in which a program executes
on a UNIX� system.
Official description.
Child process.
- process group
- A collection of processes that can be signaled
as a group.
Official description.
- process group leader
- The top-level process of a
process group.
Official description.
- product creation-time
- See data-product
creation-time.
- product identifier
- See data-product identifier.
- product metadata
- See data-product metadata.
- product origin
- See data-product origin.
- product-queue
- The memory-mapped disk-file used
by the LDM system to
cache data-products.
Typically, data-products
are written to the product-queue by LDM
ingesters and read from the
product-queue by upstream
LDM processes.
- product signature
- See data-product
signature.
- product size
- See data-product size.
R
- receiving LDM
- See downstream LDM.
- RECLASS
- An
RPC.
reply that specifies a new
data-product selection-criteria.
It may be sent from an
upstream LDM
to a downstream LDM
in response to a
FEEDME
request or from a
downstream LDM 5
to an
upstream LDM 5
in response to a
synchronous
LDM 5
HEREIS
or
COMINGSOON
message.
- regex utility
- A program for validating
ERE-s and for testing their efficiency.
- registry
- See LDM registry.
- regutil
- A program for managing the contents of the
LDM registry. Execute the command
man regutil for more information.
- REQUEST entry
- An entry in the LDM configuration-file,
ldmd.conf,
that causes a request to be made to an
upstream LDM for
data-products.
ldmd.conf webpage.
- residence time
- The amount of time a data-product spends in the product-queue before being
deleted to make room for more products.
- RPC
- Remote Procedure Call. A system by which a function-call on one
computer is turned into a function-invocation on another computer.
The
LDM
uses a particular implementation called ONC RPC.
ONC RPC specification.
ONC RPC tutorial
.
- rpcgen utility
- A program for generating
RPC-using C code from an input file.
- ldmd
- The main program of the LDM
system. An LDM 6 program.
Behavior.
- rtstats
- A utility that monitors the product-queue and
periodically sends summary statistics to a downstream LDM, which
is, typically, at the Unidata Program
Center. The utility is usually started by the top-level
ldmd process
due to an entry in the LDM
configuration-file,
ldmd.conf.
IDD rtstats webpages.
- runtime symbolic link
- A symbolic link in the LDM user's
home directory that references (i.e., points to) the
version-specific LDM subdirectory that is the current operational
one.
Removal of.
Creation of.
S
- scour
- A
crontab-driven utility for deleting old data-files
based on
pathname patterns and retention-times.
- sending LDM
- See upstream LDM.
- sequence number
- An unsigned, 32-bit number sequentially assigned by the
ingester that created the
data-product. Ignored by almost
everything but existing due to tradition and history.
- set-uid-root
- If a program is set-uid-root, then it is owned by user
"root" and the set-user-ID
bit of the file is enabled.
- set-user-ID
- If a program's set-user-ID bit is on, then, when executed,
the user-ID of the resulting process will be that of the owner of the
program rather than that of the user who executed the program.
- SIGCONT
- The UNIX� "continue" signal.
This signal is used by LDM data-product
ingesters and
downstream LDMs
to notify the
LDM process group
that a data-product has just been added to the product-queue.
- SIGINT
- The UNIX� interrupt signal.
The default action is to terminate a process.
Official description.
- signature, data-product
- See data-product signature.
- SIGTERM
- The UNIX� termination signal.
The default action is to terminate a process.
Official description.
- SIGUSR2
- The UNIX�, user-defined signal #2.
The canonical action taken by an LDM process upon receipt of this signal
is to "rotate" the logging-level: notice -> information -> debug ->
notice -> ...
- size, data-product
- See data-product size.
- standard error
- The diagnostic print-stream of a UNIX� process.
- standard input
- The standard input-stream of a UNIX� process.
- standard output
- The standard print-stream of a UNIX� process.
- su utility
- A non-standard but common utility for becoming another user.
- synchronous
- Describes something that
causes the initiator to wait until the thing is finished. For
example, a synchronous RPC message causes the sender
to wait for a reply.
- syslog daemon
- A configurable UNIX� daemon that disposes of log
messages sent to it by other processes.
- system group
- A group of UNIX� system users.
Users within a group often have
write access to the same files and directories.
Official description.
T
- TCP
- Transport Control Protocol. An low-level Internet protocol for
reliably transferring a stream of bytes from one computer to another.
TCP tutorial (including TCP State Transition Diagram).
Original
RFC 793.
- time utility
- A UNIX� utility for timing a simple command.
Official description.
- top utility
- A freely-available, 3rd-party UNIX� utility for
displaying system resources and utlization.
GNU distribution.
- transfer-mode
- One of two modes in which a
downstream LDM
can receive
data-products.
In the primary transfer-mode,
data-products
are received via
asynchronous
HEREIS
messages.
In the alternate transfer-mode,
data-products
are received via
synchronous
COMINGSOON
and
BLKDATA
messages.
U
- uldbutil
- A utility for listing and managing the contents of the
upstream LDM database.
Execute man uldbutil for the manual-page.
- Unidata Program Center (UPC)
- The UCAR office that coordinates
the Unidata program.
UPC homepage.
- upstream filter
- The
data-product
filter that an
upstream LDM 6 uses
to decide whether or not to send a
data-product
to a
downstream LDM 6.
The filter has two parts: 1) an "OK" filter that
data-products
must match in order to be sent; and 2)
an optional "not" filter that
data-products
must not match in order to be sent.
Such filters are specified in the
ALLOW entries
of the LDM configuration-file,
ldmd.conf.
- upstream host
- The computer on which a
upstream LDM is
running.
- upstream LDM
- Typically, a forked,
child process of ldmd that reads
data-products from the product-queue and sends
the data-products or the
data-product metadata
to a downstream LDM
as a result of a FEEDME
or NOTIFYME request.
In atypical situations, this phrase can also refer to
a pqsend process.
- upstream LDM 5
- An upstream LDM that
uses the LDM 5 protocol.
For backward compatibility,
LDM 6 is capable of acting as an
upstream LDM 5.
- upstream LDM 6
- An upstream LDM that
uses the LDM 6 protocol.
Behavior.
- upstream LDM database
- A database that contains information on all
upstream LDMs running on the local host.
The database is transitory: it exists for each LDM session only. It can
be accessed via the uldbutil utility.
- uptime utility
- A freely-available, 3rd-party UNIX� utility for
displaying system CPU utlization.
GNU distribution.
- user-account
- The account on a UNIX� system for a user of the system.
Official description.
V
- virtual residence time
- The time difference between when a data-product is created and
when it is removed from the product-queue. For each product,
it is the sum of that product's
latency plus that product's
residence time.
- vmstat utility
- A utility for displaying system metrics.
W
- writer-counter
- A field in the
product-queue
whose value is the number of times the
product-queue
has been opened for writing without an intervening close.
Further information.
Y
- yacc utility
- A UNIX� utility for generating a gramatical parser.
Official description.