LDM Basics Glossary and Index
This is a hyperlinked glossary and index of the terms, and
phrases used to explain the fundamentals of the Unidata LDM.
A
B
C
D
E
F
G
H
I
K
L
M
N
O
P
R
S
T
U
Y
- ACCEPT entry
- An entry in the LDM
configuration-file
ldmd.conf that allows an
upstream LDM to send
unrequested
data-products.
Decription of entry in
ldmd.conf.
- ALLOW entry
- An entry in the LDM
configuration-file
ldmd.conf that allows a
downstream LDM to connect
and indicates what
data-products it
is allowed to receive.
- asynchronous
- Describes something that does not
cause the initiator to wait until the thing is finished. For
example, an asynchronous RPC message does
not cause the sender to wait for a reply.
- BLKDATA
- A message from an
upstream LDM
to a downstream LDM
containing
data-product data.
LDM 6 protocol message.
- child process
- A UNIX® process created by a
parent process.
Official description.
- CONDUIT
- A feedtype comprising a huge
volume of model output.
- creation-time, data-product
- See data-product
creation-time.
- crontab utility
- A UNIX® utility for scheduling the periodic
execution of commands.
Official description.
- COMINGSOON
- An
RPC message from an
upstream LDM
to a downstream LDM
containing data-product metadata
asking if the
downstream LDM wants the
corresponding
data-product.
LDM 6 protocol message.
- data-product
- The fundamental quantum of data acted upon by an LDM. Structure of a data-product.
- data-product data
- The data portion of a data-product.
Further information.
- data-product creation-time
- The time when the data-product was created
by the relevant LDM
ingester.
- data-product identifier
- A character-string that uniquely identifies a data-product. Product
identifiers are created by LDM
ingesters using the policy of the administrative entity
responsible for the relevant feedtype.
- data-product latency
- The difference between when a
data-product is received
by a
downstream LDM and the
data-product's creation-time.
- data-product metadata
- The metadata portion of a data-product.
Further information.
- data-product origin
- A character-string that identifies the computer on which the
LDM
ingester
was running when it created the data-product. Typically,
this is the fully-qualified hostname of the computer.
- data-product selection-criteria
- The criteria used for selecting
data-products.
The criteria is composed of a start-time and a stop-time that must
bracket the data-product
creation-time of matching products, and a list of
pairs of feedtype and
data-product identifier
ERE-s.
- data-product signature
- An array of 16 unsigned bytes that is unique to a data-product. Signatures
are computed by LDM
ingesters
using the
MD5 checksum algorithm.
An MD5 checksum webpage.
- data-product size
- The number of bytes in the data section of a data-product. The value must
be less than or equal to 0xFFFFFFFF (4294967295).
- decoder
- A feedtype-specific program
that disposes of data-products
that it receives from the pqact
process.
Further information.
- deprecated
- This item or feature might go away in a future release.
Users are cautioned against using or depending on it.
- downstream host
- The computer on which a
downstream LDM is
running.
- downstream LDM
- Typically, a forked,
child process of rpc.ldmd that sends a
FEEDME request to an
upstream LDM and,
consequently, receives
data-products, which it
then writes into the product-queue.
In atypical situations, this phrase can also refer to a notifyme process.
- downstream LDM 5
- A downstream LDM that
uses the LDM 5 protocol.
For backward compatibility,
LDM 6 is capable of acting as a
downstream LDM 5.
- downstream LDM 6
- A downstream LDM that
uses the LDM 6 protocol.
Behavior.
- EXEC entry
- An entry in the LDM
configuration-file
ldmd.conf that tells the
top-level rpc.ldmd to
asynchronously execute a program as a
child process on startup.
- environment variable
- A generally mutable object with a name and a character-string value
that can be set by
a user via the shell or by a program. Copies of environment
variables are inherited by
child processes.
Official description.
- execvp() function
- A UNIX® function that executes a command-string.
Official description.
- ERE
- Extended Regular Expression. ERE-s provide a mechanism to select
specific strings from a set of character strings.
ERE webpage.
- FEEDME
- A message from a
downstream LDM
to an upstream LDM
requesting data-products.
LDM 6 protocol message.
- feedtype
- A class of data-products
coming from a common source (e.g., WMO, NPORT). Feedtypes may
be hierarchically arranged. A feedtype may be represented as an
integral value (e.g., 1) or as a name (e.g., PPS).
Feedtype webpage.
- fork
- A UNIX® system-call that duplicates the
invoking
parent process to create a new
child process.
Official description.
- gateway LDM
- A site's
LDM
that receives data from offsite or that sends data to an offsite
LDM
- gdbm database
- A database based on the
GNU dbm database routines.
- GEMPAK
- A UPC-supported package for
analyzing and displaying primarily meteorological data
(with emphasis on gridded model output).
GEMPAK homepage.
- grep utility
- The UNIX® utility for searching text files.
Official description.
- HEREIS
- A message from an
upstream LDM
to a downstream LDM
that contains a data-product.
- HIYA
- An unsolicited message from an
upstream LDM
to a downstream LDM
specifying what data-products
are about to be sent.
LDM 6 protocol message.
- IDD
- The Unidata Internet Data Distribution system. The IDD system is a
network of LDMs and their
associated institutions that cooperate to distribute
meteorological and related data-products for research
and education under the auspices of the Unidata program.
IDD homepage.
- IDD rtstats webpages
- A set of webpages for displaying the statistics of
non-gateway LDMs on the Unidata
IDD that are reported via the
rtstats utility.
Top-level webpage.
Use in monitoring the LDM.
- identifier, data-product
- See data-product identifier.
- ingester
- A feedtype-specific program
that creates data-products and
inserts them into the product-queue. Further information.
- iostat utility
- A commonly-available UNIX® utility for
displaying system I/O statistics.
- IS_ALIVE
- A message from a
downstream LDM
to the top-level
rpc.ldmd process on
an upstream host
inquiring whether or not the relevant
upstream LDM process is
still running.
LDM 6 protocol message.
- kill utility
- The UNIX® utility for sending signals to processes.
Official description.
- LDM
- Local Data Manager. A system of software for the distribution
and processing of
data-products in near
real-time via the Internet.
LDM homepage.
- LDM, downstream
- See downstream LDM.
- LDM logfile
- The file used to log diagnostic messages from
LDM programs.
Format and interpretation.
Rotation and purging.
- LDM, receiving
- See downstream LDM.
- LDM, sending
- See upstream LDM.
- LDM, upstream
- See upstream LDM.
- LDM 4
- An LDM that uses the
LDM 4 protocol.
- LDM 4 protocol
- The LDM protocol used by version 4
of the LDM.
- LDM 5
- An LDM that uses the
LDM 5 protocol.
- LDM 5 protocol
- The LDM protocol used by version 5
of the LDM.
- LDM 6
- An LDM that uses the
LDM 6 protocol.
- LDM 6 protocol
- The LDM protocol used by version 6
of the LDM.
LDM 6 protocol webpage.
- LDM protocol
- The set of RPC messages used by a version of the
LDM to
communicate with another LDM of the
same version.
- LDM user
- The user on a computer system that owns the local LDM installation. Typically, the
user's name is "ldm". LDM user webpage.
Creating user-account for.
Becoming the LDM user.
- ldmadmin
- Perl script for managing an operational
LDM system.
Further information.
- ldmping
- A utility that temporarily connects to an
upstream LDM to
test its availability.
- ldmd.conf
- The configuration-file for the top-level rpc.ldmd process.
ldmd.conf webpage.
- lex utility
- A UNIX® utility for generating a lexical scanner.
Official description.
- logfile, LDM
- See LDM logfile.
- make utility
- A UNIX® utility for maintaining, updating, and
regenerating groups of programs.
Official description.
- man utility
- A UNIX® utility for displaying reference-documentation on
UNIX® utilities.
Official description.
User-support example.
- MANPATH
- The environment variable that
determines what directories are searched for
manual pages and in what order.
Setting in preinstallation step.
User-support reference.
The man(1) utility.
- maximum HEREIS size
- The threshold, in bytes, above which an
upstream LDM 6 uses
a
COMINGSOON/BLKDATA
message-pair to send a
data-product rather than a
single
HEREIS message.
- McIDAS
- A UPC-supported package for
analyzing and displaying primarily meteorological data
(with emphasis on satellite imagery).
McIDAS homepage.
- metadata, data-product
- See data-product metadata.
- NOTIFICATION
- A message from an
upstream LDM to an
downstream LDM
containing data-product metadata.
LDM 6 protocol message.
- NOTIFYME
- An RPC message from a
downstream LDM to an
upstream LDM
requesting data-product metadata.
LDM 6 protocol message.
- notifyme utility
- Requests data-product metadata
from an
upstream LDM and,
optionally, writes
the metadata to an LDM
logfile.
- notifying upstream LDM 5
- An upstream LDM 5 that
sends NOTIFICATION
messages to a
downstream LDM 5
rather than data-products.
- notifyme
- A utility that makes a NOTIFYME request to an upstream LDM and,
optionally, writes the data-product metadata that it receives to an
LDM logfile.
- NULLPROC
- A message used only to flush or verify a connection.
- origin, data-product
- See data-product origin.
- parent process
- The UNIX® process that created
a child process.
Official description.
- pathname
- A character-string that identifies a file. Pathnames may be absolute
or relative to the current working directory.
Official description.
- perl utility
- A freely-available scripting utility.
Official webpage.
- PIPE action
- See pqact PIPE action.
- portmapper
- A program that maps RPC program numbers and
versions to Internet port numbers. The utility rpcbind is
a portmapper.
- pqact
- A program that reads data-products from the product-queue and disposes
of them according to user-configurable instructions in a
pqact configuration-file.
Further information.
- pqact EXEC action
- An entry in a pqact
configuration-file that tells the associated
pqact utility
to execute the associated program as a
child process when a matching
data-product is
encountered.
Configuration-file entry.
- pqact PIPE action
- An entry in a pqact
configuration-file that tells the associated
pqact utility
to pipe the data-product
data from the matching
data-product
to a program that the pqact utility starts as a
child process.
Configuration-file entry.
- pqact configuration-file
- A file that tells the associated pqact utility how to dispose of data-products.
Further information.
- pqbinstats
- A program that periodically saves LDM execution statistics in
logs/*.stats files for initializing the
rtstats utility.
If not purged by the
scour program, the
number of these files will grow without bound.
- pqcat
- A program that reads data-products from the product-queue and writes
them to a file or to
standard output.
Further information.
- pqing
- An LDM
ingester
that reads meteorological data in WMO or AFOS format from a file
or serial port.
- pqinsert
- An LDM
ingester
that inserts files.
- pqmon
- A program that prints a summary of the state of the
product-queue.
Use in monitoring the product-queue.
- pqsend
- A program that sends selected data-products from the product-queue
to an LDM using HIYA messages and then terminates.
- pqsurf
- A program that reads
data-products
that are composite WMO surface bulletins from the
product-queue,
splits them into individual bulletins, inserts them as data-products into a
separate product-queue, and
processes the individual bulletins similar to pqact.
Deprecated.
- pqutil
- An interactive program that reads from or writes to the product-queue.
- process
- The address space and system resources in which a program executes
on a UNIX® system.
Official description.
Child process.
- process group
- A collection of processes that can be signaled
as a group.
Official description.
- process group leader
- The top-level process of a
process group.
Official description.
- product creation-time
- See data-product
creation-time.
- product identifier
- See data-product identifier.
- product metadata
- See data-product metadata.
- product origin
- See data-product origin.
- product-queue
- The memory-mapped disk-file used
by the LDM system to
cache data-products.
Typically, data-products
are written to the product-queue by LDM
ingesters and read from the
product-queue by upstream
LDM processes. Product-queue webpage.
pq_size parameter in
ldmadmin script.
pqmon utility.
Creation of.
Paper on implementation details.
- product signature
- See data-product
signature.
- product size
- See data-product size.
- receiving LDM
- See downstream LDM.
- regex utility
- A program for validating
ERE-s and for testing their efficiency.
- REQUEST entry
- An entry in the LDM configuration-file,
ldmd.conf,
that causes a request to be made to an
upstream LDM for
data-products.
ldmd.conf webpage.
- RPC
- Remote Procedure Call. A system by which a function-call on one
computer is turned into a function-invocation on another computer.
An RPC webpage.
- RPC portmapper daemon
- A daemon to which RPC-using programs
connect in order to register their Internet ports or to determine which
Internet ports are used by a particular program.
- rpc.ldmd
- The main program of the LDM
system. An LDM 6 program.
Behavior.
- rtstats
- A utility that monitors the product-queue and
periodically sends summary statistics to a downstream LDM, which
is, typically, at the Unidata Program
Center. The utility is usually started by the top-level
rpc.ldmd process
due to an entry in the LDM
configuration-file,
ldmd.conf.
IDD rtstats webpages.
- runtime symbolic link
- A symbolic link in the LDM user's
home directory that references (i.e., points to) the
version-specific LDM subdirectory that is the currently operational
one.
Referenced in
preinstallation steps.
Removal of.
Creation of.
- scour
- A
crontab-driven utility for deleting old data-files
based on
pathname patterns and retention-times.
- sending LDM
- See upstream LDM.
- set-uid-root
- If a program is set-uid-root, then it is owned by user
"root" and the set-user-ID
bit of the file is enabled.
- set-user-ID
- If a program's set-user-ID bit is on, then, when executed,
the user-ID of the resulting process will be that of the owner of the
program rather than that of the user who executed the program.
- SIGCONT
- The UNIX® "continue" signal.
This signal is used by LDM data-product
ingesters and
downstream LDMs
to notify the
LDM process group
that a data-product has just been added to the product-queue.
- SIGINT
- The UNIX® interrupt signal.
The default action is to terminate a process.
Official description.
- signature, data-product
- See data-product signature.
- SIGTERM
- The UNIX® termination signal.
The default action is to terminate a process.
Official description.
- SIGUSR2
- The UNIX®, user-defined signal #2.
The canonical action taken by an LDM process upon receipt of this signal
is to "rotate" the logging-level: notice -> information -> debug ->
notice -> ...
- size, data-product
- See data-product size.
- standard error
- The diagnostic print-stream of a UNIX® process.
- standard input
- The standard input-stream of a UNIX® process.
- standard output
- The standard print-stream of a UNIX® process.
- su utility
- A non-standard but common utility for becoming another user.
- synchronous
- Describes something that
causes the initiator to wait until the thing is finished. For
example, a synchronous RPC message causes the sender
to wait for a reply.
- syslog daemon
- A configurable UNIX® daemon that disposes of log
messages sent to it by other processes.
- system group
- A group of UNIX® system users.
Users within a group often have
write access to the same files and directories.
Official description.
- TCP
- Transport Control Protocol. An low-level Internet protocol for
reliably transferring a stream of bytes from one computer to another.
TCP tutorial (including TCP State Transition Diagram).
Original
RFC 793.
- time utility
- A UNIX® utility for timing a simple command.
Official description.
- top utility
- A freely-available, 3rd-party UNIX® utility for
displaying system resources and utlization.
GNU distribution.
- Unidata Program Center (UPC)
- The UCAR office that coordinates
the Unidata program.
UPC homepage.
- upstream host
- The computer on which a
upstream LDM is
running.
- upstream LDM
- Typically, a forked,
child process of rpc.ldmd that reads
data-products from the product-queue and sends
the data-products or the
data-product metadata
to a downstream LDM
as a result of a FEEDME
or NOTIFYME request.
In atypical situations, this phrase can also refer to
a pqsend process.
- upstream LDM 5
- An upstream LDM that
uses the LDM 5 protocol.
For backward compatibility,
LDM 6 is capable of acting as an
upstream LDM 5.
- upstream LDM 6
- An upstream LDM that
uses the LDM 6 protocol.
Behavior.
- uptime utility
- A freely-available, 3rd-party UNIX® utility for
displaying system CPU utlization.
GNU distribution.
- user-account
- The account on a UNIX® system for a user of the system.
Official description.
- yacc utility
- A UNIX® utility for generating a gramatical parser.
Official description.
This page was updated on