GEMPAK decoders have standard look beginning with version 5.4. The supplied NAWIPS/GEMPAK decoders are called "DC"-style decoders. They can be used interactively from the command line or can be run in conjunction with the LDM software from Unidata. DC decoders will create a new file if one does not already exist.
Each DC-style decoder is run with the following syntax:
dcprogram [options] output_file
The available options are described below. Each individual decoder
has it's own set of defaults for these options. For more information
about these defaults, please see the help file for each program
(type "dcmetr -h" for example, at any system prompt), or see the
Decoders Chapter 6 in the GEMPAK Users Manual.
The output file name can be specified using a template. This allows better control of file name creation within the decoders, rather than relying on a name entirely created from the LDM. The template uses the date and time of the bulletin or report to replace the following characters:
YYYY Year with century YY Year without the century MM Month number DD Day HH Hour NN MinuteFor example, YYMMDD_sao.gem
-v N Set the verbosity level to "N". As N goes up, more verbose
logging occurs. The default for all decoders is "0".
-c curtim "curtim" is the pseudo-current time. This is a complete
GEMPAK format date/time (YYMMDD/HHMM) and is used as the
system time when decoding archive data. The default value
for this variable for all decoders is the system time.
-b nhours "nhours" is the number of hours prior to the current time.
Only reports within this time period are decoded into
the data file.
-d decoder_log "decoder_log" is the name and path of the decoder
log file.
-t time_out "time_out" is the time out value in seconds of the
decoder. If no data is received within the specified
time limit, the program will exit. The default time
out value for all decoders is 600 seconds.
-n The default behavior of the decoder is to store the raw
text reports in the decoded data file. Use this flag to
NOT store the raw data.
-h Print the usage statement and exit
-p prmfil Set the parameter packing table to "prmfil". See below
for a description of the GEMPAK parameter packing tables.
-s stntbl Set the station table to "stntbl". See below for a
description of the GEMPAK station tables.
-a iadstn Define the maximum number of additional stations (beyond
those in "prmfil") that can be added to the data file
as "iadstn".
-m maxtim Set the maximum number of times in a single data file
to "maxtim". The maximum times allowable in a single
surface file in NAWIPS 5.6 is 300.
-e NAME=setting Set an environmental variable "NAME" to the value
specified in "setting".
Certain decoder options are useful only with specific decoders.
Also, some decoders may provide additional flags.
Consult the individual decoder help file for which flags are valid
for each decoder.
To run the programs interactively, you must provide the raw data to be decoded as standard input. (That means use the "<" redirect in symbol to pipe the raw data into the decoder). You must also use the -c option to set the current time. Using the DCMETR decoder as an example:
dcmetr -c YYMMDD/HHNN -b 24 -d logfile YYMMDD_sao.gem < input_file
You can also configure the DC-style decoders to run automatically upon receipt of raw data from the LDM. To do this, you must configure the pattern action file "pqact.conf" that is included with the LDM software. Here is a complete set of sample pqact.conf with pattern actions for all the DC-style decoders.
The "DC"-style decoders provide the basis for converting standard products found in the FOS stream and other sources, such as the NCEP ftp server, and case study data sets. However, it is possible that you will have special data sets from your own field experiments, mesonets, or research projects. These data sets can be imported into GEMPAK for use with the plotting and analysis programs, but you will need to help GEMPAK by placing the data into a format it can recongnize. Alternatively, you may have archived data from standard reporting sites, however, the data may no longer be in its native format if someone decided they could store the data in a "better" format than it was reported in.
The GEMPAK "Edit" programs can be used to import non-standard data sets into GEMPAK, as well as to modify data values already in GEMPAK files. To modify exiting GEMPAK files, you will need to list the data out into ASCII format using one of the listing programs.
When using the GEMPAK edit programs to import data into GEMPAK file format, a surface data file must already exist; SNEDIT will create a file if one does not already exist.
The file creation programs define the number of times, stations, and parameters that will be stored in the file.
To create a surface file, use SFCFIL
To create an sounding data file use SNCFIL
To import research data sets with stations that aren't present in the
standard GEMPAK station tables, you will have to create your own station
tables as well! More information on importing
data will be covered later.
When creating surface and upperair files, the variables which will be stored in the file must be specified. GEMPAK uses the variables SFPRMF and SNPRMF for surface and upperair files respectively to specify the the parameters that will be stored. Parameters may either be listed on the variable declaration as a list delimited by semicolons, or a packing file may be used. Packing files define the parameter names, ranges, and resolution to be stored in the associated file.
The standard surface packing file is hrly.pack and looks like:
PMSL 950. 1049. .1 ALTI 24. 34. .01 TMPF -100. 150. 1. DWPF -100. 150. 1. SKNT 0. 126. 1. DRCT 0. 360. 10. GUST 0. 250. 1. WNUM 0. 512000. 1. CHC1 0. 18009. 1. CHC2 0. 8009. 1. CHC3 0. 8009. 1. VSBY 0. 51. .1 P03D 0. 8999. 1. P03I 0. 12.99 .01 P06I 0. 12.99 .01 HSUN 0. 999. 1. CSYL 0. 9. 1. CSYM 10. 19. 1. CSYH 20. 29. 1. SNOW 0. 1000.0 1. WEQS 0. 100.0 .1 T06X -100. 150. .1 T12X -100. 150. .1 T24X -100. 150. 1. TMDX -100. 150. 1. T06N -100. 150. .1 T12N -100. 150. .1 T24N -100. 150. 1. TMDN -100. 150. 1. P24I 0. 40. .01
The first column of the file is a four character identifier, the next three columns are the minimum, maximum, and resolution of the data respectively. Other packing files that are provided in $GEMTBL are snmerg.pack for sounding data, sfsyn.pack for synoptic data, sfship.pack for ship and buoy data, profiler.pack for profiler data, and nldn.pack for lightning data. Moreover, special packing files are available for particular data sets such as MOS bulletins, tropical store reports, watch box reports, and severe storm reports.
The ldmbridge program dcgrib is designed to handle GRIB products with or without product headers. Dcgrib will handle all of the models that are transmitted over the IDD, and most that are available from NCEP (except rotated and ensemble grids). The GEMPAK program nagrib is designed to be be used with GRIB bulletins without headers only. Nagrib will decode the new native staggered and ensemble grids available from NCEP.
Due to the large size and numbers of model grids, GRIB formatted data employs an efficient method for compressing the file size. DCGRIB supports a command line option PACK to allow decoded GEMPAK grid data to be written in its own compressed format. All the GEMPAK grid programs support the compressed format, so this option should be used if disk space is a consideration.
Another inovation in GRIB data is the quasi-thinned grids. Currently the global AVN (Spectral) model is transmitted using thinned grids. Thinned grids take advantage of the convergence of longitude lines as they approach the poles by eliminating certain gridpoints as the grid separation narrows. DCGRIB supports decoding of thinned grids, and provides a command line option for "thickening" the grids.
Ensemble grids may posess grid fields for a given variable in multiple instances for successive perturbations. NAGRIB will identify ensemble fields and annotate them accordingly.
Since NLDN data is being stored in surface file format, we are able to use the surface programs SFLIST and SFMAP to view the data.
Lightning data files tend to be much larger than surface
data files, in particular during active storms since each
lightning stroke is treated as a separate station.
Since lightning flashes are recorded down to the nanosecond,
typical decoder actions bin the data into more manageable
time ranges (generally 5 minute groupings) that work well
with other types of data (such as NIDS products).
