>From: Gilbert Sebenste <address@hidden>
>Organization: NIU
>Keywords: 200205201534.g4KFYWa01146 McIDAS-XCD ldm-mcidas
Gilbert,
>Gilbert here. A couple of things.
>
>I turned off the HRS feed for LDM-McIDAS on weather2.admin.niu.edu. I
>didn't want to, but I guess a 40 GB hard drive just doesn't hack it.
While I will be the first one to admit that the McIDAS GRID files
created by XCD are large, I must say that if decoding and scouring is
working correctly, then you will use a maximum of 4 GB per day of GRID
data saved. I'm pretty user that I set up your scrouing to only keep
one day, so a 40 GB hard disk should have plenty of space.
Is something not working? What: decoding or scouring?
>ANyway, I have no problem with McIDAS upgrades, but I am scared of messing
>up the LDM-McIDAS upgrade. Is there any chance you could do that, or
>provide a detailed set of instructions on how to do it? Currently, I do
>not ingest the FNEXRAD feed, but want to become a relay for the new
>mosaics as well as make some for my web site.
The ldm-mcidas upgrades are easier than the McIDAS upgrades especially
since there are binary distributions for virtually all of the platforms
that folks are using out there. Even though I don't mind doing the
upgrade for you, I would rather walk you through the process so you
won't be wary of it in the future.
>As always, thanks for all your help. It is greatly appreciated!!!
No worries.
Here goes with the 1-2-3 for upgrading ldm-mcidas. All steps are
done by the user running the LDM on your system (typically 'ldm'):
1) login as 'ldm'
2) create the ldm-mcidas directory (it may already exist)
cd ~ldm
mkdir ldm-mcidas
cd ldm-mcidas
3) download the binary appropriate to your system (you would choose the
binary for linux_2.4-i686) to the ~ldm/ldm-mcidas directory:
o either use FTP to ftp.unidata.ucar.edu as user 'anonymous' and pass
'your email address'. Make sure to download in binary mode
- or -
o download the file from the ldm-mcidas webpage:
Unidata LDM-McIDAS
http://www.unidata.ucar.edu/packages/mcidas/mcidd/
Download Software
http://www.unidata.ucar.edu/packages/mcidas/mcidd/ldm-mcidas-download.html
4) unpack the ldm-mcidas distribution:
zcat ldm-mcidas-7.8.0.tar.Z | tar xvf -
5) copy the updated decoders to a directory in the PATH of 'ldm'. This is
typically ~ldm/decoders or ~ldm/util:
cd ldm-mcidas-7.8.0/bin
cp pnga2area pngg2gini nldn2md proftomd ~ldm/decoders
6) update your LDM pqact.conf file to decode the FNEXRAD imagery. In
the ldm-mcidas distribution, in the ldm-mcidas-7.8.0/etc directory,
you will find a file named ldm-mcidas-pqact.conf. This file contains
_example_ pqact.conf entries that you should copy to your pqact.conf
file ** making sure to keep tabs in appropriate places ** and then
edit -- if necessary.
Part of this process will be you deciding where on disk you want to
save the decoded images. This decision will vary greatly from site-to-site
since each site has different objectives, storage capacities, etc.
For you, since you are not running GEMPAK, I would recommend using a
top level directory structure something like:
~ldm/data/nexrad/NEXRCOMP
Since there are up to 6 different images in the FNEXRAD stream, you would
end up with 6 subdirectories of this directory. I recommend:
~ldm/data/nexrad/NEXRCOMP/6KN0R-NAT
~ldm/data/nexrad/NEXRCOMP/10KRCM-NAT
~ldm/data/nexrad/NEXRCOMP/1KN0R-FLT
~ldm/data/nexrad/NEXRCOMP/n0r
~ldm/data/nexrad/NEXRCOMP/n1p
~ldm/data/nexrad/NEXRCOMP/ntp
If you decide to use this directory structure, your pqact.conf entries
would end up looking like:
#########################################################################
#
# FNEXRAD section
#
#########################################################################
#
#### PNG compressed FNEXRAD image (AREA) files
#
# NEXRCOMP 6 km National BREF mosaic
FNEXRAD ^pnga2area Q5 (RL) (.*) (.*) (.*) (.*) (........) (....)
PIPE -close
pnga2area -vl logs/ldm-mcidas.log
data/nexrad/NEXRCOMP/6KN0R-NAT/\4_\6_\7
# NEXRCOMP 10 km National RCM mosaic
FNEXRAD ^pnga2area Q5 (RN) (.*) (.*) (.*) (.*) (........) (....)
PIPE -close
pnga2area -vl logs/ldm-mcidas.log
data/nexrad/NEXRCOMP/10KRCM-NAT/\4_\6_\7
# NEXRCOMP 1 km Regional BREF mosaic
FNEXRAD ^pnga2area Q5 (RO) (.*) (.*) (.*) (.*) (........) (....)
PIPE -close
pnga2area -vl logs/ldm-mcidas.log
data/nexrad/NEXRCOMP/1KN0R-FLT/\4_\6_\7
#
#### PNG compressed FNEXRAD image (GINI) files
# rad/NEXRCOMP/1km/n0r_20020520_1450
#
# NEXRAD Level III radar composites in PNG-compressed GINI format
FNEXRAD ^rad/NEXRCOMP/(...)/(...)_(........)_(....)
PIPE -close
pngg2gini -vl logs/ldm-mcidas.log
data/nexrad/NEXRCOMP/\2/\2_\3_\4
Mind the tabs!!!!!!
7) after making any change to pqact.conf, you should always verify its
integrity:
ldmadmin pqactcheck
8) if the 'ldmadmin pqactcheck' did not indicate errors in pqact.conf,
you can send a HUP to pqact telling it to reread pqact.conf:
ldmadmin pqactHUP
It is possible that the new GINI decoder, pngg2gini, may not be found
by the LDM processes (it should, but I have seen this happen). If
it happens to you, I would exit the LDM; do a rehash at the Unix
command prompt; verify that you can find the decoder; and restart your
LDM:
ldmadmin stop
<wait for all LDM processes to exit>
rehash
which pngg2gini
ldmadmin start
The next item of business is setting up scouring of the ingested images.
This is a crutial step given how large the images are after decoding
(14 MB per image for each 1 km N0R composite, and they arrive every
6 minutes!).
If you follow the setup above, then you can scour all of the new composites
with one cron entry. Since the scour technique included with the LDM
scours by number of days you want to keep, it is most likely that you
will want to do something else (240 x 14 MB == lots 'o disk space used :-).
We make available a Unix C-shell script that sours data files by number
instead of by time. You can grab this script, prune_nexrcomp.csh,
from the pub/ldm5/scour directory of anonymous FTP:
<as 'ldm' on your system>
cd decoders
ftp ftp.unidata.ucar.edu
<user> anonymous
<pass> your_full_email_address
cd pub/ldm5/scour
binary
get prune_nexrcomp.csh
quit
chmod +x prune_nexrcomp.csh
After transferring the file, you need to edit it to setup values
pertinent to your setup. In particular, you may need to modify the
script's internal settings for:
PATH
KEEP
areadir
For instance, if you put prune_nexrcomp.csh in ~ldm/decoders, and this
directory is actually /usr/local/ldm/decoders, your PATH setting would
become:
setenv PATH /usr/local/ldm/decoders:${PATH}
KEEP is the number of files you want to keep on disk. This is the number
that will be left each time the scouring routine runs from cron. It is
defaulted to 12, so you may want to change it.
'areadir' is the directory under which all subdirectories containing
files to be scoured can be found. In the example we are working on,
this would be /usr/local/ldm/data/nexrad/NEXRCOMP.
After you edit the entries in prune_nexrad.csh, you need to add an entry
to the crontab file for your user 'ldm'. How often you kick off this
scouring script is up to you, but we would recommend doing it no
less than once per hour.
The final step in bringing these data "to life" on your system is to
setup ADDE access to them. Why don't we tackle this one after you
have data ingestion, decoding, and scouring setup?
Tom
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.