[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

19990428: ldm-mcidas and waldo (cont.)

>From: address@hidden
>Organization: St. Cloud State
>Keywords: 199904091934.NAA23884 ldm-mcidas

Alan,

Earlier today I wrote:

>Hold off on trying to install any ldm-mcidas stuff for right now.  I will,
>hopefully, have a usable binary for you sometime today.  At that point,
>I will FTP it over and install it under the home directory for 'ldm'.
>I will then adjust your .chsrc file so that the directory containing
>the decoders is in PATH (I will probably just copy the decoders to
>the ~ldm/decoders directory since that is already in PATH).  Lastly,
>I will setup McIDAS ROUTE PostProcess BATCH files so that your machine
>can produce the composite imagery you are undoubtedly used to seeing
>on your OS/2 boxes.  Of course, I will send you back a note telling
>you what I have done so you can review it.  After all, you will need
>to know what is going on so that you can keep things running in the
>future.

OK, I did what I promised :-).  Here is what I have done on waldo:

o FTPed over a binary distribution of ldm-mcidas built with gcc/f2c.
  I FTPed this to /usr/local/ldm, and I unpacked it into
  /usr/local/ldm/ldm-mcidas-7.4.0

o I copied the ldm-mcidas executables to /usr/local/ldm/decoders

o I changed the file permissions on the Bourne shell script, batch.k in
  /usr/local/ldm/decoders so that it was executable.  This is used
  in the McIDAS ROUTE PostProcess BATCH invocations.

o I edited this copy of batch.k and make sure that the definitions of
  McIDAS environment variables matched your setup.  This was trivial
  since you installed McIDAS in /home/mcidas; I did not have to make
  any changes because of this.

o I copied /home/mcidas/bin/xcd_run to /usr/local/ldm/decoders and
  changed its file permissions so that it is executable.  I then
  edited this file to set the McIDAS environment variables like
  in batch.k.  Like batch.k, I didn't have to make any mods since
  you installed McIDAS in /home/mcidas (yipee!)

o as the user 'ldm', I edited /usr/local/ldm/etc/pqact.conf and changed
  a couple of things:  I setup the ldm-mcidas decoders to write
  output surface and upper air MDXX files, GRID files, and TEXT files to
  /var/data/mcidasd, and I setup the image decoder and the profiler and
  NLDN lightning decoders to write to /var/data/mcidas.  (I know that
  this machine is not yet getting the NLDN lightning data, but that
  can come later.)

o I commented out the creation of FSL wind profiler MDXX files from
  the profiler data in the Unidata-Wisconsin datastream.  The reason
  I did this was that this data will disappear from the UW stream
  come July 1.  The FSL data is available in the FSL2 datastream
  that is being relayed through the IDD (it may not be flowing right
  at this minute since FSL is going through a big relocation).  We
  need to get this machine setup to get the FSL2 data and then decode
  it (hourly summary and 6-minute) for profiler stuff.

o I added commented lines to pqact.conf that will be needed to run the
  XCD decoders.  These will need to be uncommented once we feel
  comfortable with how the ldm-mcidas stuff is working (which will be
  soon, since ldm-mcidas appears to be running very smoothly).

o I added lines to pqact.conf to save frontal (analyzed and forecast)
  positions:

# fronts in ASUS1ddd for McIDAS
DDS     ^ASUS01 .... ([0-3][0-9])([0-2][0-9])
        FILE    data/surface/front/ASUS1%j

# forecast fronts in FSUS2ddd
DDS     ^FSUS02 .... ([0-3][0-9])([0-2][0-9])
        FILE    data/surface/front/FSUS2%j

o As the user 'ldm', I sent a HUP to pqact so it would reread pqact.conf
  and act on the changes:

ps -eaf | grep pqact
     ldm  8966  8963  0   Apr 27 ?        0:27 pqact
kill -HUP 8966

o I copied /home/mcidas/data/SCHEMA to /var/data/mcidas and made a hard
  link to it in /var/data/mcidasd:

cd /var/data/mcidasd
cp /home/mcidas/data/SCHEMA .
ln SCHEMA ../mcidas

  SCHEMA will be needed for the XCD decoders (that is why it was copied
  to /var/data/mcidas).

o I copied /home/mcidas/workdata/ROUTE.SYS to /var/data/mcidas and made
  a hard link to it in /var/data/mcidasd:

cd /var/data/mcidas
cp /home/mcidas/workdata/ROUTE.SYS .
ln ROUTE.SYS ../mcidasd

  ROUTE.SYS controls how many AREA files are produced by the ldm-mcidas
  decoder, lwtoa3, and it also specifies what ROUTE PP BATCH files are
  to be run upon receipt of data from the Unidata-Wisconsin datastream

o I copied /home/mcidas/data/SYSKEY.TAB to /var/data/mcidas and made
  a hard link to int in /var/data/mcidasd:

cd /var/data/mcidas
cp /home/mcidas/data/SYSKEY.TAB .
ln SYSKEY.TAB ../mcidasd

  SYSKEY.TAB is updated by both ldm-mcidas and XCD decoders (MD
  decoders, for XCD) and is used by the Unidata Fkey menu system and
  GUI for McIDAS-X.

o I setup McIDAS data file scouring by creating a cron file for 'mcidas'
  with the entry:

00 21 * * * /home/mcidas/workdata/mcscour.sh

  I edited /home/mcidas/workdata/mcscour.sh and set it up to keep 3 days
  of MD file data, etc.  You should take a look at this file.

  CAUTION:  do NOT setup LDM scouring for the McIDAS data directories!

o I finished off the McIDAS ADDE remote server installation.  To do this
  I simply followed the instructions available in the online McIDAS
  documentation.  Basically, as the user 'mcidas', this followed:

cd /home/mcidas/data
cp EXAMPLE.NAM LOCAL.NAM
<edit LOCAL.NAM and set REDIRECTion directories to match your machine setup>
cp DSSERVE.BAT LSSERVE.BAT
cp DATALOC.BAT LOCDATA.BAT
<edit LOCDATA.BAT and change 'fully_qualified_host_identifier' to
 waldo.stcloudstate.edu>
cd /home/mcidas/workdata
<start a McIDAS session and do>
REDIRECT REST LOCAL.NAM
TE XCDDATA "/var/data/mcidas
BATCH "LSSERVE.BAT
BATCH "LOCDATA.BAT
BATCH "XCDDEC.BAT
<XCDDEC.BAT suspends the products MA (surface MD data), RM (mandatory
 upper air data), and RS (significant level upper air data) from the
 routing table.  Since I am not yet ready to turn on XCD decoders
 (even though running XCDDEC.BAT was done in preparation to do this),
 I reenabled MA, RM, and RS decoding>
ROUTE REL MA RM RS
<I then enabled creation of composite and topographic composite imagery
 from the images in the Unidata-Wisconsin datastream>
ROUTE REL C
ROUTE REL N

(The products with product codes beginning with C are east and west
composites (IR, VIS, and H2O).  The products with product codes beginning
wtih N are satellite image and topography composites.)

I then became 'root' to enable the ADDE remote server (mods to /etc/services
and /etc/inetd.conf):

su -
<password>
cd /home/mcidas
sh ./mcinet7.5.sh install mcadde
exit

I verified that your remote ADDE server is working by doing a listing
from my machine here at Unidata of the image data that is located
on your machine.  The following are the commands I ran on our machine:

dataloc.k ADD RTIMAGES waldo.stcloudstate.edu
dsinfo.k IMAGE RTIMAGES

        Dataset Names of Type: IMAGE in Group: RTIMAGES

Name         NumPos   Content
------------ ------   --------------------------------------
ANTARCTIC       10    Antarctic IR Composite
EDFLOATER-I     10    Educational Floater
EDFLOATER-II    10    Educational Floater II
GE-IR           10    GOES-East North America IR
GE-IRTOPO       10    GOES-East IR/TOPO Composite
GE-VIS          10    GOES-East North America VIS
GE-VISTOPO      10    GOES-East VIS/TOPO Composite
GE-WV           10    GOES-East North America H2O
GEW-IR          10    GOES-East/West IR Composite
GEW-IRTOPO      10    GOES-East/West IR/TOPO Composite
GEW-VIS         10    GOES-East/West VIS Composite
GEW-VISTOPO     10    GOES-East/West VIS/TOPO Composite
GEW-WV          10    GOES-East/West H2O Composite
GW-IR           10    GOES-West Western US IR
GW-IRTOPO       10    GOES-West IR/TOPO Composite
GW-VIS          10    GOES-West Western US VIS
GW-VISTOPO      10    GOES-West VIS/TOPO Composite
GW-WV           10    GOES-West Western US H2O
MDR             10    Manually Digitized Radar
MDRTOPO         10    MDR/TOPO Composite
MOLL-IR         10    Mollweide Composite IR
MOLL-IRTOPO     10    Mollweide IR/TOPO Composite
MOLL-WV         10    Mollweide Composite H2O
RESFLOATER      10    Research Floater

DSINFO -- done

imglist.k RTIMAGES/GE-IR.ALL
Image file directory listing for:RTIMAGES/GE-IR
 Pos Satellite/         Date       Time      Center   Band(s)
     sensor                                 Lat  Lon
 --- -------------  ------------  --------  ---- ---- ------------
   1  G-8 IMG       28 APR 99118  20:15:00    23   71 4
   2  G-8 IMG       28 APR 99118  21:15:00    23   71 4
   3  G-8 IMG       28 APR 99118  18:15:00    23   71 4
   4  G-8 IMG       28 APR 99118  19:15:00    23   71 4
imglist.k: done

You can see from this listing that you had, at the time I ran this
command, ingested 4 GOES-8 Infrared images.  As time goes on, this
list will expand to 10 images since that is how the entry in the
routing table, /var/data/mcidas/ROUTE.SYS, is configured.

So, that is pretty much it!  You can verify that data is being ingested
and decoded by running ROUTE LIST from a McIDAS session (as user 'mcidas'):

ROUTE LIST                                                                     
S Pd         Description         Range       Last      Received  Post Process C
- -- ------------------------- --------- ------------ ---------- ------------ -
  CI GOES-E/W IR Composite       80-89   AREA0082     99118 2241     none     3
  CV GOES-E/W VIS Composite      90-99   AREA0092     99118 2244     none     3
  CW GOES-E/W H2O Composite      70-79   AREA0072     99118 2241     none     3
  LD NLDN Lightning Flashes      71-71       none        none        none     3
  MA Surface MD data            default  MDXX0008     99118 2235 SFC.BAT      3
  N1 GOES-East IR/TOPO Composi  220-229  AREA0222     99118 2241     none     3
  N2 GOES-East VIS/TOPO Compos  230-239  AREA0232     99118 2244     none     3
  N3 GOES-West IR/TOPO Composi  240-249  AREA0241     99118 2222     none     3
  N4 GOES-West VIS/TOPO Compos  250-259  AREA0251     99118 2224     none     3
  N5 MDR/TOPO Composite         260-269  AREA0261     99118 2208     none     3
  N6 Mollweide IR/TOPO Composi  270-279  AREA0271     99118 2235     none     3
  N7 GOES-E/W IR/TOPO Composit  280-289  AREA0282     99118 2241     none     3
  N8 GOES-E/W VIS/TOPO Composi  290-299  AREA0292     99118 2244     none     3
  NF Global Initialization Gri  101-102  GRID0101     99118 2242 GLOBAL.BAT   3
  NG Early Domestic Products      1-2        none     99118 1716 ADDGRID.BAT  3
  R1 Base Reflectivity Tilt 1   300-339      none        none        none     3
  R2 Base Reflectivity Tilt 2   340-379      none        none        none     3
  R3 Base Reflectivity Tilt 3   380-419      none        none        none     3
  R4 Base Reflectivity Tilt 4   420-459      none        none        none     3
  R5 Composite Reflectivity     460-499      none        none        none     3
  R6 Layer Reflect SFC-24 K ft  500-539      none        none        none     3
  R7 Layer Reflect 24-33 K ft   540-579      none        none        none     3
  R8 Layer Reflect 33-60 K ft   580-619      none        none        none     3
  R9 Echo Tops                  620-659      none        none        none     3
  RA Vertical Liquid H2O        660-699      none        none        none     3
  RB 1-hour Surface Rain Total  700-739      none        none        none     3
  RC 3-hour Surface Rain Total  740-779      none        none        none     3
  RD Storm Total Rainfall       780-819      none        none        none     3
  RE Radial Velocity Tilt 1     820-859      none        none        none     3
  RF Radial Velocity Tilt 2     860-899      none        none        none     3
  RG Radial Velocity Tilt 3     900-939      none        none        none     3
  RH Radial Velocity Tilt 4     940-979      none        none        none     3
  RI 248 nm Base Reflectivity   980-1019     none        none        none     3
  RJ Storm-Rel Mean Vel Tilt 1 1020-1059     none        none        none     3
  RK Storm-Rel Mean Vel Tilt 2 1060-1099     none        none        none     3
  RM Mandatory Upper Air MD da  default      none        none    MAN.BAT      3
  RS Significant Upper Air MD   default      none        none    SIG.BAT      3
  U1 Antarctic IR Composite     190-199  AREA0190     99118 2115     none     3
  U2 FSL2 hourly wind profiler  default      none        none        none     3
  U3 Manually Digitized Radar   200-209  AREA0201     99118 2208 MDR.BAT      3
  U5 GOES-West Western US IR    130-139  AREA0131     99118 2222 GW-IR.BAT    3
  U6 FSL2 6-minute Wind profil  default      none        none        none     3
  U9 GOES-West Western US VIS   120-129  AREA0122     99118 2224 GW-VIS.BAT   3
  UA Educational Floater I      160-169  AREA0162     99118 2236     none     3
  UB GOES-West Western US H2O   170-179  AREA0171     99118 2219 GW-WV.BAT    3
  UC Educational Floater II      60-69   AREA0062     99118 2237     none     3
  UI GOES-East North America I  150-159  AREA0152     99118 2241 GE-IR.BAT    3
  UR Research Floater           180-189      none        none        none     3
  US Undecoded SAO Data         default  UNIDATAS     99118 2236     none     1
  UV GOES-East North America V  140-149  AREA0142     99118 2244 GE-VIS.BAT   3
  UW GOES-East North America H  210-219  AREA0212     99118 2241 GE-WV.BAT    3
  UX Mollweide Composite IR     100-109  AREA0101     99118 2235 MOLL.BAT     3
  UY Mollweide Composite H2O    110-119  AREA0110     99118 2238     none     3
ROUTE: Done 

You can see from the listing that data is coming in and composites are
being created.  Right now, the REDIRECTions in place do not allow you
to look at the surface and upper air MD data being decoded from the
Unidata-Wisconsin datastream.  The REDIRECTions for these files are
set to look in /var/data/mcidas.  This is where XCD-produced MD files
will be stored.

I am currently of a mind to:

o let this setup run for a day or so to make sure that things are working
  smoothly (things seem to be going along nicely right now)

o if no problems occur, then I propose to startup XCD creation of MDXX
  and GRID files.  Before doing the GRID data, however, we need to decide
  what you want to decode (there is a LOT of data!!) and how much you
  want to keep online

I suggest that you logon as 'mcidas' and play around a bit.  Let me know
if you run into anything you think is strange/wrong/unusual.

Tom