>From: "Karli Lopez (McIDAS)" <address@hidden> >Organization: University of Puerto Rico >Keywords: 200002010754.AAA03412 NIDS NOWRAD ADDE NIDS.CFG Karli, re: 'WSI<tab>...' line in pqact.conf >Oh ok, that's what i'd done (merge it into a single line) but I missed placing >the first tab before the "FILE" entry. >I had: >---------------- >WSI<tab>...<newline> >FILE<tab>... >---------------- >instead of: >---------------- >WSI<tab>...<newline> ><tab>FILE<tab>.... >---------------- >it is now "syntactically correct"! Excellent. >Now do I need the directories to be /data/.../directoryname/NIDS, or just /dat > a/.../directoryname where "/data/directoryname" is the directory where the >files are located (in out case it was set up orginally as "/unidata/ldm/data/m > cidas/") >my guess is that it's just the directory name and that NIDS is not >necessary, but I want to be sure. You are right. The location information in pqact.conf is where you want to put the data. This is up to you. The DIRMASK entry in the McIDAS ADDE configuration file for NIDS data, ~mcidas/workdata/NIDS.CFG, will need to be set to match whatever directory location you choose. The concept here is that the NIDS ADDE server needs to be able to find where the NIDS data files are and how they are named. The regular expression that is the value for the DIRMASK= parameter in NIDS.CFG describes where the files are located. The regular expression for the FILEMASK= parameter in NIDS.CFG describes how the files are named. Perhaps a little "graphical" example of how this would look might be useful. Remember, these are only examples; you have to setup your system according to the locations you choose for the data files: pqact.conf entry says: WSI ^NEX/(.*)/(.*)/([1-2][0-9])([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9]) FILE data/nexrad/\1/\2/\2_\3\4\5\6_\7\8 In this example, data/nexrad/... refers to the ~ldm/data/nexrad... directory. Remember that ~ldm/data is typically a link to some other directory. For the purpose of illustration, let us assume that ~ldm/data is a link to /data. The files will be stored in directories that look like: cd ~ldm/data/nexrad pwd /data/nexrad ls ABX CYS ENX FTG GRK OKX PUX As you can see, we are getting NIDS data for several sites: ABS, CYS, ENX, FTG, GRK, OKX, and PUX. These directory names are the NIDS station ID names. cd FTG ls BREF1 BREF4 CREF16 LREF3 PRE3 PRET SRMV1 TOPS VEL2 VIL BREF2 BRLR1 LREF1 PRE1 PRE3X PRETX SRMV2 VAD VEL3 BREF3 CREF LREF2 PRE1X PREA RESP STAT VEL1 VEL4 There is one directory for each of the NIDS product types: BREF1, BREF2, etc. cd BREF1 ls BREF1_20000318_1115 BREF1_20000318_1320 BREF1_20000318_1525 BREF1_20000318_1125 BREF1_20000318_1330 BREF1_20000318_1530 BREF1_20000318_1130 BREF1_20000318_1335 BREF1_20000318_1540 BREF1_20000318_1135 BREF1_20000318_1340 BREF1_20000318_1545 BREF1_20000318_1140 BREF1_20000318_1345 BREF1_20000318_1550 BREF1_20000318_1145 BREF1_20000318_1350 BREF1_20000318_1555 BREF1_20000318_1155 BREF1_20000318_1355 BREF1_20000318_1600 BREF1_20000318_1200 BREF1_20000318_1405 BREF1_20000318_1605 BREF1_20000318_1205 BREF1_20000318_1410 BREF1_20000318_1615 BREF1_20000318_1210 BREF1_20000318_1415 BREF1_20000318_1620 ... (these are the actual data files. They are named BREF_CCYYMMDD_HHMM where: CCYYMMDD_HHMM ^ ^ ^ ^ ^ ^__ minute of the hour | | | | |____ hour of the day | | | |_______ day of the month | | |_________ month of the year | |___________ year of the century |_____________ century BREF1 is the product type. In the way I setup the ADDE NIDS data set (in ~mcidas/data/DSSERVE.BAT), it is also part of the data set name: RTNIDS/BREF1 <- realtime NIDS Base Reflectivity at tilt 1 data RTNIDS/BREF2 <- realtime NIDS Base Reflectivity at tilt 2 data RTNIDS/BREF3 <- realtime NIDS Base Reflectivity at tilt 3 data RTNIDS/BREF4 <- realtime NIDS Base Reflectivity at tilt 4 data RTNIDS/LREF1 <- realtime NIDS Layer Reflectivity at level 1 data RTNIDS/LREF2 <- realtime NIDS Layer Reflectivity at level 2 data RTNIDS/LREF3 <- realtime NIDS Layer Reflectivity at level 3 data RTNIDS/CREF <- realtime NIDS Composite Reflectivity data and so on. The NIDS.CFG file will contain entries that look like: DIRMASK=/data/nexrad/\ID/\TYPE FILEMASK=\TYPE_* IDMASK=* The "replaceable" token '\ID' represents the value of the ID= keyword that will be passed by the IMGDISP command that will load images. The \TYPE "replaceable" is the NIDS product type (e.g., BREF1, BREF2, etc.). The "replaceable" concept is not part of regular expressions; it is a concept that I added to make setting up the configuration file easier. A full regular expression for a NIDS data file then becomes the concatenation of the DIRMASK and FILEMASK values: /data/nexrad/\ID/\TYPE/\TYPE_* A load request to the server that looks like: IMGDISP RTNIDS/BREF1 ID=FTG results in the regular expression being expanded to: /data/nexrad/FTG/BREF1/BREF1_* This will match all of the data files in /data/nexrad/FTG/BREF1 that begin BREF1_. Similarily, an image load that looks like: IMGDISP RTINDS/CREF ID=PUB will result in the regular expression: /data/nexrad/PUB/CREF/CREF_* This flexibility allows one to actually put all of the NIDS data files into a single directory (like for a case study). For instance, suppose that you had a directory named: /var/data/sanjuan/case1 that had all NIDS products for the San Juan, Puerto Rico NEXRAD (station ID JUA _I think_). A directory listing might look like: cd /var/data/sanjuan/case1 ls BREF1_20000212_1530 BREF1_20000212_1535 ... BREF2_20000212_1530 BREF2_20000212_1535 ... BREF3_20000212_1530 BREF3_20000212_1535 ... BREF4_20000212_1530 BREF4_20000212_1535 ... LREF1_20000212_1530 LREF1_20000212_1535 ... LREF2_20000212_1530 LREF2_20000212_1535 ... ... The DIRMASK= and FILEMASK= parameters in NIDS.CFG would look like: DIRMASK=/var/data/sanjuan/case1/ FILEMASK=\TYPE_* Notice that since there is only data for JUA in the directory that I do not need to use the \ID replaceable. Following on with the same kind of example, suppose I have all NIDS products for FTP (Denver, CO) and PUB (Pueblo, CO) in the same directory, /var/data/colorado. A directory listing might look like: cd /var/data/colorado ls FTG-BREF1_20000212_1530 FTG-BREF1_20000212_1535 ... FTG-BREF2_20000212_1530 FTG-BREF2_20000212_1535 ... FTG-BREF3_20000212_1530 FTG-BREF3_20000212_1535 ... FTG-BREF4_20000212_1530 FTG-BREF4_20000212_1535 ... FTG-LREF1_20000212_1530 FTG-LREF1_20000212_1535 ... FTG-LREF2_20000212_1530 FTG-LREF2_20000212_1535 ... PUB-BREF1_20000212_1530 PUB-BREF1_20000212_1535 ... PUB-BREF2_20000212_1530 PUB-BREF2_20000212_1535 ... PUB-BREF3_20000212_1530 PUB-BREF3_20000212_1535 ... PUB-BREF4_20000212_1530 PUB-BREF4_20000212_1535 ... PUB-LREF1_20000212_1530 PUB-LREF1_20000212_1535 ... PUB-LREF2_20000212_1530 PUB-LREF2_20000212_1535 ... ... The DIRMASK= and FILEMASK= parameters would look like: DIRMASK=/var/data/colorado/ FILEMASK=\ID-\TYPE_* As you can see, this setup can be pretty flexible. >So is McIDAS Area conversion turned off after i uncomment the original WSI >lines from pqact.conf? I want to have both things active. No. In order to turn off conversion to AREA files, you would have to comment out all of the WSI lines that call for the invocation of the nids2area decoder. >many Thanks, You are welcome. By the way. I have been meaning to talk with you about your ADDE setup. It appears that the executables 'compress' and 'uncompress' can not be found by the ADDE routines on your system. A quick look at our IRIX machines here at Unidata show: uname -a IRIX chevy 6.5 01200531 IP22 which compress uncompress /usr/ucb/compress /usr/ucb/uncompress If compress and uncompress are not located in /usr/ucb on your system, then the ADDE servers can not send/receive compressed data. Compressed ADDE data transfers are controlled by an environment variable named MCCOMPRESS that has to be set for each user that is going to run McIDAS-X. If MCCOMPRESS is not defined, it is assumed to mean that data is to be transferred in an uncompressed format. This may be the reason that your IMGDISP RTIMAGES/GE... invocations were so slow when your DATALOC for RTIMAGES was pointing back here to Unidata. MCCOMPRESS must be set _before_ the McIDAS-X session is started. Here is a sequence that will demonstrate how much transferring compressed data helps: <login as 'mcidas'> unsetenv MCCOMPRESS (assumes you are using the C shell; adjust the syntax if you are using the Bourne or Korn shells) cd workdata mcidas DATALOC ADD RTIMAGES ADDE.UNIDATA.UCAR.EDU IMGDISP RTIMAGES/GE-IR STA=TJSU EU=IMAGE SF=YES REFRESH='EG;MAP H 1' (try to get a feel for how long this took to complete) EXIT setenv MCCOMPRESS TRUE mcidas IMGDISP RTIMAGES/GE-IR STA=TJSU EU=IMAGE SF=YES REFRESH='EG;MAP H 1' (compare how long this took with the uncompressed transfer) DATALOC ADD RTIMAGES BREEZE.UPRM.EDU IMGDISP RTIMAGES/GE-IR STA=TJSU EU=IMAGE SF=YES REFRESH='EG;MAP H 1' (this invocation should fail until 'compress' and 'uncompress' and can be found in the environment in which the remote ADDE server on breeze runs. The remote ADDE server routine that looks for 'compress' and 'uncompress' searches the following directories (in this order): /bin /usr/ucb /usr/bin /usr/bsd If 'compress' and 'uncompress' are in different directories on your system, then you should make a link from where they are to the /usr/ucb directory (has to be done as 'root'). This should get things working. I hope that this email was not too long to digest. I try to explain things in detail since these email exchanges go into an inquiry tracking system that all users can search and, hopefully, learn from. Tom
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.