>From: Anthony James Wimmers <address@hidden> >Organization: UVa >Keywords: 200004091534.JAA08082 XCD DMGRID ETA MRF FTP backup RTMODELS.CFG >NOGRIB.CFG GRIBDEC.CFG HRS.CFG RESOLV.SRV DECINFO Tony, >As Dr. Frankenstein said, "It works!" Here's to hoping that it >keeps going. Oh ye of little faith :-) >Thanks again for all the help, and the education. You are welcome, and the education was on both of our ends. The "fix" was not as simple as you may think. This message is intended to bring you up to date on and review exactly what was done on your machine to get things working correctly. (Note that all modified files are in a new addendum that I put together on April 13). It is written so that others looking through our tracking system can (hopefully) use the information to their benefit: o modify RTGRIDS.CFG: o move decoded NGM grids start from 5051 to 5501; this was done in order to make room in the GRID namespace for the ETA > H+48 grids The lines from RTMODELS.CFG that demonstrate this are: ETA= 1 5011 120000 240000 480000 (see next comment) NGM= 3 5501 120000 240000 480000 NOTE: if one wanted to file each ETA run into its own set of GRID files (i.e., 0Z run in one set of files; 6 Z run in a different set of files; etc.), then it is advised to move the ETA grids to a new set of GRID numbers and leave the NGM grids where they are. o change how ETA grids are stored: change: ETA= 3 5011 120000 240000 480000 to: ETA= 1 5011 120000 240000 480000 This says to store all grids with forecast times > 48 into a set of grid files (that did not previously exist) o change decoding of MRF to MRF and AVN; this change required mods to not only RTMODELS.CFG, but also to gbtbpds001.av1. See the section on gbtbpds001.av1 below for details. o modify the ADDE dataset definitions to match the new GRID file numbers for ETA, NGM, and ALL (ALL is all grids, so it also needs modifying). The quickest way to do this is to edit the ADDE dataset definition file, RESOLV.SRV (this would have to be done for each user that has datasets defined for him/herselves. Sites using the remote ADDE server technique will only need to modify the copy in ~mcidas/workdata (~mcidas/uvaworkdata in your case)). The modified lines from RESOLV.SRV will look like: N1=RTGRIDS,N2=ALL,TYPE=GRID,RT=Y,K=GRID,R1=5001,R2=5540,C=Real-Time Grids, N1=RTGRIDS,N2=ETA,TYPE=GRID,RT=Y,K=GRID,R1=5011,R2=5070,C=Real-Time ETA Grids, N1=RTGRIDS,N2=NGM,TYPE=GRID,RT=Y,K=GRID,R1=5501,R2=5540,C=Real-Time NGM Grids, o update gbtbpds001.av1: A modified version of gbtbpds001.av1 has been in a McIDAS-X 7.6 addendum since Feb 12, 2000, but it was not installed in your active McIDAS-X working directory, uvaworkdata. In addition to previous changes in gbtbpds001.av1 was yet another change in the model ID for ECMWF grids (now referenced as 195 instead of 194). This model number change has been semi-continuous for at least a couple of years. SSEC came up with a way to get around this: they provide two new 'gbtb' tables which I uploaded and installed on your system. More on this in the next section. For reference, the lines of meaning in this context are: 068 | 80 Wave triangular, 18-layer spectral model from aviation run | AVN 069 | 80 Wave triangular, 18-layer spectral model from MRF run | MRF 077 | 126 wave triangular, 18-layer Spectral Model from aviation run | AVN 078 | 126 wave triangular, 18-layer Spectral Model from MRF run | MRF 080 | 62 wave triangular, 18-layer Spectral Model from MRF run | MRF 081 | Spectral Statistical Interpolation from aviation run | AVN 082 | Spectral Statistical Interpolation from final run | MRF 083 | ETA Model - 80 km version | ETA 084 | ETA Model - 40 km version | ETA 085 | ETA Model - 30 km version | ETA 089 | ETA Model - 48 km version | ETA 090 | 62 Wave triangular, 28 layer 'Medium Range Forecast' | MRF 091 | 62 Wave triangular, 28 layer 'Aviation' | AVN 092 | 62 Wave triangular, 28 layer 'Medium Range Forecast' final | MRF 093 | 62 Wave triangular, 28 layer 'GDAS' | MRF 110 | ETA Model - 15 km version | ETA 195 | European Center For Medium Range Weather Forecasting Model | ECMF o install two new tables used by dmgrid.k: gbtbpds001.a74v1 and gbtbpds001.a98v1. This gets around having to ever modify gbtbpds001.av1 for ECMWF model ID changes (yahoo!). o modify GRIBDEC.CFG: change MAXGRD=2000 to MAXGRD=5000. Since there are more grids for the various models, the GRID file needed to have room to store more. This is especially true since the 6 Z and 18 Z ETA runs are, by default, stored in the 0 Z and 12 Z ETA run files, respectively. The effect of increasing MAXGRD= is to increase the GRID file header size. o modify NOGRIB.CFG: change '89' and '85' both to '84'. The PDS octets 6 part of ETA grib messages were all changed to be '84' on about March 29, 2000. NOGRIB.CFG needed to be updated to reflect this change. o install code modifications for routines used in dmgrid.k (_THIS_ was the real fix!): grib.h Mcgribdecoder.c The following is part of an alert I received from SSEC the same day I was talking to you by phone. I had a chance to really go over their update after we hung up: "During the week of 2 April 2000, the National Weather Service began transmitting what appears to be a radar product in GRIB format on the NOAAPORT broadcast. The product contains approximately 165,000 points and therefore exceeds an array size in the GRIB decoder. This results in an array overflow error when the GRIB decoder attempts to decode the product. The exact behavior of the error is platform dependent, but resulted in segmentation violations on both our Sparc system running Solaris 7 and our PC system running Solaris Intel 7. The error may result in groups of needed grids missing from your grid files. This addendum corrects the overflow error and instructs the GRIB decoder to not decode these radar products since it is unable to due to navigation issues." NOTE the comment "The error may result in groups of needed grids missing from your grid files." This was, in fact, exactly what you were seeing on your machine (a Sun Solaris 7 box). We were not seeing the problem on our Solaris x86 box, but was probably just pure luck! The installation of the new code required (everything done as the user 'mcidas'): o FTP the new modules, grib.h and Mcgribdecoder.c, to your ~mcidas/mcidas7.6/src directory o stop the McIDAS-XCD grid data monitor: cd ~mcidas/uvaworkdata ( for other sites this would be ~mcidas/workdata) decinfo.k SET DMGRID INACTIVE o "touching" Mcmkmcgrid.c cd ~mcidas/mcidas7.6/src touch Mcmkmcgrid.c o making a new executable for the grid data monitor, dmgrid.k: make dmgrid.k o installing a new dmgrid.k: rm ~/bin/dmgrid.k ln dmgrid.k ~/bin o restarting the grid data monitor: cd ~/uvaworkdata ( for other sites this would be ~mcidas/workdata) decinfo.k SET DMGRID ACTIVE By the way, the new, large product is a 10 km national radar summary created from Radar Coded Messages (RCM) that are generated at NEXRAD PUPs. McIDAS does not _yet_ decode these into anything useful, but it _will_ sometime in the future. Also, the following inquiry by Bryan Batson of the Johnson Space Flight Center (JSFC) to SSEC makes me feel that the effort of changing the size of the grid spool file, HRS.SPL, from 16 to 128 MB was not wasted effort: "Since reconfiguring our XCD system to accommodate the new ETA data, I've noticed a problem in that DMGRID does not seem to be processing all the grids it should be. My initial thinking on this is that the HRS.SPL file is wrapping before DMGRID can catch up (also, my XCD system running near 100% CPU load..with HP 755/99 MHZ system). I've told Chad about this and he thinks this may be the case, also. I will probably be increasing the size of HRS.SPL to see if that helps." I don't think that windfall was necessarily having a problem as extreme as Bryan's, but his comment and proposed solution sound mighty suspicous. I believe that increasing HRS.SPL from 16 to 32 MB may have been all that was necessary. Bumping it up to 128 MB was most likely massive overkill. If you feel like it, and if you want to recoop some of your disk space, then I would recommend stepping the size of your HRS.SPL file back to 32 MB. For reference for others, this is done as follows: o stop the LDM (done as user running the LDM): ldmadmin stop o change the size of HRS.SPL in the HRS.CFG file (done as 'mcidas'): cd ~mcidas/uvaworkdata (for most sites this would be ~mcidas/workdata) change: SPOOLENG=16 # size of the spool file to use in megabytes to: SPOOLENG=32 # size of the spool file to use in megabytes o delete HRS.SPL and GRIBDEC.PRO (done as user 'mcidas'): cd ~mcidas/uvaworkdata (for most sites this will be ~mcidas/workdata) lwu.k DEL HRS.SPL lwu.k DEL GRIBDEC.PRO o restart the LDM (done as user running the LDM): ldmadmin start HRS.SPL and GRIBDEC.PRO will be recreated by DMGRID after grid products start coming in again. Simple fix, heh ;-) Please let me know if you see any irregularities on your system after these changes. Tom
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.