Hi Robert, First, I must apologize for not getting back to you on your other inquiries. Just so you know, it is workshop time here in Unidata, and I am, as usual, struggling to get the materials and machines ready for my session which starts on Monday. re: > I started a new support thread because this about the new MySQL GRIB server > specifically. OK. > I wanted to make sure I could use standard NOAAport data before ruling out > being able to use AMPS data this way. Good thinking. > I did the following: > > 1) Installed mysql4 from www.blastwave.org under Solaris 10 x86. This is > a binary install and installs in /opt/csw/mysql4. I started the database > using the svcadm method and also set passwords per the MySQL docs. I verified > with svcs -a that the database is running. OK, so far so good. > 2) Did a make clobber on my McIDAS install and went back and did: > make all -mysql=/opt/csw/mysql4 VENDOR=-vendor > then did a make install.all > I had no errors. Hmm... It might be the case that the -mysql flag value is not being used correctly in the 'makefile'. To verify if it is being used correctly, please check your ~mcidas/mcidas2006/src/makelog file for the compilation of routines used in GRIB processing. This will include, but may not be limited to, xcdgrib.c in the main McIDAS section, and all of the C routines in the XCD section. If the -mysql flag is not being used correctly, I will have to figure out what modification I need to make to have things work like they do in the SSEC makefile. > 3) Edited gribadmin script so that it matches my setup (bloastwave uses > Sun compilers to compile mysql, not gcc: > > ########################################################### > # This script is to be used for the administration of the # > # MySQL database used in conjunction with the GRIB filer. # > ########################################################### > > #========================================================= > # Configuration Section: > # > # The variables in this section may need to be modified > # to conform to your system. > > # Database and user names for MySQL calls > database_name="mcrtgrib" > database_user="gribread" > > # Path to the MySQL executable > mysql_path=/opt/csw/mysql4/bin > > # Path to gcc shared libraries > gcc_lib_path=/opt/SUNWspro/lib > > #========================================================= > > > # Add MySQL to the PATH > PATH=$PATH:$mysql_path > export PATH > > # Add path to gcc shared libararies > LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$gcc_lib_path:/opt/csw/mysql4/lib > export LD_LIBRARY_PATH > > > 4) From command line ran: > gribadmin makedb > It asked me for the password I setup the mysql user to have, and it > did create mrctgrib in my mysql db directory: > > drwx------ 5 mysql mysql 512 Oct 26 13:45 . > drwxr-xr-x 8 root bin 512 Oct 25 17:27 .. > -rw-rw---- 1 mysql mysql 5242880 Oct 25 17:28 ib_logfile0 > -rw-rw---- 1 mysql mysql 5242880 Oct 25 17:28 ib_logfile1 > -rw-rw---- 1 mysql mysql 10485760 Oct 25 17:28 ibdata1 > drwx------ 2 mysql mysql 512 Oct 26 13:45 mcrtgrib > -rw-r--r-- 1 mysql mysql 2468 Jul 29 15:50 my.cnf > drwx------ 2 mysql mysql 1536 Oct 25 17:27 mysql > -rw-rw---- 1 mysql mysql 6 Oct 25 17:28 mysql.pid > -rw-rw---- 1 mysql root 1019 Oct 25 17:28 psnwx.err > drwx------ 2 mysql mysql 512 Oct 25 17:27 test > # pwd > /opt/csw/mysql4/var OK, but the question remains if the McIDAS routines were actually comipiled with the mysql infomation. > 5) used DECINFO to set al inactive, except for DMBIN whihc is set to active. > > 6) Copied xcd_run to ~ldm/decoders and modified as needed. > > 7) Edited ldmd.conf to ass EXEC xcd_run MONITOR > > 8) Edited my pqact file as such: > > EXP|HDS .* > PIPE xcd_run HRS > > 9) Created directoty /data/ldm/mcidas/grib and made sure redirect was correct: > *.gr* /data/ldm/mcidas/grib > redirect.k: Done These steps look correct. > 9) I then stopped and restarted the LDM. > > 10) dmbin.k is running, and there are files in /data/ldm/mcidas/grib that are > put there by McIDAS: > > > ls > AWC.190.2006299.1300.1.255.grib NAM.84.2006299.1200.30.207.grib > AWC.190.2006299.1348.0.255.grib NAM.84.2006299.1200.30.211.grib > AWC.190.2006299.1353.0.255.grib NAM.84.2006299.1200.30.212.grib > AWC.190.2006299.1358.0.255.grib NAM.84.2006299.1200.30.215.grib > AWC.190.2006299.1403.0.255.grib NAM.84.2006299.1200.30.216.grib OK. This portion of the processing chain is separate from the actual use of the GRIB files by the server. > 11) I used DSSERVE to add entry for the NAM: > > DSSERVE ADD NAM/GRID212 GRIB TYPE=GRID DIR='/data/ldm/mcidas/grib/NAM*212*' > > 12) I ran GRDDISP and was able to display data from this set, however I > noticed access was slow, so I ran: > gribadmin latest > and this returned nothing. This tells me that the mysql info was _not_ passed to the routines that need it. I still would like you to verify this by reviewing your makelog entries. > 13) I checked XCD_START.LOG and indeed it said that MySQL was not being used: > > Starting MONITOR at 06299.135123 > processing from spool file HRS.SPL RTMODELS.CFG > DMBIN Starting: GRIB/BUFR filer > startxcd.k: m0monexe - first startup of :DMBIN > 15034 > DMBIN: > DMBIN: Invalid Processed pointer. > DMBIN: first POINTER= argument is too small --> -2139062144 > DMBIN: Must be valid integer value within range 4096 thru 16781312. > DMBIN: > DMBIN: GRBFILER.PRO not found. Processed pointer set to last byte written. > DMBIN: Not using MySQL database for GRIB > decoder error -3 message at 13096354 6299 135134 > decoder error -8 message at 1941391 6299 135257 > decoder error -8 message at 11449977 6299 135741 > decoder error -8 message at 899321 6299 140259 > > Obviously GRDDISP is just reading the GRIB directly without using the > database. Yes. >This won't work for me as it's too slow. OK. >Any ideas where I went wrong? The mysql stuff was most likely not passed in for the compilation step. > Obviously GEMPAK is superior to McIDAS for model data, but there are > some advantages to using McIDAS with model data and satellite imagery > together so OK. I can see benefits in having both datasets available in the same package. > if the MySQL method is fast enough I'd like to give it a try. OK. > Reading GRIB messages directly is not fast enough. I understand. My problem is that I do not have much if any free time at the moment to delve into what could be missing in the McIDAS makefile. As I understand things, you are heading off for Antarctica "soon" so your time is short. What I don't know is what sort of timeframe we are talking about. My workshop starts on Monday and ends on Thursday afternoon. After that, I will should have some time to dive into the makefile to see what is missing. The problem with that is that I am helping out with the LDM workshop on Friday and Saturday, and then I fly to Washington, DC for an AGU meeting on Nov. 6-7. Now, without going through all of the emails you sent in previously in detail, can you let me know if you have attempted to decode the AMPS data using the DMGRID method? I recall from a quick skim of one of your messages that one of the questions was where the AMPS model data would be decoded. Since I can't answer that off of the top of my head, I have to offer the following: If you setup McIDAS-XCD grid decoding (i.e., DMGRID) to process _only_ the AMPS data, then it should become obvious which file(s) the data is being decoded into _if_ they are decoded at all. Given the setup in RTMODELS.CFG, I would expect that the AMPS data would be decoded into the SCRATCH set, or GRID files 5001 - 5010, again _if_ they are decoded at all. So, I would setup a test system to: - only pipe the AMPS data into 'xcd_run HRS' in pqact.conf - make sure you are ingesting the AMPS data you want through an appropriate ldmd.conf request - make sure that the test system is not already looking at a number of GRID files into which the output could be put - startup XCD - monitory the log output in XCD_START.LOG - see if the AMPS data gets decoded A short comment here: if the DMGRID method does not work, then it means that the GRIB serving method won't work either. After all, it is the same code used to decode the GRIB 1 messages and file them into GRID files as it is to decode the data from GRIB 1 messages on the fly by the grib server. Now, if the AMPS data does get decoded correctly, then you should be able to use GRDLIST with the FORM=ALL keyword to list out the originating center, parameter ID, etc. so that you can setup an RTMODELS.CFG entry(ies) to file the data into a GRID file(s) of choice. The pieces that will be needed to be addressed at this point are: - REDIRECTion entries to point at your desired output GRID files - DSSERVE additons to create a dataset(s) to access the decoded data Of course, the downside of decoding using the DMGRID approach is the size of the output GRID files. Please let me know your thoughts at this point... Cheers, Tom **************************************************************************** Unidata User Support UCAR Unidata Program (303) 497-8642 P.O. Box 3000 address@hidden Boulder, CO 80307 ---------------------------------------------------------------------------- Unidata HomePage http://www.unidata.ucar.edu **************************************************************************** Ticket Details =================== Ticket ID: JIN-832252 Department: Support McIDAS Priority: Normal Status: Open
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.