[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[ldmMcidas #PUC-404507]: MD storage limits

Samuel,

re:
> You're comment about the 'MySQL' "stuff" not being in the "standard" location 
> motivated
> me to search Unidata's website to re-visit the McIDAS install instructions, 
> focusing on
> how to setup MySQL.  After sifting thru the McIDAS install instructions, and 
> e-mail, I
> stumbled upon an e-mail you wrote in 2007 instructing a user to perform an 
> install of MySQL
> Server(yum install mysql-server), client, and devel.  I must inform you that 
> I did no such
> thing to get MySQL working.  I downloaded the latest version of MySQL from 
> mysql.com, and
> installed it into /usr/local.

OK.  This is what led to the "non-standard" (aka user-added) installation.

> I think I ran into a similar problem with TCL/TK(and other RedHat packages), 
> when I was
> originally trying to install McIDAS on Shu.

You should not have run into any problem with Tcl/Tk as Tcl/Tk is bundled with 
McIDAS.
If it was not installed with the OS, then one could always alter the Tcl-based 
scouring
scripts scourBYnumber and scourBYday to use the tclsh binary created from the 
McIDAS
build.

> Unidata's online documentation for installing McIDAS never mentions 'yum', 
> and I had
> always used RedHat's package manager(rpm) to install such products.

'yum' is the standard utility for installing RPM-based packages on a variety of 
Linux
distributions including RedHat variants (e.g., RedHat Enterprise, Fedora, 
CentOS, etc.). 
The nice thing about 'yum' is that it takes care of dependency resolutions for 
you.  You
could have used 'yum' to install the MySQL pieces (client, server, and 
development)
from network repositories as easily as:

<as 'root'>
yum install mysql-server
yum install mysql-client
yum install mysql-devel

> I think we could have avoided lots of trouble if the install instructions for 
> 3rd party
> products would be explicitly included in McIDAS's installation documentation, 
> and not
> implicitly included in historical support e-mail.

Three things:

- it is literally impossible for the installation documents to include the 
information
  on how to install 3rd party packages since the way they are installed is 
dependent
  on the operating system being used.  For instance, the number of Linux 
distributions
  being used in the Unidata community is approaching 10, and a number of them 
use
  different approaches to package management.  Also supported are Sun Solaris 
(SPARC
  and x86), SGI IRIX, IBM AIX, and FreeBSD.

- the shell-definition file examples have instructions for the need to set
  MySQL_ROOT when the MySQL installation is in a non-standard location.  This
  value was not set when I first logged on.

- I think you missed one of the points I was trying to make: it is not yet clear
  to me why the McIDAS link process is _not_ using the static MySQL library 
since
  it is found in /usr/local/mysql/lib.  I assume that shared libraries are used 
in
  preference to static ones.  The problem is that the man page for 'ld' seems
  to indicate that static linking is an all or nothing proposition which would
  prevent using the -Bstatic/-Bdynamic constructs around the MySQL library.
  This is what I will investigate further to see if there is a clean way of
  forcing the user of the static MySQL libary instead of the dynamic one.

> I know this may be an argument about semantics, and I'm sorry for the rant 
> (again),
> but this all could have been avoided.

Another point:  way back when, we were able to build McIDAS without resorting to
having to define LD_LIBRARY_PATH.  In fact, I think I recall telling you that 
having
to do so was a bad thing.  The fact that we are not able to do this now
means that something changed.

Why having to define LD_LIBRARY_PATH is bad?  To get things working
correctly, I had to add LD_LIBRARY_PATH definitions that include
/usr/local/mysql/lib in:

~mcidas/.bash_profile
~mcidas/.mcenv

~ldm/.bash_profile
~ldm/decoders/xcd_run
~ldm/decoders/batch.k
~ldm/util/mcscour.sh

And, LD_LIBRARY_PATH will have to be defined to include the 
/usr/local/mysql/lib directory
for _every_ user of McIDAS on shu.  If a new version of MySQL gets installed in 
a new,
non-standard directory, all of these files will have to be reedited.  Also, 
when a
new McIDAS distribution is installed, the installer will need to make sure that
all of the files edited here get reedited to reset the values (e.g., xcd_run, 
mcscour.sh,
batch.k, etc.).

At present, the GRIB filing using MySQL is now working:

<as 'mcidas'>
cd $MCDATA
-bash-2.05b$ gribadmin num

  Model Number   % of 
  Name  of Grids Total
  ===== ======== =====
  AWC         10   <1%
  GFS      19976   75%
  ICNG       330    1%
  MAPS      2079    7%
  NWFF         9   <1%
  RCM         28   <1%
  RFC         16   <1%
  RUC       2970   11%
  UKMT       909    3%
  x           15   <1%
  ----- -------- -----
  Total    26342

And ADDE serving of the data is also working:

<still as 'mcidas'>
cd $MCDATA

dataloc.k LIST RTGRIBS

Group Name                    Server IP Address
--------------------         ----------------------------------------
RTGRIBS                      <LOCAL-DATA>

<LOCAL-DATA> indicates that data will be accessed from the local data directory.
DATALOC -- done

grdlist.k RTGRIBS/GFS-ALL NUM=10
Dataset position 1      Directory Title= /GFS.96.2008071.1200.12.44.grib
PAR  LEVEL      DAY          TIME     SRC  FHR  FDAY         FTIME    GRID  PRO
---- ---------- ------------ -------- ---- ---- ------------ -------- ----- ----
V      600 MB   11 MAR 08071 12:00:00  GFS   12 12 MAR 08072 00:00:00   N/A MERC
V      500 MB   11 MAR 08071 12:00:00  GFS   12 12 MAR 08072 00:00:00   N/A MERC
V      400 MB   11 MAR 08071 12:00:00  GFS   12 12 MAR 08072 00:00:00   N/A MERC
V      300 MB   11 MAR 08071 12:00:00  GFS   12 12 MAR 08072 00:00:00   N/A MERC
V      250 MB   11 MAR 08071 12:00:00  GFS   12 12 MAR 08072 00:00:00   N/A MERC
U      250 MB   11 MAR 08071 12:00:00  GFS   12 12 MAR 08072 00:00:00   N/A MERC
U      400 MB   11 MAR 08071 12:00:00  GFS   12 12 MAR 08072 00:00:00   N/A MERC
U      300 MB   11 MAR 08071 12:00:00  GFS   12 12 MAR 08072 00:00:00   N/A MERC
V      200 MB   11 MAR 08071 12:00:00  GFS   12 12 MAR 08072 00:00:00   N/A MERC
V      150 MB   11 MAR 08071 12:00:00  GFS   12 12 MAR 08072 00:00:00   N/A MERC
Number of grids listed = 10
GRDLIST - done

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: PUC-404507
Department: Support ldm-mcidas
Priority: Normal
Status: Closed