[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[ldmMcidas #KQH-987099]: Make error: ldm-mcidas on Mac OS X 10.5.1

Hi Dave,

re:
> Tom (probably),

Yup, its me :-)

> I've got a nice new Mac Pro Quad-core running Mac OS X 10.5.1 that
> sooner (with good luck) or later (with poor luck) is going to be my
> local data server, running LDM and some decoders.

Sounds like fun!

> It seems to be
> running LDM OK (thanks to some intervention by Steve Emmerson), other
> than the fact that it's not logging to logs/ldm.log correctly (though
> it is logging to system.log).
> 
> In any case, one of the decoders that I need is pnga2area, which is
> probably the only one in the ldm-mcidas package that I need. I'm
> having problems compiling ldm-mcidas, though.

OK.

> I've attached config.log, configure.log, and make.log files that
> testify to the difficulty.

Thanks.  You giving me a login was even more helpful.

> The results don't seem to be sensitive to the compiler I specify in my
> CUSTOMIZE file (c89, cc, or gcc)--the make attempt fails at the same
> point, regardless.
> 
> I know you don't have a Mac (at least of this sort) available at
> Unidata, so if you need to intervene directly, you can do so as follows:

Thanks for the access to your machine!

> ldm-mcidas is located in /usr/local/unidata on my new machine.

I logged on to virga (using SSH, of course) this morning and started poking
around.  Here is what I found and what I did to get most of ldm-mcidas to
build:

- first, I created an ldm-mcidas subdirectory under /usr/local/unidata and
  I moved the ldm-mcidas-2004 tree and distribution under the new directory

- next, I copied over an as yet unreleased version of ldm-mcidas, v2006, to
  your /usr/local/unidata/ldm-mcidas and unpacked it creating the
  /usr/local/unidata/ldm-mcidas/ldm-mcidas-2006 directory tree

- in preparation for running 'configure', I defined a set of environment
  variables in your Tcsh defintion file ~ldm/.cshrc:

setenv CC /usr/bin/gcc
setenv FC /usr/local/bin/g95
setenv CPP_LDM -I/Users/ldm/include
setenv LD_LDM '-L/Users/ldm/lib -lldm'
setenv CPP_MCIDAS -I/Users/mcidas/mcidas2007/src
setenv LD_MCIDAS '-L/Users/mcidas/lib -lmcidas'
setenv CPP_NETCDF -I/Users/mcidas/mcidas2007/netcdf/libsrc
setenv LD_NETCDF '-L/Users/mcidas/mcidas2007/netcdf/libsrc -lnetcdf'

  Comments (mainly for myself):

  - defines for all flags was reasonably straightforward

  - CPP_MCIDAS, LD_MCIDAS, CPP_NETCDF, and LD_NETCDF point to non-existent
    locations.  This was done just to be able to produce a Makefile.

  - it took me a couple of minutes to determine that the only Fortran
    compiler installed was 'g95' -- no biggie as pnga2area is an all-C
    application

- after making the environment variable definitions active in ~ldm/.cshrc,
  I ran 'configure' as follows:

  <as 'ldm'>
  cd /usr/local/unidata/ldm-mcidas/ldm-mcidas-2006/src
  ./configure

- I then ran make.  This is when I ran into the problems you reported:

  1) the configure step defined a macro named UD_SYSTEM_LIMITS_H 
     as ///usr/lib/gcc/i686-apple-darwin9/4.0.1/include/limits.h.
     This file exists and contains the appropriate limits.h stuff
     for the installation/compiler (gcc is version 4.0.1)

  2) gcc complained as follows:

gcc -c -O -I/usr/local/src/mcidas/src -I/usr/local/unidata/ldm/include 
-I../libpng -I../zlib -I../port/misc -I../port/cfortran -I/usr/local/include  
commonsubs.c
In file included from ../port/misc/limits.h:12,
                 from commonsubs.c:10:
///usr/lib/gcc/i686-apple-darwin9/4.0.1/include/limits.h:10:25: error: no 
include path in which to search for limits.h
make[2]: *** [commonsubs.o] Error 1
make[1]: *** [decode/current] Error 1

     Evidently, gcc did not like the code produced by 'configure' in 
src/port/misc/limits.h:

#define UD_SYSTEM_LIMITS_H      
"///usr/lib/gcc/i686-apple-darwin9/4.0.1/include/limits.h"
#include UD_SYSTEM_LIMITS_H

     Experimentation showed that the following handmade modification allowed 
the build to
     proceed further (but not completely)

#define UD_SYSTEM_LIMITS_H      <limits.h>
#include UD_SYSTEM_LIMITS_H

     I need to talk to Steve Emmerson about this issue.

- after fixing-up the port/misc/limits.h file (again, created by 'configure'), I
  continued with the make, but ran into the cfortran code not knowing about this
  platform.  Reviewing the cfortran code (port/cfortran/cfortran.h) showed that
  it had no knowledge of Macs in general.

  A Google search using 'cfortran MacOS' resulted in a number of hits the most
  relevant/useful of which was:

  https://setisvn.ssl.berkeley.edu/svn/lib/cfitsio/cfortran.h

  That page showed some modifications made for Macintosh:

#if defined(macintosh)     /* 11/1999 */
#define f2cFortran
#endif
#if defined(__APPLE__)     /* 11/2002 */
#define f2cFortran
#endif
#if defined(__hpux)             /* 921107: Use __hpux instead of __hp9000s300 */
#define       hpuxFortran       /*         Should also allow hp9000s7/800 use.*/
#endif

  I added the two defines related to Macs (macintosh and __APPLE_) to 
  port/cfortran/cfortran.h and was then able to build all ldm-mcidas executables
  that did not rely on the McIDAS library (including proftomd which needs to
  link against the netCDF library since netCDF is installed on the machine!)

- since building of 'nldn2md' failed, I was unable to fully run 'make install':

  <still as 'ldm'>
  cd /usr/local/unidata/ldm-mcidas/ldm-mcidas-2006/src
  make install        <- this fails at some point, but it does create the needed
                         output bin, lib, and etc directories

- I tried running 'make pnga2area.test' from the decode directory, but I ran 
into
  a problem where MacOS does not differentiate between upper and lower case 
file names.
  This will result in 'route.sys' and 'syskey.tab' being deleted when a 'make 
clean'
  or 'make distclean' is run from the src or decode directories.  This, in turn,
  forced me to re-extract these files from the compressed tar file.  Argh!@#$%

  In order to get 'make pnga2area.test' to run, I was forced to edit the 
Makefile:

  <as 'ldm'>
  cd /usr/local/unidata/ldm-mcidas/ldm-mcidas-2006/src/decode
  -- edit Makefile and comment out the copying of route.sys to ROUTE.SYS and
     syskey.tab to SYSKEY.TAB

  I was then able to run the test:

ldm_virga% make pnga2area.test
# cp route.sys ROUTE.SYS
# cp syskey.tab SYSKEY.TAB
rm -f pngarea.test AREA0200;
././pnga2area -vl- -a SATANNOT -b SATBAND -r U3 < pngarea
Feb 09 19:54:03 pnga2area[45786] NOTE: Starting Up
Feb 09 19:54:03 pnga2area[45786] INFO: output file pathname: ./AREA0200
Feb 09 19:54:03 pnga2area[45786] NOTE: unPNG::     5549    225088  40.5637
Feb 09 19:54:03 pnga2area[45786] NOTE: Exiting
rm -f AREA0200;
././pnga2area -vl- -a SATANNOT -b SATBAND -r ZZ,200 < pngarea
./batch.k: line 121: cd: ~mcidas: No such file or directory
Feb 09 19:54:03 pnga2area[45795] NOTE: Starting Up
Feb 09 19:54:03 pnga2area[45795] INFO: output file pathname: ./AREA0200
Feb 09 19:54:03 pnga2area[45795] NOTE: unPNG::     5549    225088  40.5637
Feb 09 19:54:03 pnga2area[45795] NOTE: Exiting
././area2png -vl- -a SATANNOT -b SATBAND -f AREA0200 pngarea.test
Feb 09 19:54:03 area2png[45798] NOTE: Starting Up
./batch.k: line 168: cd: 
/Users/ldm/unidata/ldm-mcidas/ldm-mcidas-2006/src/decode/workdata: No such file 
or directory
Feb 09 19:54:03 area2png[45798] INFO: output file pathname: pngarea.test
Feb 09 19:54:03 area2png[45798] NOTE: doPNG::   225088      5549  0.0247
PNGA2AREA test successful

- since I had a working version of pnga2area, I hand-copied the executables 
that had
  been built to the ldm-mcidas installation bin directory:

  <still as 'ldm' in the src/decode directory>
  cp area2png pnga2area proftomd zlibg2gini batch.k pngg2gini uwgrid.sh 
../../bin

- I then made some symbolic links:

  cd /usr/local/unidata/ldm-mcidas
  ln -s ldm-mcidas-2006 runtime
  ln -s runtime/* .

- next, I copied the newly created version of pnga2area to the decoders 
directory
  defined in 'ldm's PATH:

  cp bin/pnga2area /usr/local/unidata/decoders/bin

- since Dave had mentioned some problems with LDM logging, I decided to modify
  the pqact.conf file to log pnga2area transactions to ~ldm/logs/ldm-mcidas.log.

  N.B., logging to ldm-mcidas.log will require that you keep an eye on this 
file.
  The best approach is to rotate the file from a crontab entry like:

#
# Rotate ldm-mcidas logs
#
00 16 * * * bin/newlog logs/ldm-mcidas.log 7

  N.B., I did not add this entry to your crontab.

- since I did not see any UNIWISC (aka MCIDAS) images arrive while requesting
  from freshair.atmos.washington.edu, I added a redundant request for
  DDPLUS|IDS|UNIWISC to the ~ldm/etc/ldmd.conf file and restarted the LDM.

- after a bit of time, I saw UNIWISC images being received and processed.  Here
  is a listing of pnga2area processing in ~ldm/logs/ldm-mcidas.log:

  <cd ~ldm>
  less logs/ldm-mcidas.log

Feb 09 19:16:55 pnga2area[27354] NOTE: Starting Up
Feb 09 19:16:55 pnga2area[27354] INFO: output file pathname: 
/data/mcidas/gewv/08020918_swe.mca
Feb 09 19:16:55 pnga2area[27354] NOTE: unPNG::   738391   4724488  6.3984
Feb 09 19:16:55 pnga2area[27354] NOTE: Exiting
Feb 09 19:16:58 pnga2area[27463] NOTE: Starting Up
Feb 09 19:16:58 pnga2area[27463] INFO: output file pathname: 
/data/mcidas/geir/0802091800_sie.mca
Feb 09 19:16:58 pnga2area[27463] NOTE: unPNG::  1389722   4724488  3.3996
Feb 09 19:16:58 pnga2area[27463] NOTE: Exiting
  ...

I believe at this point that you have a functional pnga2area decoder running on 
virga.

However, I have some questions/comments:

- the UNIWISC actions defined in your ~ldm/etc/pqact.conf file all decode to the
  /data/mcidas directory hierarchy.  Since there are lots of subdirectories
  under /data/mcidas (e.g., gevis, etc.), and since these subdirectories have
  lots of current images in them, I am left wondering:

  Is another machine decoding the same images into the same directory hierarchy?

  If the answer is yes, you need to either change your output directory on virga
  or turn off decoding on whatever other machine is writing to the same 
directories.
  If you don't change this setup, the images being decoded may well end up 
corrupted.

- a 'locate pnga2area' shows that there is a copy in /Volumes:

ldm_virga% locate pnga2area
/Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area
...

  This decoder is for the PPC architecture:

ldm_virga% file /Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area
/Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area: Mach-O 
executable ppc

  The newly-built version of pnga2area is for Intel hardware as expected:

ldm_virga% file `which pnga2area`
/usr/local/unidata/decoders/bin/pnga2area: Mach-O executable i386

  Interestingly, your Intel-based MacOS can run PPC executables:

ldm_virga% /Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area
Example Usage:
       /Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area <-a 
SATANNOT> <-b SATBAND> <-vxl-> IMGfile < infile 
       /Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area <-a 
SATANNOT> <-b SATBAND> <-vxl-> -d datadir -r pcode < infile 
       /Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area <-a 
SATANNOT> <-b SATBAND> <-vxl-> -f infile IMGfile
       /Volumes/virgabackup/usr/local/src/decoders-4.1.0/bin/pnga2area <-a 
SATANNOT> <-b SATBAND> <-vxl-> -f infile -d datadir -r pcode
Flags:
    Help:
    -h              Print the help, then exit the program

    Satellite definitions:
    -a SATANNOT     McIDAS SATANNOT pathname (def: /home/mcidas/data/SATANNOT)
    -b SATBAND      McIDAS SATBAND pathname (def: /home/mcidas/data/SATBAND)
    -f infile       Pathname for input PNG compressed file (default is stdin)

    Logging:
    -l logname      Log file name (default uses local0; use '-' for stdout)
    -v              Verbose logging
    -x              Debug logging
 ...


So, building pnga2area on virga was apparently not needed at all!

Mind you, this exercise was valuable since:

- it pointed out some work that is needed in the 'configure' part of the 
ldm-mcidas
  package

- the v2006 version of pnga2area fixes some shortcomings in the v2004 version so
  it is preferable to run the newer version

Final comment:

The current versions of McIDAS, GEMPAK, and _I believe_ the IDV can all use the
PNG-compressed AREA files directly.  Because of this, we have turned off 
decoding
of the images by pnga2area and replaced the actions by FILEs on a number of the
systems under our control.  You may want to do the same _if_ you are not using 
the
images for other applications that do not know how to deal with the compressed 
imagery.

Please let me know if you have any questions and/or comments about the above.
Again, most of what I wrote was designed to be useful to me, not you, so it
may seem a bit obtuse.  The possible issue of decoders on different machines
writing the same images to the same directory needs to be addressed.

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: KQH-987099
Department: Support ldm-mcidas
Priority: Normal
Status: Closed