[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #UKU-190526]: LDM - PIPE usage in pqact.conf and bash scripts

Hi Mike,

re:
> This is a seemingly simple operation but it's rapidly becoming more
> complex than anticipated!  I've tried many variations and searched the
> archives but no luck so far.  Maybe you guys can push me in the correct
> direction.

We'll try :-)

> I'm attempting to use LDM to download netCDF from MADIS, unzip it, then
> write it to a text file for later inclusion into a database.

OK.

> Everything works well but for the "gunzip/ncdump" portion.  This last
> iteration I tried a named pipe attempt (you can't see it but there's a
> "mkfifo pipencdump" command.)  There are lots of other attempts and all
> logs are saved.

Just so we are on the same page, please remember that ncdump gets its input from
a disk file, not STDIN.  Here are the first few lines of the man page for 
ncdump:

NCDUMP(1)                      UNIDATA UTILITIES                     NCDUMP(1)



NAME
       ncdump - Convert netCDF files to ASCII form (CDL)

SYNOPSIS
       ncdump  [-c] [-h] [-v var1,...]  [-b lang] [-f lang] [-l len] [-n name]
              [-p f_digits[,d_digits]] file

DESCRIPTION
       ncdump generates an ASCII representation of a specified netCDF file  on
       standard  output.   The  ASCII  representation  is in a form called CDL
       (��network Common Data form Language��) that can be viewed, edited,  or
       serve  as input to ncgen.  ncgen is a companion program that can gener-
       ate a binary netCDF file from a CDL file.  Hence ncgen and  ncdump  can
       be used as inverses to transform the data representation between binary
       and ASCII representations.  See ncgen for  a  description  of  CDL  and
       netCDF representations.

 ...

> This particular attempt, the most recent, worked well, apparently, but
> resulted in empty files!
> 
> In addition, I wondered if I needed to use "fopen()" and "fclose()"
> with this approach?

No, I don't think so.

> Any other help you could give me would be great.

Try the following revision to your ldmfile3.sh script:

#!/bin/bash

#---------------------------------------------------------------------
#
#  Name:  ldmfile.bash
#
#  Purpose: file an LDM product and log the receipt of the product
#  Note: modify the "log" file for a specific use!
#
#  History: 20070412 - Created from example Bourne shell script
#                        provided by Unidata
#           20070416 - Modified by Tom at Unidata
#
#------------------------------------------------------------------------

# set log file
LOG=/usr/local/ldm/MADIS_data/logs/MADIS_log
exec >>$LOG 2>&1

# Create directory structure
FNAME=$(basename $1)
DIRS=$(echo $1 | sed s/$FNAME//)
mkdir -p $DIRS

# Write stdin to the designated file and create an ASCII version with ncdump
gunzip -cf > $DIRS/pipencdump
ncdump $DIRS/pipencdump > $1

# Log results
if false; then
  echo $(date -u +'%b %d %T') $(basename $0)\[$$\]: Error writing ncdump 
output, exit status = $?"
else
  echo $(date -u +'%b %d %T') $(basename $0)\[$$\]: FILE $1
  rm -f $DIRS/pipencdump
fi

# done

Note that I:

- moved logging into the 'if' statement
- changed error logging to the same file that is specified for success logging
  (using logger forces the log message to be written to ~ldm/logs/ldmd.log)
- added deletion of the uncompressed netCDF file if ncdump was successful.  You
  may or may not want to do this.

> Then I promise to go away and stop bugging you!

:-)  No worries; you are not bugging us.

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: UKU-190526
Department: Support LDM
Priority: Normal
Status: Closed