[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #PJU-328972]: pqact.conf question

Hi Kevin,

re:
> Thank you so much for your help.

No worries.

re:
> I have been trying to make the new -XCD (yes, still working on this, but I’m
> only one person) GRIB processing more efficient.
> I was doing FILE -close, then running a processing script to do the UUID.
> This looks to be a much better approach.

I'm curious why you are adding the UUID to the file name?  If it is to make
the file name unique, you could add the process ID of the script that wrote
the product to disk, and, since there is a new invocation of the script
for each product processed, the name would be unique.  Something like:

#!/bin/bash
 ...
fname=myfilename.$$

cat > $fname
if [ $? -ne 0 ]; then
  cat > /dev/null
fi

The other thing that you may want to do is logging to your script so
that you will have a record of every product processed. I created a
simple function that I use in all of my BASH scripts that creates
timestamped log messages using the same date/time format that is used
in the LDM log file.

For example:

#!/bin/bash

#
# XcdGrib2Processing - write GRIB2 messages to disk and log the result
#
# Invocation:  XcdGrib2Processing filename <log file name>
#
# History:     20221204 - Created as an example
#

# Function to write out log messages
log_message () {
  # Date format for LOG file follows: 20170901T000001.826060Z
  local dfmt="%Y%m%dT%H%M%S"
  local program="`date -u +$dfmt`.`date -u +'%N' | cut -b 1-6`Z `basename 
$0`[$$]:"
  echo $program $1
}

# LDM information directory
LDMHOME=$(echo ~ldm)
LDMBIN=$LDMHOME/bin
LDMLOG=$($LDMBIN/regutil /log/file)
LDMLOGDIR=$(dirname $LDMLOG)

# Set log file name
if [ $# -ge 2 ]; then
  logfile=$2
else
  logfile=$LDMLOGDIR/XcdGrib2Processing.log
fi

# Send all messages to the log file
exec >>$logfile 2>&1

# Announce start
log_message "INFO: Starting up"

# Get the name of the output file
if [ $# -ge 1 ]; then
  myfilename=$1
else
  log_message "ERROR: output file name must be specified, exiting..."
  cat > /dev/null
  exit
fi

# Make the output file name unique
fname=myfilename.$$

# Write the product to disk and log the result
cat > $fname
if [ $? -ne 0 ]; then
  log_message "ERROR: failed to write $fname, flushing STDIN and exiting..."
  cat > /dev/null
  exit
fi

# File successfully writen
log_message "INFO: FILE: $fname"

# Done
log_message "INFO: Done"
exit 0

Cheers,

Tom

> > On Dec 4, 2022, at 9:40 AM, Unidata LDM Support <address@hidden> wrote:
> >
> > Kevin - I apologize, but my earlier response was incomplete.
> >
> > The pqact.conf entries in the earlier examples were too abbreviated.  For 
> > the sake of example, if you edited the script example, and it was in the 
> > path:
> >
> > /home/ldm/util
> >
> > And you called it "myuuid.sh"
> >
> > The full pqact.conf entry for the first example script using your question 
> > would be:
> >
> > NGRID .*
> >    PIPE -close /bin/bash /home/ldm/util/myuuid.sh /data/myfilename
> >
> > And the pqact.conf entry for the second example would be:
> >
> > NGRID .*
> >    PIPE -close /bin/bash /home/ldm/util/myuuid.sh 
> > /data/myfilename.UUID.mysuffix
> >
> > You would need to put the shell reference if you do not chmod on the new 
> > script, but if you do make the new script executable, then you can omit the 
> > "/bin/bash":
> >
> > # First example
> > NGRID .*
> >    PIPE -close /home/ldm/util/myuuid.sh /data/myfilename
> >
> >
> > # Second example
> > NGRID .*
> >    PIPE -close /home/ldm/util/myuuid.sh /data/myfilename.UUID.mysuffix
> >
> > Let us know if you would like additional clarification.
> >
> > Stonie
> >
> > Ticket Details
> > ===================
> > Ticket ID: PJU-328972
> > Department: Support LDM
> > Priority: Normal
> > Status: Open
> > ===================
> > NOTE: All email exchanges with Unidata User Support are recorded in the 
> > Unidata inquiry tracking system and then made publicly available through 
> > the web.  If you do not want to have your interactions made available in 
> > this way, you must let us know in each email you send to us.
> >
> >
> 
> 

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: PJU-328972
Department: Support LDM
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.