[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

19990803: NIDS pattern/scouring


Mark,

In the workshop I mentioned I files NIDS directly from the LDM now
and don't use the nids_link, and nids_floater scripts any longer.

Here is the pattern I use for NIDS data to store directly into 
the $SAT directory format (appropriate tabbing of course):
WSI
        
^NEX/(.*)/(.*)/([1-2][0-9])([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
        FILE    data/gempak/nexrad/NIDS/\1/\2/\1-\2_\3\4\5\6_\7\8
WSI
        
^(NOW)/USRAD/([1-2][0-9])([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
        FILE    data/gempak/nexrad/USRAD/8km/\1/\1_\2\3\4\5_\6\7
WSI
        
^(NOW)/MASTER/([1-2][0-9])([0-9][0-9])([0-1][0-9])([0-3][0-9])([0-2][0-9])([0-6][0-9])
        FILE    data/gempak/nexrad/USRAD/2km/\1/\1_\2\3\4\5_\6\7

Then I run the following crontab from ldm:
15 * * * * util/prune_nexrad.csh

here is the prune_nexrad.csh script which sits in the ~ldm/util
directory. Since the script calls itself, PATH must include the
util directory. This routine will scour at -mtime +0 days, or KEEP=60
files per directory.

#!/bin/csh -f
#
# Copyright (c)1993 UCAR/Unidata
#
# Permission to use, copy, modify, and distribute this software and its
# documentation for any purpose without fee is hereby granted, provided
# that the above copyright notice appear in all copies, that both that
# copyright notice and this permission notice appear in supporting
# documentation, and that the name of UCAR/Unidata not be used in
# advertising or publicity pertaining to distribution of the software
# without specific, written prior permission.  UCAR makes no
# representations about the suitability of this software for any purpose.
# It is provided "as is" without express or implied warranty.  It is
# provided with no support and without obligation on the part of UCAR or
# Unidata, to assist in its use, correction, modification, or enhancement.
#
# Steve Chiswell        4/29/99
#
setenv PATH /usr/local/ldm/util:${PATH}
set KEEP=60
if($#argv == 0) then
   set areadir=/usr/local/ldm/data/gempak/nexrad
else
   set areadir=$1
endif

############################################################################## 

cd $areadir
@ NUMSUBS=0
#
set DIRS=`ls -d *`
foreach DIR ($DIRS)
   if(-d $DIR) then
      @ NUMSUBS = $NUMSUBS + 1
      prune_nexrad.csh $DIR
      if($status != 0) rmdir $DIR
   endif
end


@ IRET = 0
# scour down to $KEEP files in this leaf directory
if($NUMSUBS == 0) then
   set FILES=`find . -mtime +0 -print -exec rm -f {} \;`
   #if($#FILES > 0) echo removed old files $FILES
   set FILES=`ls -r`
   if ($#FILES > $KEEP) then
      @ COUNT = $KEEP + 1
      while($COUNT <= $#FILES)
         #echo SCOUR to $FILES[$COUNT]
         rm $FILES[$COUNT]
         @ COUNT = $COUNT + 1
      end
   endif
endif

set FILES=`ls`
if($#FILES == 0) @ IRET = 1


exit(${IRET})


-----------------------------------------------------------------