[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[McIDAS #KDN-271049]: /home/data/mcidas/images isn't scouring, filling disk



Hi Gilbert,

re: /home/data/mcidas/images isn't scouring, filling disk
> Says it all on weather, weather2 and weather3.

Yesterday afternoon I logged onto weather2 and started a hard look at the
setup there.  I found that the NEXRAD Level III images were being written
into /home/data/gempak/nexrad/NIDS, not /home/data/mcidas/images/NIDS.
Because of this difference, the directories were not being scoured by
the scourBYnumber entry in your crontab.  The /home/data/gempak/nexrad
directory structure was apparently setup to be scoured in ~ldm/etc/scour.conf,
but I found that some of the subdirectories had thousands of files.  Because
of this, I changed the crontab entry to scour /home/data/gempak/nexrad/NIDS
instead of /home/data/mcidas/images/NIDS.  I also ran the scour entry by
hand to verify that it worked correctly; it did.

> Isn't mcscour.sh supposed to do this?

No.  mcscour.sh is designed to scour XCD-produced data files (e.g., MD files,
*.IDX files; etc.).  mcscour.sh was running correctly on weather2 when I
just looked, but I changed it nonetheless:

- changed the output log file from /home/mcidas/workdata/scour.log to
  /home/ldm/logs/mcscour.log.  This makes your scouring match other site's
  scouring

I did not get a chance to finish looking at weather2 yesterday afternoon.  I
had intended to revisit the scouring setup on weather2 this evening.  Your
email reminds me that I need to visit all three machines and regularize
the cron-initiated scouring on each.

Question:

- do I have carte blanche for setting up the scouring?  Of course, I will
  not do anything drastic, but I think some tidying up is needed.

> Why hasn't it been updated since 2004? Just curious.

What, mcscour.sh?  If the setup on weather is the same as it was on weather2
when I logged on this morning, then it was running OK.  Like I said above, 
however,
I want to regularize the scouring and scour logging so that the log file for
mcscour.sh is ~ldm/logs/mcscour.log.

As for /home/data/mcidas/images not being scoured, it is on weather2.  If it
is not on weather/weather3 it would mean that the example crontab entries
created by the McIDAS configuration script mcxconfig have not been incorporated
into the crontab on those machines.  A quick look at weather shows that this is,
in fact, the case.  So, I just added the mcxconfig-generated scouring entries
to the crontab file on weather.

... a few minutes later ...

I just noticed that you added the use of the mcxconfig-generated 
pqact.conf_mcidas[AB]
files on weather.  The problem is that you did not comment out McIDAS actions 
out
of ~ldm/etc/pqact.conf.  The result of this is that several process are being 
run
redundantly.  This can/will cause excessive system use and potential corruption
of output data files.  I have taken the liberty of commenting out the redundant
actions from ~ldm/etc/pqact.conf.

... after a few more minutes ...

OK, I am confused.  Before proceeding I need to know what you really want to
do wrt decoding of imagery.  The reason I ask is I see that on weather you
have actions in ~ldm/etc/pqact.gempak that are decoding imagery using pnga2area
and storing it into a hierarchy under /home/data/gempak/images.  You also have
the mcxconfig-generated actions doing the same decoding into a different 
directory
structure.  I figure that the directory that is not getting scoured (gempak)
is the one that is filling up your disk.

Help!  I need a more clear picture before being able to proceed!!

Cheers,

Tom
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: KDN-271049
Department: Support McIDAS
Priority: Normal
Status: Closed


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.