Re: [ldm-users] ldmadmin-pl.config bug?? and scour inefficiency

On Thu, Jul 22, 2010 at 9:02 AM, Jeff Lake - Admin
<admin@xxxxxxxxxxxxxxxxxxxx> wrote:
> also..
> is it just me or when scour is running do others see your loads and memory
> usage go through to roof??
> the loads every once and a while would hit 1.5 with maybe 7-7.5 Gb ram used
> during normal usage
> when scour ran loads would bounce from 4 to 6 and max out all 12Gb of ram
>
> so I wrote a simple bash script to do essentially the same thing. The loads
> and memory usage barely move with it running
> after switching to my NEXRAD2 directory
> these are run
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> find K*/K* -type f -mmin +120 -exec rm {} \;
>
> find P*/P* -type f -mmin +120 -exec rm {} \;
>
> find N*/N* -type f -mmin +120 -exec rm {} \;
>
> find T*/T* -type f -mmin +120 -exec rm {} \;
>
> find R*/R* -type f -mmin +120 -exec rm {} \;
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>
> I do it this way as each radar directory has a php file,
> and yes my scour is altered to delete by minutes old..
> scour handles NEXRAD3 with no problem, but
> kills me when doing NEXRAD2..
> ideas??

Do a "top" during a scour run and you'll probably see that it's an IO
issue more than a CPU issue. When I've had slower disks and journaling
file systems as the main data store, scour would essentially bring
down the system each night.

I went so far as to mess with the find/rm's like you, but went back to
the original scour after upgrading disks and converting to a
non-journaling file system. Scour is using a variation of the find
command with the "xargs" form of 'rm'. According to xargs/find/rm
documentation it should be less CPU intensive than your format above
that essentially forks an "rm" for every file. However, the IO should
be the same.

-Tyler



  • 2010 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: