Good question - isn’t it the same I/O process, just a matter of when?
I'm wondering if the programming overhead of a file modification time parameter
test conducted by the ‘find’ command makes it worth avoiding.
Gerry Creager is suggesting that a solution is to just increase the frequency
of running a find command (scour). It’s worth a try, but I don’t see how that
escapes the convergence of time to eval delete qualifications inode list
expansion rate.
But if that’s a working solution, then — in the limit — so is a file delete
action executed coincident with a new file write.
I interpret Tom Yoksas as saying to fail over to a different file system
catalogue operation - removal of groups of inodes instead of removal of each
inode separately.
Good idea. And the one Donna Cote settled on. Which will require me to adjust
all my friggin gempak GUI configs and scripted real-time graphics product
generation.
Which I'm getting paid to do ...
-Neil
> On Oct 28, 2015, at 10:28 AM, Donna Cote <d-cote@xxxxxxxx> wrote:
>
> Neil, while it is possible to use the EXEC argument, given the filename, to
> create the rm command line statment (on that filename minus x number of
> days), I don't see how this would make less removes than you are already
> doing.
>
> I would like to ask for a more io-friendly scour utility.
>
> Donna
> Texas A&M University
> The Academy
>
>
> On Wed, Oct 28, 2015 at 10:17 AM, Smith, Neil R <n-smith2@xxxxxxxxxxxxx>
> wrote:
> Scouring 10 days of online, or even nearline, NIDS and NEXRCOMP being held
> for a teaching environment is a big headache.
> Donna Cote has recently polled the community about this issue.
>
> At this file count, crawling thru the millions of inodes with a scheduled
> find command will not keep up with the accumulation rate.
> (sounds like the intersection of two curves, beyond which there is no return;
> would anyone care to derive that solution?)
>
> This leads me to wonder if one can construct a pqact entry that will do the
> FILE or STDIOFILE action on the incoming product plus remove the same product
> of the same date minus a desired day count. Or the remove action, -exec,
> could be a separate pqact entry on the same product regular expression minus
> the day count, just so it’s done at the time of the new product arrival. Is
> there a clever way to manipulate the syntax of temporal subexpression
> replacement described in the pqact man pages to generate the desired filename
> to remove?
>
> Would that even solve the problem?
>
> Or is there a pqact solution? Are you just left with scripting OS commands
> (find; eg. scour)?
>
> -Neil
>
> --
> Neil R. Smith, Senior IT Specialist II neils@xxxxxxxx
> Atmospheric Sciences, Texas A&M Univ. 979/845-6272
> --
>
> _______________________________________________
> ldm-users mailing list
> ldm-users@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
