[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

19990406: disk space issues with mcidas and mcidas data

>From: Dana Quinn <address@hidden>
>Organization: College of DuPage
>Keywords: 199904062258.QAA22216 McIDAS-XCD disk usage

Dana,

>Hi!  I've got an installation of mcidas 7.4 on one of my
>servers here, and it's working pretty well.

Glad to hear it.

>My only
>problem currently is how much disk space it takes up.
>Specifically, it's the XCD data files that are taking up
>too much room.

McIDAS-XCD data files can occupy a LOT of disk.

>Right now, getting close to the time for
>the scour script to run (runs right after 0Z), the xcd
>directory is at 600 MB of data.  The directory is
>relatively clean - there isn't very much old data
>(more than two days old) sitting in the directory.
>Chief cuplrits are the 163 MB DD990960.XCD file, and
>about 6 files that are ~30 MB, like SAOMETAR.RAT.

The ".XCD" datafiles contain an entire day's worth of textual observations.
The 163 MB reflects just how much textual data is in the the feed for
a single day.

>I'm bringing in the reduced NOAAPORT feed, btw, not the
>full feed.

Perhaps you are limiting the number of GRIDs you are requesting?  It doesn't
look like you are limiting textual data.

>I just can't believe how big these files are,

Yes, but everything is there.

>and how much is in the directory.  I had over 1 GB free on
>the disk when I started, and I'm amazed that it wasn't enough
>to fit everything on.

Our recommendation has been that a site needs to have at least 1 GB of
disk to keep several days worth of surface, upper air, airep/pirep,
FOUS14, etc. point source files; 1 day of the full text stream (the .XCD
files); one day of model runs; etc.  With NOAAPORT, it looks like this
number is only going to increase.  Usually the big culprit is the
McIDAS GRID files since the data in them is not packed at all.

>Is there anything I can do to reduce the space these data
>files take?

You could cut down on the textual data that you send to xcd_run.  This
would cut down on the size of the *.XCD files.  The RAT and RAP files
will be sized to match the amount of data in the *.XCD files, so
cutting down on their size would help.  If GRID files were the real
problem, then I would advise you to pare down the number of grids that
you wanted to decode.

>Perhaps I've configured something incorrectly?

I don't think so.  There is just a LOT of data in the feeds now and
McIDAS-XCD wants to handle all of it.

>I'm really having problems with this machine, because it
>keeps scraping bottom.  Is there anything I can do?

What point source data files are you decoding?  Perhaps this is the first
place to start.  Otherwise, and I hate to say this, it is pretty cheap
to add disk to PCs now adays ($130 for 6.4 GB).

Tom