[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Huge Files



Hi Brian,

With some help from others I've tracked the problem down to a problem with /home/ldm/util/ucnids. Those very large files are filled not with data but with this:

address@hidden:/mnt/ldmdata/NNEXRAD_NCR/KOAX/20041123> tail KOAX_SDUS53_NCR_200411231638.raw

Read: 0 21 1
Cpy: 21
Read: 0 21 1
Cpy: 21
Read: 0 21 1
Cpy: 21
Read: 0 21 1
Cpy: 21
Read: 0 21 1
Cpy: 21

which is a ucnids issue. If you execute this:
strings /home/ldm/util/ucnids
you'll see that the above is ucnids output, i.e., output from this command includes these lines:
Read: %d %d %d
Cpy: %d
which are the lines that are generating the above. Maybe ucnids is trying to simultaneously read and write the same file? I would carefully examine the relevant pqact.conf entries to see if you're having some sort of a race condition that is causing this.


Please note that this has nothing to do with the upgrade. It must have been occuring before the upgrade as well.

Are you using WXP? We don't support WXP and haven't for some time. FWIW, our GEMPAK expert here pointed out that if you're using GEMPAK or MCIDAS you don't need to uncompress the radar data upon receipt as both those packages will uncompress for you when you ask them to display the data.

You could try to debug this problem by running ucnids on a good version of the problematic data. We have the correct data files available on our archive - you can go to http://motherlode.ucar.edu/ and get the correct version of the corrupted files and try running ucnids on them, which might reveal the problem. If ucnids runs properly on the good data from the command line then the problem is with interaction across or within pqact.conf entries.

If this is really a problem for you I may be able to help you after the holiday. Unfortunately, I can't debug this problem for you now. I was planning on starting my vacation over an hour ago. For now could you deal with the problem by turning off the decompression?

Anne

Miller, Brian S wrote:
Anne-
I removed all of the huge files from /home, but there are still a few in
/mnt/ldmdata/NNEXRAD_NCR
In that directory there are 6 radars.  Look at the data for today in
each radar folder and you will see what I am talking about.
Thanks again.
Brian

-----Original Message-----
From: Anne Wilson [mailto:address@hidden Sent: Tuesday, November 23, 2004 3:25 PM
To: Miller, Brian S
Subject: Re: Huge Files


Hi Brian,

Where are these files?  Please respond ASAP if you can.

Anne

Miller, Brian S wrote:

Anne-

Since the upgrade there are random HUGE files that are being created

on


our LDM server. Every now and then instead of having a 1MB file there


is a 2GB file in its place.



Can you take a look at what it is doing and let me know if I can do anything to stop the "bad" data files? It is filling up my storage

fast.

Brian




-- *************************************************** Anne Wilson UCAR Unidata Program address@hidden P.O. Box 3000 Boulder, CO 80307 ---------------------------------------------------- Unidata WWW server http://my.unidata.ucar.edu/ ****************************************************


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.