[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: netcdf file returns corrupt byte data if using fill



Hi Bryce:

Thanks for taking the time on this. So far, I havent been able to reproduce this problem, or else im not looking in the right place. One think thats a little funny is that this line dimData[0] = netCDFFile.addDimension("TIME", -1); //dynamic dimension

needs to be changed to: dimData[0] = netCDFFile.addDimension("TIME", -1, true, true, false); //explicit declaration of unlimited dimension

so im not sure how you got the program to run.

I have attached my resulting netcdf file, which seems ok to me. does it look 
bad to you ?

BTW, the default value of fill i think has always been false; in the newer 
version, you should use

 public NetcdfFileWriteable(String location, boolean fill);

So maybe theres still some confusion of versions, or some other nefarious 
environmental difference between our two tests ??



Lee,Bryce [Ontario] wrote:
Hi John,

Sorry for the delay. We've been finishing up a new release here.

I've now downloaded netcdf-2.2.10.jar and adapted the example source to
it.

The problem still occurs. I've attached the update source.

Regards,

Bryce
-----Original Message-----
From: John Caron [mailto:address@hidden] Sent: Wednesday, November 30, 2005 8:36 PM
To: Lee,Bryce [Ontario]
Subject: Re: netcdf file returns corrupt byte data if using fill


Hi Lee, I see now that you are using an older version of the library
(2.1). Are you able/willing to retry this with the latest version (2.2).
I know there are a number of changes you have to make, so perhaps thats
not feasible for you. However, the new version has lots of bugs fixed.

If so, the new version is at

http://www.unidata.ucar.edu/software/netcdf-java/

Lee,Bryce [Ontario] wrote:

Hi John,

Here's the sample java program.

Regards,

Bryce Lee

-----Original Message-----
From: John Caron [mailto:address@hidden]
Sent: Wednesday, November 30, 2005 12:10 PM
To: Lee,Bryce [Ontario]
Subject: Re: netcdf file returns corrupt byte data if using fill


Hi Bryce:

Thanks for reporting this bug. A sample program would be great.

Lee,Bryce [Ontario] wrote:



Hopefully I'm not repeating a known issue but I searched the mail archives and didn't come across it. I've been using java netcdf to

store


radar scans, using a dynamic dimension of time to write data as it
arrives throughout the day. Data was being written in float format and


everything was working fine. Recently switched to store data in byte
format instead, to reduce the size of information being stored. Do not


require fill mode, as the entire each radar scan arrives in a complete


chunk. Discovered afterwards that if the server was restarted new data


written to the file became corrupted. I downloaded the netcdf source

and


traced thru it to find that when an existing netcdf file is opened in
write mode, fill mode defaults to true. (This is interesting because

the


default when you first create a file, is off. I see in more recent
documentation this is clarified a bit better.) Anyway, it turns out,

the


fill logic, ends up corrupting the file and adds an extra 4K chunk of
'fill' data. A hex dump of the file easily shows the problem.  Luckily


for me the work around was simple, use the setFill() method to disable


fill as I didn't need fill capability.

The exact version I was testing  2.1.10 Build Date = 2004-07-08
08:59:07.

Not sure if this stream is still being maintained etc. but if interested, I will send a small sample program that demonstrates this

bug.




Attachment: netcdftest4.nc
Description: Binary data