large file support

[Christopher.Moore at noaa.gov]

Ah - found the problem.  I was using an old jar file in my netbeans project.

I'm glad you guys finally started naming the jar file with the version.  For 
2.1, the web site gave me a file called netcdfAll.jar (same name as 2.0), but 
now the base jar is ncCore-2.2.16.jar.

Not only is my app reading my new 64bit-offset files, but the 
Negative-seek-offset problem cleared up, too.

Keep up the good work,
Chris

__________________________________________________________________
Christopher W. Moore             email: Christopher.Moore at noaa.gov
Research Scientist, Oceanography                 tel: 206.526.6779
University of Washington/JISAO/NOAA-PMEL         fax: 206.526.6744
NOAA Center for Tsunami Research                       Seattle, WA
------------------------------------------------------------------

On Thu, 26 Oct 2006, John Caron wrote:

> Hi Chris:
>
> Sounds like a bug!
> Any way tou can put your data on an ftp or http server so I can recreate the 
> problem?
>
> thanks
>
> Christopher.Moore at noaa.gov wrote:
>> 
>> Hi John & others,
>> 
>> A lot of us are creating large netcdf files these days, and I was wondering 
>> if anyone is having the same kind of problems I am:
>> 
>> When I open a large (~2Gb) netcdf file using the latest stable release, 
>> I've been getting an error.  The code was working when the file size was 
>> smaller (fewer time steps, smaller lat/lon resolution).
>> 
>> My data is simply
>> dimensions:
>>         LON = 1276 ;
>>         LAT = 950 ;
>>         TIME = UNLIMITED ; // (601 currently)
>> variables:
>>         double LON(LON) ;
>>                 LON:units = "degrees_east" ;
>>                 LON:point_spacing = "even" ;
>>         double LAT(LAT) ;
>>                 LAT:units = "degrees_north" ;
>>                 LAT:point_spacing = "uneven" ;
>>         double TIME(TIME) ;
>>                 TIME:units = "SECONDS" ;
>>         float HA(TIME, LAT, LON) ;
>> 
>> I start by opening the file, getting the time dimension & variable.  It 
>> croaks when reading the variable into an Array (note that I'm only reading 
>> the TIME axis array, fairly small):
>>
>>             ...
>>             Variable testVar = timeDim.getCoordinateVariable();
>>             System.out.println("testVar.getName: " + testVar.getName());
>>             System.out.println("testVar.getDataType: " + 
>> testVar.getDataType());
>>             System.out.println("testVar.getRank: " + testVar.getRank());
>>             int[] testShape = testVar.getShape();
>>             for (int q=0; q<testShape.length; q++)
>>                 System.out.println("testShape["+q+"]: " + testShape[q]);
>>             int[] testOrigin = new int[testVar.getRank()];
>>             Array testArr2 = testVar.read(testOrigin,testShape);
>> 
>> error: java.io.IOException: Negative seek offset
>> java.io.IOException: Negative seek offset
>>         at java.io.RandomAccessFile.seek(Native Method)
>>         at ucar.netcdf.RandomAccessFile.read_(RandomAccessFile.java:508)
>>         at ucar.netcdf.RandomAccessFile.seek(RandomAccessFile.java:350)
>>         at 
>> ucar.netcdf.NetcdfFile$V1DoubleIo.readArray(NetcdfFile.java:1520)
>>         at ucar.netcdf.NetcdfFile$V1Io.copyout(NetcdfFile.java:896)
>>         at ucar.netcdf.Variable.copyout(Variable.java:276)
>>         at ucar.nc2.Variable.read(Variable.java:237)
>> 
>> 
>> I immediately thought it was a Large File Support problem, but a little 
>> investigation turned up the fact that I don't (yet) need 64-bit addressing 
>> because my record sizes are small.  In fact, I can read the file fine with 
>> applications that use the C and FORTRAN netcdf libraries (version 3.6 or 
>> higher).  And ncdump works.
>> 
>> But I went ahead and created the large file with 64-bit addressing anyway 
>> by simply changing one line in my FORTRAN model from
>>
>>         iret = nf_create(ncfn, NF_CLOBBER, ncid)
>> to
>>
>>         iret = nf_create(ncfn, OR(NF_CLOBBER, NF_64BIT_OFFSET), ncid)
>> 
>> It creates the file just fine, checking with:
>>
>>       od -An -c -N4 foo.nc
>> gives the expected
>>       C    D    F 002
>> 
>> and ncdump gives
>>
>>      netcdf s_2903-563_ha64 { // format variant: 64bit
>>      dimensions: ...
>> 
>> but my java code still croaks (in the same way Ferret does if not compiled 
>> against netcdf lib > 3.6):
>> 
>> Exception in thread "AWT-EventQueue-0" java.lang.IllegalArgumentException: 
>> Not a netcdf file
>>         at ucar.netcdf.NetcdfFile.readV1(NetcdfFile.java:1745)
>>         at ucar.netcdf.NetcdfFile.<init>(NetcdfFile.java:130)
>>         at ucar.netcdf.NetcdfFile.<init>(NetcdfFile.java:148)
>>         at ucar.nc2.NetcdfFile.<init>(NetcdfFile.java:61)
>> 
>> 
>> Anybody else out there creating java applications that read large files?
>> 
>> I'm running RHEL4 on an EMT64 x86_64 machine, with java 1.5.0_06 and java 
>> netcdf version 2.2.16
>> 
>> Yours,
>> Chris
>> 
>> __________________________________________________________________
>> Christopher W. Moore             email: Christopher.Moore at noaa.gov
>> Research Scientist, Oceanography                 tel: 206.526.6779
>> University of Washington/JISAO/NOAA-PMEL         fax: 206.526.6744
>> NOAA Center for Tsunami Research                       Seattle, WA
>> ------------------------------------------------------------------
>> 
>> =============================================================================== 
>> To unsubscribe netcdf-java, visit:
>> http://www.unidata.ucar.edu/mailing-list-delete-form.html
>> =============================================================================== 
>
> ==============================================================================
> To unsubscribe netcdf-java, visit:
> http://www.unidata.ucar.edu/mailing-list-delete-form.html
> ==============================================================================
>
>

==============================================================================
To unsubscribe netcdf-java, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================