Re: Large file issue

Before the 3.6.1beta release there was no large file option, so the file size limit was 2Gb. With that release, the limit was increased. See:

        http://www.unidata.ucar.edu/software/netcdf/docs/faq.html

and look under the heading "Large File Support".

HTH,

-Roy M.


On Jan 15, 2007, at 7:17 AM, Hathaway, Simon wrote:

Hello,

I've recently run across a problem reading large NetCDF files, and I'm hoping someone can tell me what I'm doing wrong.

I'm using the MEXNC utility within Matlab to create several NetCDF files on my Windows machine. These are large files ranging from 1 to 30 GB. I can create these files without any problem. However, when I try to read from them, I get an error on files greater than 2 and less than 4GB, greater than 6 and less than 8GB, greater than 10 and less than 12GB, etc. The error string I get back is: "Arg list too long". I get the same error whether I try to open the files using MEXNC in Matlab, or whether I use the ncdump.exe utility. The same problem persists when I try and open these files on other Windows machines, but I was able to successfully open and read all the files on a co-workers Linux machine.

Up to this point, all the systems were running NetCDF version 3.6.0.

I then up graded to version 3.6.1, but the same problem remained, whether I used the pre-built DLL or whether I built it myself in Visual Studio .NET 2003. On a whim, I decided to try the pre-built 3.6.1-beta version of NetCDF, and suddenly I was able to open and read all the files without error.

I'd really like to get this working with either 3.6.0 or 3.6.1. I spent some time going through the 3.6.1 source code, and it looks like the ncio_filesize function (in posixio.c) is making a call to fstat, when it should be making a call fstat64, and so it's not getting the correct file size for files larger than 2GB. I been reading through the Large File Support help page, but I can't find anything that matches this problem. It seems like I must be setting a compiler flag incorrectly or something.

Any help would be appreciated.  Thanks.

Simon Hathaway


**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division 
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."



==============================================================================
To unsubscribe netcdfgroup, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================