Re: Large file issue

"Hathaway, Simon" <shathaw@xxxxxxxxxx> writes:

> <P><FONT SIZE=2 FACE="Arial">Hello,</FONT>

Howdy!

> <P><FONT SIZE=2 FACE="Arial">I've recently run across a problem reading large 
> NetCDF files, and I'm hoping someone can tell me what I'm doing wrong.</FONT>
> </P>
>
> <P><FONT SIZE=2 FACE="Arial">I'm using the MEXNC utility within
> Matlab to create several NetCDF files on my Windows machine.&nbsp;
> These are large files ranging from 1 to 30 GB.&nbsp; I can create
> these files without any problem.&nbsp; However, when I try to read
> from them, I get an error on files greater than 2 and less than 4GB,
> greater than 6 and less than 8GB, greater than 10 and less than
> 12GB, etc.&nbsp; The error string I get back is: &quot;Arg list too
> long&quot;.&nbsp; I get the same error whether I try to open the
> files using MEXNC in Matlab, or whether I use the ncdump.exe
> utility.&nbsp; The same problem persists when I try and open these
> files on other Windows machines, but I was able to successfully open
> and read all the files on a co-workers Linux machine.</FONT></P>
>
> <P><FONT SIZE=2 FACE="Arial">Up to this point, all the systems were
> running NetCDF version 3.6.0.</FONT> </P>
>
> <P><FONT SIZE=2 FACE="Arial">I then up graded to version 3.6.1, but
> the same problem remained, whether I used the pre-built DLL or
> whether I built it myself in Visual Studio .NET 2003.&nbsp; On a
> whim, I decided to try the pre-built 3.6.1-beta version of NetCDF,
> and suddenly I was able to open and read all the files without
> error.</FONT></P>

Do you mean that the pre-built 3.6.2-beta release worked?

Or do you mean that the 3.6.1 beta release worked, but not the 3.6.1
final release? If so, which beta are you using?

Also, you must define the preprocessor symbol DLL_NETCDF for code that
uses the netCDF DLL on windows. This causes the lseeki64 function to
be used (among other things).

Are you defining DLL_NETCDF?

>
> <P><FONT SIZE=2 FACE="Arial">I'd really like to get this working
> with either 3.6.0 or 3.6.1.&nbsp; I spent some time going through
> the 3.6.1 source code, and it looks like the ncio_filesize function
> (in posixio.c) is making a call to fstat, when it should be making a
> call fstat64, and so it's not getting the correct file size for
> files larger than 2GB.&nbsp; I been reading through the Large File
> Support help page, but I can't find anything that matches this
> problem.&nbsp; It seems like I must be setting a compiler flag
> incorrectly or something.</FONT></P>

Windows is unique, as always.

Thanks,

Ed
-- 
Ed Hartnett  -- ed@xxxxxxxxxxxxxxxx

==============================================================================
To unsubscribe netcdfgroup, visit:
http://www.unidata.ucar.edu/mailing-list-delete-form.html
==============================================================================


  • 2007 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: