[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: large netCDF data sets

Tom,

I wrote:

> I've just tried building netCDF-3.6.0-beta3 with all the above
> options (except I set CXX='' since our C++ compiler is not up-to-date)
> and it built OK.  Then I tried compiling the wrf_test.f file I made
> available by changing its suffix to .f90 and building it with
> 
>   /bin/xlf90_r -qfixed -qsuffix=f=f90 -qmaxmem=-1 -qarch=auto -q64 -o 
> wrf_test -I../include -L../lib -lnetcdf -b64 
> /scratch/russ/lfs-tests/wrf_test.f90
> 
> in hopes that explicitly adding the "-q64" compile flag and "-b64"
> link flag might make things work.  When I run the resulting
> executable, it only creates a file of about 2**30 bytes and quits with
> an assertion violation in the library:
> 
>   Assertion failed: *posp == OFF_NONE || *posp == lseek(nciop->fd, 0, 
> SEEK_CUR), file  posixio.c, line 210
>   IOT/Abort trap(coredump)
> 
> which Ed is also seeing in his test cases on AIX.  You seem to have
> gotten further than this, producing a file that's about 2**31 bytes
> before getting a different error, so we can't reproduce the problem
> you are seeing. ...

It turns out this error (the assertion violation) has nothing to do
with the netCDF library, but is instead an artifact of our AIX system
not being configured to write files larger than 2**30 bytes (that's
1073741824) on an NFS-mounted file system!  I proved this to myself by
just trying to write a file slightly larger than 2**30 bytes using
nothing but system calls to open() and write() and getting the error:

  write failed!: File too large

in trying to write more than 1073741312 bytes on our desktop AIX
platform (zasu.unidata.ucar.edu).

Unfortunately on AIX, lseek() returns the wrong current position in
the file with no error indication just before a write() error occurs,
so we don't get the nice error message about "file too large", just
the ugly assertion violation message.

Until we can get a bigger local disk on zasu or get it configured to
support writes of files larger than 1 GiB across NFS, I'll have to
continue debugging this problem on bluesky instead of our local AIX.

Ed: this is no doubt the cause of all the assertion violations like
this that we've seen on zasu, so we should drop it as a testing
platform until this gets fixed ...

--Russ