[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: large netCDF data sets on AIX platforms

Tom,

I wrote:

> Until we can get a bigger local disk on zasu or get it configured to
> support writes of files larger than 1 GiB across NFS, I'll have to
> continue debugging this problem on bluesky instead of our local AIX.

The bottom line is that after rebuilding everything on bluesky (the
supercomputer AIX system NCAR makes available for research) and
running the tests in a directory where quotas didn't prevent me from
creating large files, the large file tests seem to work as intended,
including the example in src/examples/C/large_files.c and the
wrf_test.f test I made available on the web.

The large_files.c C program writes the file:

  -rw-r--r--   1 russ     ncar     4294955206 Aug 30 21:01 rec2Gp.nc

which is identical to the same netCDF file generated on a Sun
platform, as verified by cmp.  

Similarly, running the wrf_test.f program on bluesky results in the
file

  -rw-r--r--   1 russ     ncar     2661494644 Aug 30 21:11 wrf_test.nc

which cmp demonstrates is identical to the file generated using the
same program with netcdf-3.6.0-beta3 on a Sun.

netcdf-3.6.0-beta3 was compiled on bluesky using something close to
what you tried, though perhaps some of this is unnecessary (for
example, for these tests the xlc compiler probably suffices rather
than the xlc_r you used for multi-threaded applications):

  CC='/usr/vac/bin/xlc_r -q64' 
  CFLAGS='-g -D_LARGE_FILES' 
  FC='/bin/xlf_r -q64'
  FFLAGS='-g -qarch=auto' 
  F90='/bin/xlf90_r -q64' 
  F90FLAGS='-g -qsuffix=f=f90 -qmaxmem=-1 -qarch=auto' 
  CXX='' 
  ARFLAGS='-X64 cru' 
  NMFLAGS=-X64

I don't think the ARFLAGS or NMFLAGS are relevant to netCDF, but I set
them anyway.  Also, the current beta release ignores the CXX=''
setting saying not to compile a C++ interface, so I had to manually
override the generated CXX lines in the src/macros.make file to be:

CXX             =
CXXFLAGS        =

but it would probably work OK to use the AIX C++ compiler.  I just
didn't try it, because it wasn't relevant to the wrf_test.f test.

I should have tried to duplicate your problem on bluesky earlier
rather than a small AIX system, but it's more trouble, because we
can't mount files remotely and logging in is awkward.  But now that I
have a testing framework set up there, maybe we can get to the bottom
of why your tests and ours differ.

Could you please send us the items mentioned in the "Reporting
Problems" section of the "Installing netCDF" document, at

  
http://my.unidata.ucar.edu/content/software/netcdf/INSTALL.html#ReportingProblems

Meanwhile, we'll try to reduce uncertainty about which environment
variables and compiler options are really necessary for our large file
tests to work on an IBM-SP/AIX supercomputer, which should help
isolate the differences we are seeing.

--Russ