[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #GXY-620794]: nf_open_par() Unknown file format

THe nc_open_par (called by nf_open_par) will automatically
use pnetcdf if the netcdf-c library supports it.
THe HDF5 library will not be used at all, so its capabilities
will not matter.

> ...The site that runs the application claims they use the
> nf_open_par() branch - how is it they do not get my error?
> The flags in my test program are the same as those used by
> the application.  Did you try my test program?
> 
> Is it possible the site uses NetCDF3?  I ask because of my
> suspicion that they use Thread-Safe HDF5, which normally
> excludes parallel operation; and building NetCDF with that
> HDF5 would not allow configuration with "--enable-netcdf4"
> so would have only NetCDF3 functionality.
> 
> Is nf_open_par() available in NetCDF built this way?
> 
> 
> On Tue, Aug 28, 2018 at 09:49:12AM -0600, Unidata netCDF Support wrote:
> > So it turns out that that file you sent me (iconR2B04-grid.nc)
> > is actually a netcdf-3 file with 64bit offset. Reading it will not
> > use HDF5 at all. You can read it in using mpio by using the
> > NC_PNETCDF flag when opening the file. That also requires that you
> > build the netcdf library with pnetcdf enabled. The pnetcdf library
> > in turn used mpio.
> > BTW, to see the file format use this command:
> >     ncdump -k iconR2B04-grid.nc
> >
> >
> > > ...Just in case there is another problem with my email
> > > attachment, you may download the file here:
> > >
> > > http://www.performancejones.com/iconR2B04-grid.nc.gz
> > >
> > > I have tested this just to be sure.
> > >
> > >
> > > On Tue, Aug 28, 2018 at 08:22:30AM +0000, Max R. Dechantsreiter wrote:
> > > > On Mon, Aug 27, 2018 at 01:33:22PM -0600, Unidata netCDF Support wrote:
> > > > > I saw no email attachment. All I saw was this:
> > > >
> > > > OK, for the fourth time I am attaching the file.
> > > > It is possible your mail and mine (Mutt) are not plauying nice,
> > > > so please check the size of the message - including attachment
> > > > it should be several MB.
> > > >
> > > > My original inquiry, including test program, is below.
> > > >
> > > >
> > > > > Dear NetCDF Support:
> > > > >
> > > > > I have an input file that I can read with my ncdump, open
> > > > > with nf_open(), but which gives an error with nf_open_par():
> > > > >
> > > > > Testing nf_open()...
> > > > >
> > > > > nf_open():
> > > > > No error
> > > > >
> > > > > nf_close():
> > > > > No error
> > > > >
> > > > > Testing nf_open_par()...
> > > > >
> > > > > nf_open_par():
> > > > > NetCDF: Unknown file format
> > > > >
> > > > > What does this mean - what is it about the file that that
> > > > > allows it to be opened by nf_open() but not nf_open_par()?
> > > > > I get the same error with different versions of NetCDF,
> > > > > and with different versions of HDF5 (if that matters).
> > > > >
> > > > > Below is my test script.
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Max
> > > > > ---
> > > > > Max R. Dechantsreiter
> > > > > President
> > > > > Performance Jones L.L.C.
> > > > > address@hidden
> > > > >
> > > > >
> > > > > #!/bin/sh
> > > > >
> > > > > cat > test_nf_open_par.F90 << EOF
> > > > > program test_nf_open_par
> > > > > !USE mpi, ONLY: MPI_INFO_NULL
> > > > > USE mpi
> > > > > INCLUDE 'netcdf.inc'
> > > > >
> > > > > character (len=*), parameter :: grid="iconR2B04-grid.nc"
> > > > > integer :: ierr, ncid, status
> > > > >
> > > > > call MPI_Init(ierr)
> > > > >
> > > > > print '()'
> > > > >
> > > > > print '(a,/)', "Testing nf_open()..."
> > > > > status = nf_open(grid, nf_nowrite, ncid)
> > > > > print *, "nf_open(): ", nf_strerror(status)
> > > > > if (status == nf_noerr) then
> > > > > status = nf_close(ncid)
> > > > > print *, "nf_close(): ", nf_strerror(status)
> > > > > end if
> > > > >
> > > > > print '(a,/)', "Testing nf_open_par()..."
> > > > > status = nf_open_par(grid, IOR(nf_nowrite, nf_mpiio), &
> > > > > &                        MPI_COMM_WORLD, MPI_INFO_NULL, ncid)
> > > > > print *, "nf_open_par(): ", nf_strerror(status)
> > > > > if (status == nf_noerr) then
> > > > > status = nf_close(ncid)
> > > > > print *, "nf_close(): ", nf_strerror(status)
> > > > > end if
> > > > >
> > > > > call MPI_Finalize(ierr)
> > > > >
> > > > > end program test_nf_open_par
> > > > > EOF
> > > > >
> > > > > module purge
> > > > > module load intel/2017.5.239
> > > > > module load impi/2017.4.239
> > > > > module list
> > > > >
> > > > > ZLIB_VERSION=1.2.11
> > > > > SZIP_VERSION=2.1.1
> > > > > HDF5_VERSION=1.10.2
> > > > > HDF5_VERSION=1.8.18
> > > > > NETCDF_C_VERSION=4.6.1
> > > > > NETCDF_FORTRAN_VERSION=4.4.4
> > > > >
> > > > > LIB=/global/scratch/users/maxd/ICON/lib-avx512
> > > > >
> > > > > ZLIB=$LIB/zlib-$ZLIB_VERSION
> > > > > SZLIB=$LIB/szip-$SZIP_VERSION
> > > > > HDF5=$LIB/phdf5-$HDF5_VERSION
> > > > > NETCDF4=$LIB/netCDF4-$NETCDF_C_VERSION
> > > > >
> > > > > export 
> > > > > CPATH=$NETCDF4/include:$HDF5/include:$ZLIB/include:$SZLIB/include:$CPATH
> > > > > LD_LIBRARY_PATH=$HDF5/lib:$ZLIB/lib:$SZLIB/lib:$LD_LIBRARY_PATH
> > > > > LD_LIBRARY_PATH=$NETCDF4/lib:$LD_LIBRARY_PATH
> > > > > LIBRARY_PATH=$LD_LIBRARY_PATH
> > > > > export LD_LIBRARY_PATH # shared libraries
> > > > > export LIBRARY_PATH    # static libraries
> > > > >
> > > > > mpiifort test_nf_open_par.F90 -lnetcdff -lnetcdf -lhdf5 -lsz -lz
> > > > >
> > > > > mpiexec -n 1 ./a.out
> > >
> > >
> > >
> >
> > =Dennis Heimbigner
> >   Unidata
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: GXY-620794
> > Department: Support netCDF
> > Priority: Normal
> > Status: Open
> > ===================
> > NOTE: All email exchanges with Unidata User Support are recorded in the 
> > Unidata inquiry tracking system and then made publicly available through 
> > the web.  If you do not want to have your interactions made available in 
> > this way, you must let us know in each email you send to us.
> >
> >
> 
> 

=Dennis Heimbigner
  Unidata


Ticket Details
===================
Ticket ID: GXY-620794
Department: Support netCDF
Priority: Normal
Status: Open
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.