Re: [netcdf-hdf] [netcdfgroup] NetCDF: HDF error, and now what?

  • To: John Urbanic <urbanic@xxxxxxx>
  • Subject: Re: [netcdf-hdf] [netcdfgroup] NetCDF: HDF error, and now what?
  • From: "Hernan G. Arango" <arango@xxxxxxxxxxxxxxxxxx>
  • Date: Mon, 24 Oct 2011 14:09:57 -0400
Hi,
Thanks for the quick follow-up, Ed.  Two amplifications on your answer would
be helpful:

nc_set_log_level(3);
(There is also a fortran version.)
I noticed this mentioned in passing in the mail archives, but do not see it
in any of the manuals.  I'll just give "nc_set_log_level(5, err)" a try in
the f90 code.  I am using the Version 4.1.3 NetCDF Installation and Porting
Guide and Fortran 90 Interface Guide, among others.  Am I missing a useful
source of documentation?

If this doesn't work, fire up the parallel debugger and see where HDF5 and
netCDF are failing to get along...

That is my normal inclination.  However, the code isn't crashing.  put_var()
is just returning a ""NetCDF: HDF error" and continuing on its way - with
data missing in the file.  The parameters look the same as other valid
iterations (I am trapping the error and printing them out) so I am not sure
what to look for at that point.  Any clues appreciated.
I have tried the TotalView debugger with the libraries and it
is really a nightmare if you don't know the code intimately.  The
problem is that there a lot of recursive calls which make
debugging more difficult.  This is not a trivial library to
debug.  I spend several days looking and I was not able to
find the source of my parallel problem.

Our parallel I/O in ROMS is broken with the new versions of
the NetCDF library. It works only with NetCDF 4.1.1.
Several changes were done after that and the parallel I/O no
longer works with independent I/O access .  I need to
go back to the parallel debugger when I get the chance. This
is the third time that it happens so it is a little annoying.

Still, the performance of parallel I/O with the NetCDF4/HDF
libraries is not that great... I cannot even match the
performance of serial I/O. There is room for a lot of
improvements.  I did noticed a lot of inefficient MPI
communications during my debugging session.  In my opinion,
the parallel stuff needs to be redesigned and written from
scratch to see if we can improve the performance.

It is nice to know about the nc_set_log_level call to get
detailed information.

Best, H

----------------------------------------------------------------------
Hernan G. Arango           Institute of Marine and Coastal Sciences
arango@xxxxxxxxxxxxxxxxxx  Rutgers University
off: (732) 932-6555 x266   71 Dudley Road
FAX: (732) 932-6520        New Brunswick, NJ 08901-8521, USA

http://marine.rutgers.edu/po/arango
http://www.myroms.org
http://www.ocean-modeling.org