Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.

Re: [netcdfgroup] Predictable NetCDF: HDF error

All good suggestions.
The goal is to try to isolate the problem
to see if the error is associated with HDF5, or mpi, or
the core netcdf-c library.

On 6/23/2017 2:17 PM, Gus Correa wrote:

Hi Ian, Dennis

May I suggest a workaround?
If your files are netCDF "Classic" or "64 bit offset" ('ncdump -k' will tell) they don't need 
NetCDF-4 or HDF5 support to be processed.
Am I right?
If so, you could compile the netCDF 3.6.3 library and link your program to it instead. Alternatively, you could compile netCDF 4.4.1 disabling netCDF-4 (run configure without --enable-netcdf4). 
Either one doesn't require HDF5, and *may* not have these errors.

Other than this, perhaps lowering the compiler optimization level,
or turning off OpenMP, if it is on.

I hope this helps,
Gus Correa

On 06/23/2017 03:32 PM, dmh@xxxxxxxx wrote:

Is there any chance that you can run this using valgrind?
My speculation is that it is a memory leak of some kind.
=Dennis Heimbigner
 Unidata


On 6/23/2017 7:38 AM, Ian Harris (ENV) wrote:

Hi,
I have NetCDF installed (netcdf library version 4.4.1 per ncdump) on a fairly powerful iMac (4GHz i7, 16GB). I use Fortran 77 (yes, I know, but it’s very fast).
I’m processing a lot of data, inputs and outputs all NetCDF, in loops, and it’s dying at predictable and reproducible points.
One example is a simple prog I knocked up; it opens a nc file, then continually reads the first time value in a loop until it dies.
The last successful loop is 16777187, which may sound familiar, as it’s been here before:
http://www.unidata.ucar.edu/mailing_lists/archives/netcdfgroup/2014/msg00173.html
The error is usually 'NetCDF: HDF error', though occasionally 'NetCDF: Error initializing for parallel access'.
I should note that a much more complicated program, which reads in three files and writes one, also dies like this after a fixed number of runs through the loops (taking over an hour). 

The simple test program is below.

Please can anyone hazard a guess as to what’s happening?

Cheers

Harry


Here’s the entire program:

       program netcdftestloop
c NETCDFTESTLOOP  Testing iteration counts before it dies
       implicit none
       include 'netcdf.inc'
       integer nits,st,ncid,tvarid
       real    rday

       nits = 0
       st = nf_open('Prec_daily_WFD_CRU_grid_198605.nc',0,ncid)
        if (st.ne.nf_noerr) call handle_err(st,'openff')
       st = nf_inq_varid(ncid,'time',tvarid)
        if (st.ne.nf_noerr) call handle_err(st,'inqtvi')
       do
         nits=nits+1
if (mod(nits,100000).eq.0) write(*,'(i10)')nits ! keep track st = nf_get_var1_real(ncid,tvarid,1,rday) ! get day 1 time value 
          if (st.ne.nf_noerr) then
            write(*,'("N. iterations = ",i10)')nits
            call handle_err(st,'getday')
          endif
        enddo
st = nf_close(ncid) ! totally unnecessary close file 
          if (st.ne.nf_noerr) call handle_err(st,'closef')
        stop
        end
c *********************************************************************************************** 
       subroutine handle_err(errcode,code)
       implicit none
       include 'netcdf.inc'
       integer errcode
       character code*6

       print *, 'Error: ', code, ': ', nf_strerror(errcode)
       stop
       end
c *********************************************************************************************** 

In a more complicated program, I get a different error, ('')


Ian "Harry” Harris
NCAS-Climate
Climatic Research Unit
School of Environmental Sciences
University of East Anglia
Norwich NR4 7TJ
United Kingdom
IMPORTANT NOTICE - This email is intended for the named recipient only. It may contain privileged and confidential information. If you are not the intended recipient, notify the sender immediately and destroy this email. You must not copy, distribute or take action in reliance upon it. Whilst all efforts are made to safeguard emails, The School of Environmental Sciences cannot guarantee that attachments are virus free or compatible with your systems and does not accept liability in respect of viruses or computer problems experienced. 











_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/ 



_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/ 


_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/
  • 2017 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: