[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #XIX-708449]: Memory leaks using netcdf4

Ludovic,

I copied your tst_mem_leak.c into the libsrc4 directory of 4.1.2-rc1 and 
was able to compile and link it with on a Linux system on which 4.1.2-rc1
had been installed using

   $ cc -c $(nc-config --cflags) -I. -I.. -I../include tst_mem_leak.c
   $ cc -o tst_mem_leak tst_mem_leak.o $(nc-config --libs)

and it ran fine, producing the output

   $ LD_LIBRARY_PATH=/usr/local/lib:/share/ed/local/spike/lib ./tst_mem_leak
   *** Testing netcdf-4 variable with unlimited dimensions.

after which I interrupted it.  I then changed it to run the main loop
1000 times (100 sec) so that it would call exit, and ran it under valgrind
to detect memory leaks or access problems.  The output was:

   $ LD_LIBRARY_PATH=/usr/local/lib:/share/ed/local/spike/lib valgrind 
./tst_mem_leak
   ==31909== Memcheck, a memory error detector
   ==31909== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
   ==31909== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info
   ==31909== Command: ./tst_mem_leak
   ==31909== 

   *** Testing netcdf-4 variable with unlimited dimensions.
   *** Testing file with one var, one unlim dim...ok.
   *** Tests successful!
   ==31909== 
   ==31909== HEAP SUMMARY:
   ==31909==     in use at exit: 0 bytes in 0 blocks
   ==31909==   total heap usage: 6,488 allocs, 6,488 frees, 2,114,762 bytes 
allocated
   ==31909== 
   ==31909== All heap blocks were freed -- no leaks are possible
   ==31909== 
   ==31909== For counts of detected and suppressed errors, rerun with: -v
   ==31909== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4)

indicating no detectable memory leaks.  (The LD_LIBRARY_PATH settings are
so the right shared library will be found.  I could have avoided these with 
link-time arguments to put the location of the shared library in the
executable.)

What kind of error are you getting when calling nc_create?  How are you
determining the existence of a memory leak?

--Russ

> Le 17/03/2011 18:28, Unidata netCDF Support a écrit :
> > Hi Ludovic,
> >
> >> I am working with netcdf4.1.1 and hdf5-1.8.6 (configure with
> >> --enable-using-memchecker).
> >>
> >> My process creates a netcdf4 file, receives data and adds it to the file.
> >> It can run during several days.
> >> 2 problems with that:
> >> 1) My file has 42 variables, each takes 8Mo in RAM so the memory
> >> consumption of my process is more than 336Mo.
> >> 2) Moreover each call to nc_put_var takes memory that is not released.
> >>
> >> To solve the first point each time I do:
> >> nc_def_var(_fileID, _varName, _varType, 1, _varDim,&l_varID)
> >> I have added this line:
> >> nc_set_var_chunk_cache(_fileID, l_varID, 0, 0, 0.75)
> >> (as described in this post:
> >> http://www.unidata.ucar.edu/mailing_lists/archives/netcdfgroup/2010/msg00169.html)
> > That's still the recommended way to avoid wasting too much memory with chunk
> > caches.
> >
> >> And I still have the second problem: each call to nc_put_var take memory
> >> that is not released.
> >>
> >> How can I avoid these memory leaks?
> > I believe the second problem is fixed in the just announced netCDF-4.1.2 
> > release
> > candidate, which now passes all of our tests using valgrind with no memory 
> > leaks.
> >
> > If it's convenient, we'd like to hear whether 4.1.2-rc1 fixes the memory 
> > leak
> > problems you're seeing.
> >
> > --Russ
> >
> >
> > Russ Rew                                         UCAR Unidata Program
> > address@hidden                      http://www.unidata.ucar.edu
> >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: XIX-708449
> > Department: Support netCDF
> > Priority: Normal
> > Status: Closed
> >
> Hi Russ,
> I think I have download a bad netcdf-4.1.2 rc1 : (I have download it
> here:
> http://www.sfr-fresh.com/unix/misc/netcdf-4.1.2-rc1.tar.gz/index_tp.html)
> 
> So I have tried with the netcdf-4.1.2 rc1 available here:
> ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4.1.2-rc1.tar.gz
> But the test I have sent you (tst_mem_leak.c) failed at this line:
> if (nc_create(FILE_NAME, NC_NETCDF4, &ncid)) ERR;
> 
> Regards
> Ludovic
> 
> 

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: XIX-708449
Department: Support netCDF
Priority: Normal
Status: Closed