[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDFJava #FFE-268601]: Fwd: Fix for GRIB File Descriptor Leak



Hi Aaron:

Yes, I think you are right that there is a file leak here. I have to look 
closely at your changes to see if there is any collateral problems.

In the future, the easiest thing for us is to issue a pull request against that 
branch on gihub.

John 


> My apologies, I may have sent this to the wrong list.
> 
> Aaron
> 
> -------- Original Message --------
> Subject:      Fix for GRIB File Descriptor Leak
> Date:         Fri, 16 May 2014 15:22:05 -0600
> From:         Aaron Braeckel <address@hidden>
> Reply-To:     address@hidden
> Organization:         National Center for Atmospheric Research / RAL
> To:   address@hidden
> CC:   Rob Weingruber <address@hidden>
> 
> 
> 
> Hello,
> 
> Rob (CCed) emailed in February (subject: "EOFException: Reading
> file.grb2.ncx at 0 file length = 0") about a file descriptor leak
> observed with the Java netCDF when reading invalid GRIB index files.  I
> was able to reliably replicate the file leak locally by manually setting
> the index file to 0 size before opening a GRIB file.  I observed the
> file leak through the set of open file descriptors (through both the
> Java UnixOperatingSystemMXBean class and the lsof command).
> 
> The attached source files are a patch for this issue against the
> edu.ucar:grib:4.3.21 source code.  This patch is quite simple, below I
> describe the individual changes.  Would it be possible to apply these
> changes and make a new release?  If so, when would the release be made?
> We are currently maintaining our own branch of the GRIB library because
> we need this fix in operations, which is undesirable for a number of
> reasons.  I can provide a main() method that demonstrates this problem
> if it would be of interest.
> 
> Thanks!
> Aaron and Rob
> 
> ===============================
> 
> The source of the problem was in Grib1CollectionBuilder and
> Grib2CollectionBuilder (the code changes to each are identical):
> 
> In /readIndex/(String filename) a new RandomAccessFile was being created
> but was never closed.  This fix definitively resolved our file leak
> issue, and logically the created RandomAccessFile must be closed in this
> method since it is only used there.
> 
> <     return readIndex( new RandomAccessFile(filename, "r") );
> ---
> >     RandomAccessFile raf = new RandomAccessFile( filename, "r" );
> >     boolean success = readIndex( raf );
> >     raf.close();
> >     return success;
> 
> 
> In /readOrCreateIndex/() there is more code that opens a
> RandomAccessFile but may not close it.  This code did not resolve our
> issue, but is similar to the fix above and may resolve a file leak in
> other situations.  In this case I note that gc.setIndexRaf(indexRaf) is
> called just before the changed code segment, and therefore the RAF may
> be closed properly without this change.  I don't know the code and flow
> of operations well enough to be sure.
> 
> <       readIndex(indexRaf);
> ---
> >       boolean success = readIndex(indexRaf);
> >       if( !success ){
> >         indexRaf.close();
> >       }
> 
> 
> 
> 
> 
> 
> 

Ticket Details
===================
Ticket ID: FFE-268601
Department: Support netCDF Java
Priority: Normal
Status: Open


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.