[netcdf-java] Thread safe read access

hi there,

I'm having a question regarding thread safety and concurrency with the netcdf 
java library.

basically i'm running a thread that scans a large file to calculate overall 
statistics, and then when i try to plot that file in the meantime, i get 
spurious errors like this:

Exception in thread "AWT-EventQueue-0" 
java.lang.ArrayIndexOutOfBoundsException: -41846
        at ucar.unidata.io.RandomAccessFile.read(RandomAccessFile.java:538)
        at ucar.unidata.io.RandomAccessFile.readInt(RandomAccessFile.java:1134)
        at ucar.unidata.io.RandomAccessFile.readLong(RandomAccessFile.java:1218)
        at ucar.nc2.iosp.IospHelper.readData(IospHelper.java:153)
        at ucar.nc2.iosp.IospHelper.readDataFill(IospHelper.java:79)
        at ucar.nc2.iosp.netcdf3.N3raf.readData(N3raf.java:63)
        at ucar.nc2.iosp.netcdf3.N3iosp.readData(N3iosp.java:453)
        at ucar.nc2.NetcdfFile.readData(NetcdfFile.java:1941)
        at ucar.nc2.Variable.reallyRead(Variable.java:859)
        at ucar.nc2.Variable._read(Variable.java:831)
        at ucar.nc2.Variable.read(Variable.java:709)
        at at.iem.sysson.Implicits$SyRichVariable.readSafe(Implicits.scala:111)

The `readSafe` in the last line actually uses a `synchronized` lock on the 
file, so at least I'm making sure that no two threads are calling 
`Variable.read` at the same time. But still it seems that something is messed 

So are there any pitfalls in this approach that could result in mutable state 
being mixed up when using the file from multiple threads? Is there a technique 
to ensure consistency (other than locking calls to Variable.read)?

I have a vague feeling that the problem might be that read doesn't actually 
read the whole data, but ma2.Array uses some cache and runs actual I/O after 
Variable.read has completed. I'm using iterators on the array with 
`getNextFloat` and `getNextDouble`. Do I need to isolate them as well somehow?

Thanks a lot!

Best regards,

Hanns Holger Rutz

University of Music and Performing Arts – KUG
Institute of Electronic Music and Acoustics – IEM
Inffeldgasse 10/III
8010 Graz

  • 2013 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: