Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Thurber, Fred wrote:
I profiled our Grib code (2.2.18) and found a bottleneck in some of the low-level routines such as Grib1BinaryDataSection.bits2UInt(), ucar.unidata.io.RandomAccessFile.read(), and, to a lesser degree, in ucar.grib.GribNumbers.float4().Has the UCar team ever thought of moving to Java NIO?
we measured NIO and RandomAccessFile when NIO first came out, and there was no improvement. However, java 1.6 has had lots of improvements, so any new measurements would be welcome.I rewrote the
ucar.grib.GribNumbers.float4() method in NIO and got 10 fold performance boost. I will post the new float4() if anyone is interested.
yes, we'd like to see it.
As far as RandomAccessFile.read(), the reason our program was spending so much time in it was because of a multiplier inside the Grib1BinaryDataSection constructor. So 2,558 calls to the Grib1BinaryDataSection constructor resulted in 273,175,727 calls to RandomAccessFile.read()! Is there anything I can do about that?
since RandomAccessFile.read() is buffered, these are not actual I/O calls. Still, there may be improvements in the way Grib1BinaryDataSection works, and any help you can give would be welcome.
Increase the defaultBufferSize size? It looks as if we have it set to 8K, but I was thinking of increasing that substantially. Can I do this without a recompile?
Ive done some experiments with different buffer sizes and havent seen a big improvements with bigger sizes. OTOH, I havent targeted grib files. You can experiment by using: NetcdfFile.open(String location, int buffer_size, ucar.nc2.util.CancelTask cancelTask); Im not sure if you are using NetcdfFile or the grib library as standalone.
netcdf-java
archives: