Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
John Caron wrote:
Hi Jon and Nick: Ive been considering whether an object cache like ehcache would be the way to go, but havent had time to investigate. 1. Since NetcdfFileCache/NetcdfDatasetCache encapsolates an open file handle, I need to control the number of such handles. Ive assumed that i want to avoid the overhead of opening and closing files, but havent actually timed this (BAD programmer!). 2. The first few versions of netcdf java were serializable, but ive stopped supporting that for various reasons (eg Bloch "Effective Java" item 54). Could be revisted, but serialization is a pain. "Do not implement Serializable lightly, since it restricts future flexibility, and publicly exposes class implementation details which are usually private. As well, implementing Serializable correctly is not trivial." "Serialization demands not only backward compatibility, but forward compatibility. You might have people running slightly older versions of software needing to read the newer versions, and vice versa. Serialization incompatibilities are even more serious than API incompatibilities: we are talking about customers not being able to read data that they innocently stored away with a previous version of your software!"
Random thoughts -I think those comments above are wise in urging caution for people expecting to use serialization 1) long-term persistence with something like built-in ObjectOutputStream, 2) serialization for any form of exchange between software components.
But regarding the concerns with migration issues, we're not talking about long-term persistence here or data exchange (eg messaging) - more a very short-term store that is highly disposable and likely to be discarded and rebuilt upon any migration. And my understanding is that, given the need to read/write netcdf files across a variety of platforms, the point is that the format does not change within a version - v3 is v3 as specified (I'm guessing).
Perhaps the IOSPs provide some method for serialziation already? It doesn't have to be just using EHCache/ObjectOutputStream directly on NetcdfDataset of course. You could be caching a more abstract notion of a netcdf file that lazy instantiates and re-opens file handle as needed. May require some refactoring though as having NetcdfDataset directly extend NetcdfFile could be an issue.
Sorry I can't be of more help. To be honest I don't have a great understanding of the code internals.
netcdf-java archives: