John Caron wrote:
Hi Jon and Nick:
Ive been considering whether an object cache like ehcache would be the way to
go, but havent had
time to investigate.
1. Since NetcdfFileCache/NetcdfDatasetCache encapsolates an open file handle, I
need to control the
number of such handles. Ive assumed that i want to avoid the overhead of
opening and closing files,
but havent actually timed this (BAD programmer!).
2. The first few versions of netcdf java were serializable, but ive stopped
supporting that for
various reasons (eg Bloch "Effective Java" item 54). Could be revisted, but
serialization is a pain.
"Do not implement Serializable lightly, since it restricts future flexibility,
and publicly exposes
class implementation details which are usually private. As well, implementing
Serializable correctly
is not trivial."
"Serialization demands not only backward compatibility, but forward
compatibility. You might have
people running slightly older versions of software needing to read the newer
versions, and vice
versa. Serialization incompatibilities are even more serious than API
incompatibilities: we are
talking about customers not being able to read data that they innocently stored
away with a previous
version of your software!"
Random thoughts -
I think those comments above are wise in urging caution for people
expecting to use serialization 1) long-term persistence with something
like built-in ObjectOutputStream, 2) serialization for any form of
exchange between software components.
But regarding the concerns with migration issues, we're not talking
about long-term persistence here or data exchange (eg messaging) - more
a very short-term store that is highly disposable and likely to be
discarded and rebuilt upon any migration. And my understanding is that,
given the need to read/write netcdf files across a variety of platforms,
the point is that the format does not change within a version - v3 is v3
as specified (I'm guessing).
Perhaps the IOSPs provide some method for serialziation already? It
doesn't have to be just using EHCache/ObjectOutputStream directly on
NetcdfDataset of course. You could be caching a more abstract notion of
a netcdf file that lazy instantiates and re-opens file handle as
needed. May require some refactoring though as having NetcdfDataset
directly extend NetcdfFile could be an issue.
Sorry I can't be of more help. To be honest I don't have a great
understanding of the code internals.
