Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
The latest release of netcdf java (http://www.unidata.ucar.edu/packages/netcdf-java/) has a contributed class InMemRandomAccessFile.java which does what you want. We havent finished integrating it by adding a NetcdfFile constuctor for it, but that wouldnt be hard (see public NetcdfFile(URL url)). I could add that if you are interested, and willing to test it. ----- Original Message ----- From: "Russ Rew" <russ@xxxxxxxxxxxxxxxx> To: "ahoward" <ahoward@xxxxxxxxxxxx> Sent: Monday, April 01, 2002 4:47 PM Subject: Re: in memory netcdf > >To: russ@xxxxxxxxxxxxxxxx > >From: ahoward <ahoward@xxxxxxxxxxxx> > >Subject: Re: 20020327: in memory netcdf > >Organization: NOAA/FSL > > Hi Ara, > > > don't know if you recall my name, but i was one of the early users of your > > netcdf java library a few years back. in any case, i've switched > > departments here at fsl and am now in the ITS group responsible for the > > realtime data ingest. we're currently in the process of revamping our > > systems design and implementation language (c -->> c++). i'm working on > > generic handling of point data and need a good general data structure in > > which to store all point data for eventual munging into output files of > > either netcdf or grib format. netcdf itself would be an ideal data > > structure as the metadata contained in the file is complete enough for my > > purposes (with some creative uses of attributes) and the typed accessor > > methods (c++ interface) are exactly the type of thing needed. > > > > my problem is that any use of netcdf requires a disk hit to read in an > > existing cdf (output of ncgen) and each subsequent put/get also hits disk. > > i'm looking into replacing the IO layer (posixio.c) with a layer that > > operates entirely in memory. that would leave only the hit to read in an > > existing template (i'm also planning on looking at ncgen to see if it > > could be modified to place a netcdf file structure in memory instead of on > > disk). eg. i want to eliminate most, if not all, of the disk accesses in > > the c/c++ netcdf api. > > > > alternatively, applying the same set of requirements, and availibility of > > a constructor to the NcVar class might prove sufficient for our purposes. > > > > my questions are : > > > > * do you know of any work along these lines, 3rd party or > > otherwise? > > Andrew Janke <janke@xxxxxxxxxxx> proposed implementing something > similar in the Java interface, which you can read about in this reply > from John Caron to his question: > > http://www.unidata.ucar.edu/cgi-bin/mfs/70/4439 > > Glenn Davis also once implemented an experimental memory-mapped > version of the i/o layer for netCDF 3.4 built on top of Unix mmap(2), > which is described here: > > http://www.unidata.ucar.edu/cgi-bin/mfs/70/2756 > > and available in this file: > > ftp://ftp.unidata.ucar.edu/pub/netcdf/contrib/mmapio.c > > that saves some disk I/O. > > > * am i completely off base in thinking the IO layer could be > > replaced with an in memory IO layer > > > > any input appreciated, i was planning on cc'ing john caron on this as well > > but have lost his email. > > The I/O layer could be replaced with an in-memory layer. But if you > just want to access a small slice of data from a huge file, using > random-access to read from the disk is cheaper than reading the whole > file into memory first before you try to access the data slice from > the in-memory copy. But it sounds like that's not a concern for your > application. > > By the way, Charlie Zender has developed a new C++ interface to > netCDF, known as libnco_c++, which he describes in this netcdfgroup > posting: > > http://www.unidata.ucar.edu/glimpse/netcdfgroup-list/1484 > > which emphasizes easy migration from the C interface ... > > --Russ >
netcdf-java
archives: