Re: [netcdf-java] Opening a GridDataset for large NcML aggregations

• To: netcdf-java <netcdf-java@xxxxxxxxxxxxxxxx>
• Subject: Re: [netcdf-java] Opening a GridDataset for large NcML aggregations
• From: John Caron <caron@xxxxxxxxxxxxxxxx>
• Date: Mon, 17 Mar 2008 20:21:32 -0600

HiJon:

Are these JoinExisting or JoinNew ? How is the coordinate value specified? You 
are doing this in your own application, not in TDS?

Jon Blower wrote:

Hi all,

Using the Java NetCDF libraries, version 2.2.22, we have notice that
opening a GridDataset using TypedDatasetFactory.open() is slow when
the dataset in question is an NcML aggregation over a long timeseries
(around a thousand timesteps, with one timestep per file).  The call
to TDF.open() takes around 3 seconds every time, which is a problem
for our application, where performance is important.

We are using the NetcdfDatasetCache, which doesn't seem to help with
this problem.  Is there something we can do to speed up the process of
creating a GridDataset?  I don't think this was a problem for us in
previous library versions, where we used a different syntax
("GridDataset gd = new GridDataset(nc);").

I guess we could cache the GridDataset object in memory at the
application level, but is there a reason not to do this (e.g. if this
is a very large object)?

Here's what we do using version 2.2.22:

   NetcdfDataset nc = NetcdfDataset nc = NetcdfDatasetCache.acquire(filename);
   GridDataset gd =
(GridDataset)TypedDatasetFactory.open(DataType.GRID, nc, null, null);

Thanks, Jon

• Follow-Ups:
  • Re: [netcdf-java] Opening a GridDataset for large NcML aggregations
    • From: Jon Blower

• References:
  • [netcdf-java] Opening a GridDataset for large NcML aggregations
    • From: Jon Blower