[netcdf-java] Opening a GridDataset for large NcML aggregations

• To: netcdf-java <netcdf-java@xxxxxxxxxxxxxxxx>
• Subject: [netcdf-java] Opening a GridDataset for large NcML aggregations
• From: "Jon Blower" <jdb@xxxxxxxxxxxxxxxxxxxx>
• Date: Mon, 17 Mar 2008 11:54:31 +0000

Hi all,

Using the Java NetCDF libraries, version 2.2.22, we have notice that
opening a GridDataset using TypedDatasetFactory.open() is slow when
the dataset in question is an NcML aggregation over a long timeseries
(around a thousand timesteps, with one timestep per file).  The call
to TDF.open() takes around 3 seconds every time, which is a problem
for our application, where performance is important.

We are using the NetcdfDatasetCache, which doesn't seem to help with
this problem.  Is there something we can do to speed up the process of
creating a GridDataset?  I don't think this was a problem for us in
previous library versions, where we used a different syntax
("GridDataset gd = new GridDataset(nc);").

I guess we could cache the GridDataset object in memory at the
application level, but is there a reason not to do this (e.g. if this
is a very large object)?

Here's what we do using version 2.2.22:

   NetcdfDataset nc = NetcdfDataset nc = NetcdfDatasetCache.acquire(filename);
   GridDataset gd =
(GridDataset)TypedDatasetFactory.open(DataType.GRID, nc, null, null);

Thanks, Jon

-- 
--------------------------------------------------------------
Dr Jon Blower Tel: +44 118 378 5213 (direct line)
Technical Director Tel: +44 118 378 8741 (ESSC)
Reading e-Science Centre Fax: +44 118 378 6413
ESSC Email: jdb@xxxxxxxxxxxxxxxxxxxx
University of Reading
3 Earley Gate
Reading RG6 6AL, UK
--------------------------------------------------------------

• Follow-Ups:
  • Re: [netcdf-java] Opening a GridDataset for large NcML aggregations
    • From: John Caron