Hi John,
Yes, I thought it might be the scanning that takes the time. I had
assumed that the scanning takes place when you open a NetcdfDataset,
but I guess I'm wrong? Perhaps this implies that I need a
GridDatatypeCache as well as a NetcdfDatasetCache? (Caching is making
my head hurt, there are too many in my application already ;-)
Cheers, Jon
On Tue, Mar 18, 2008 at 4:22 PM, John Caron <caron@xxxxxxxxxxxxxxxx> wrote:
> that should all work. the time may be in the scanning of 1000 files. Still, I
> have had other reports
> of extra slowness, I will investigate when I can.
>
>
>
> Jon Blower wrote:
> > Hi John,
> >
> > It's a JoinNew, because the data files do not contain a time axis.
> > Hence we use a <scan> element and the time axis values are deduced
> > from the file names using dateFormatMark. Our ncml file is attached.
> >
> > Yes, this is in our own application (ncWMS) not the TDS.
> >
> > Cheers, Jon
> >
> > On Tue, Mar 18, 2008 at 2:21 AM, John Caron <caron@xxxxxxxxxxxxxxxx> wrote:
> >> HiJon:
> >>
> >> Are these JoinExisting or JoinNew ? How is the coordinate value
> specified? You are doing this in your own application, not in TDS?
> >>
> >>
> >> Jon Blower wrote:
> >> > Hi all,
> >> >
> >> > Using the Java NetCDF libraries, version 2.2.22, we have notice that
> >> > opening a GridDataset using TypedDatasetFactory.open() is slow when
> >> > the dataset in question is an NcML aggregation over a long timeseries
> >> > (around a thousand timesteps, with one timestep per file). The call
> >> > to TDF.open() takes around 3 seconds every time, which is a problem
> >> > for our application, where performance is important.
> >> >
> >> > We are using the NetcdfDatasetCache, which doesn't seem to help with
> >> > this problem. Is there something we can do to speed up the process of
> >> > creating a GridDataset? I don't think this was a problem for us in
> >> > previous library versions, where we used a different syntax
> >> > ("GridDataset gd = new GridDataset(nc);").
> >> >
> >> > I guess we could cache the GridDataset object in memory at the
> >> > application level, but is there a reason not to do this (e.g. if this
> >> > is a very large object)?
> >> >
> >> > Here's what we do using version 2.2.22:
> >> >
> >> > NetcdfDataset nc = NetcdfDataset nc =
> NetcdfDatasetCache.acquire(filename);
> >> > GridDataset gd =
> >> > (GridDataset)TypedDatasetFactory.open(DataType.GRID, nc, null, null);
> >> >
> >> > Thanks, Jon
> >> >
> >> _______________________________________________
> >> netcdf-java mailing list
> >> netcdf-java@xxxxxxxxxxxxxxxx
> >> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
> >>
> >
> >
> >
>
--
--------------------------------------------------------------
Dr Jon Blower Tel: +44 118 378 5213 (direct line)
Technical Director Tel: +44 118 378 8741 (ESSC)
Reading e-Science Centre Fax: +44 118 378 6413
ESSC Email: jdb@xxxxxxxxxxxxxxxxxxxx
University of Reading
3 Earley Gate
Reading RG6 6AL, UK
--------------------------------------------------------------
