Aggregations are done when NetcdfDataset is opened, which should get cached. But I remember noticing
that the aggregation was not getting cached. I will check it out.
Jon Blower wrote:
Hi John,
Yes, I thought it might be the scanning that takes the time. I had
assumed that the scanning takes place when you open a NetcdfDataset,
but I guess I'm wrong? Perhaps this implies that I need a
GridDatatypeCache as well as a NetcdfDatasetCache? (Caching is making
my head hurt, there are too many in my application already ;-)
Cheers, Jon
On Tue, Mar 18, 2008 at 4:22 PM, John Caron <caron@xxxxxxxxxxxxxxxx> wrote:
that should all work. the time may be in the scanning of 1000 files. Still, I
have had other reports
of extra slowness, I will investigate when I can.
Jon Blower wrote:
> Hi John,
>
> It's a JoinNew, because the data files do not contain a time axis.
> Hence we use a <scan> element and the time axis values are deduced
> from the file names using dateFormatMark. Our ncml file is attached.
>
> Yes, this is in our own application (ncWMS) not the TDS.
>
> Cheers, Jon
>
> On Tue, Mar 18, 2008 at 2:21 AM, John Caron <caron@xxxxxxxxxxxxxxxx> wrote:
>> HiJon:
>>
>> Are these JoinExisting or JoinNew ? How is the coordinate value specified?
You are doing this in your own application, not in TDS?
>>
>>
>> Jon Blower wrote:
>> > Hi all,
>> >
>> > Using the Java NetCDF libraries, version 2.2.22, we have notice that
>> > opening a GridDataset using TypedDatasetFactory.open() is slow when
>> > the dataset in question is an NcML aggregation over a long timeseries
>> > (around a thousand timesteps, with one timestep per file). The call
>> > to TDF.open() takes around 3 seconds every time, which is a problem
>> > for our application, where performance is important.
>> >
>> > We are using the NetcdfDatasetCache, which doesn't seem to help with
>> > this problem. Is there something we can do to speed up the process of
>> > creating a GridDataset? I don't think this was a problem for us in
>> > previous library versions, where we used a different syntax
>> > ("GridDataset gd = new GridDataset(nc);").
>> >
>> > I guess we could cache the GridDataset object in memory at the
>> > application level, but is there a reason not to do this (e.g. if this
>> > is a very large object)?
>> >
>> > Here's what we do using version 2.2.22:
>> >
>> > NetcdfDataset nc = NetcdfDataset nc =
NetcdfDatasetCache.acquire(filename);
>> > GridDataset gd =
>> > (GridDataset)TypedDatasetFactory.open(DataType.GRID, nc, null, null);
>> >
>> > Thanks, Jon
>> >
>> _______________________________________________
>> netcdf-java mailing list
>> netcdf-java@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
>>
>
>
>
