[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: netCDF question follow up....

> 1. To answer some of you questions,  I think that we are currently using an
>    older version of netCDF - version 2.4.3 I think.  What level of increase
>    in speed of access do you suspect we will see going from version 2.x.x
>    to version 3.x.x?

It varies, of course with platform and application.
One of the design goals of netcdf-3 was improved performance, and
we believe we achieved significant improvement.

If you are going to build netcdf-3,
use
ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-3.4a.tar.Z.
It is the latest and greatest and about to be released.
We are still testing on supercomputers, but you don't
need to worry about that.

Netcdf-3.4 has some additional 'performance interfaces' which can
be used to tune things somewhat.

> 2. The files we are accessing are on the local disk.

Good.

> 3. I'm not exactly sure how to use the "contributed mmapio i/o layer
>    to memory map the file..."  Could you explain this further?

Netcdf-3 has the i/o layer encapsulated in such a way that different
schemes can be plugged in. On our ftp site in pub/netcdf/contrib
is a i/o layer built on top UNIX mmap(2). The mmap facility uses the
machines virtual memory system to make the file appear as if it were
all in memory, paging on demand. It eliminates a level of data copying
when compared with UNIX read(), write(), and so on.

> 4. When we read the netCDF data files we read in the whole file, one record
>    at a time, from the first record to the last record.

So the only thing to be sure of is that you read the variables within
a 'record' in index (variable id) order. Then you will be reading sequentially.

> I have included an "ncdump -h" output below the double dashed line (=====)
> of one of our (smaller) LDAD netCDF data files.
>
> Thanks a lot.

You are welcome.

-glenn