[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #UZM-707371]: Possible problem with zlib on OS X

> > Hello,
> >
> > I have a quick question about netCDF4 on OS X. Specifically, the
> > performance of reading netCDF4 files. This appears to be extremely
> > slow; in the following (approximate) timings, I am using a GHRSST
> > format AMSRE SST file in netCDF3 format (amsre.nc3) and the same file
> > converted manually to a netCDF4 file (with zlib on, shuffle on and
> > chunking off), called amsre.nc4.
> >
> > I have BOTH the nc3 and nc4 libraries and utillities installed (so I
> > call them ncdump3 / ncdump4....)
> >
> > Time to ncdump3 amsre.nc3 -> about 1 second
> > Time to ncdump4 amsre.nc4 (with zlib off) -> about 2 seconds
> > Time to ncdump4 amsre.nc4 (with zlib on) -> 4 MINUTES!!!!!!
> > Time to gzip amsre.nc3 -> about 1 second
> >
> > Clearly, there is a problem with the zlib implementation on my version
> > of netCDF4 (4.0.3-beta). It takes about 300 times longer to read the
> > netCDF4 file with the ncdump4, than to convert it to netCDF3 and read
> > it from there!
> >
> > I am using netcdf3 (3.6.3), apple OS X 10.5.6 (intel)
> >
> > I don't doubt that I have just missed something, but just in case, I
> > thought I should report this.
> >
> > Do have any suggestions?
> >
> > Many thanks
> >
> > Dave Poulter
> >
> > David J. S. Poulter, Satellite Oceanographer
> > National Oceanography Centre, Southampton
> > European Way, Southampton, SO14 3ZH, UK
> > Tel: +44(0)23 80596107
> > E-mail: address@hidden
> >
>


Howdy Dave!

Any progress on this? Did you resolve your performance problems with netCDF-4
by setting chunksizes?

I suggest you get the just-released version 4.0.1 which offers several
performance enhancements that might help your performance. Get the release
here:
ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4.0.1.tar.gz

If you try the new release, please let me know if it works for you, and if it
helps with your performance.

Thanks,

Ed

Ticket Details
===================
Ticket ID: UZM-707371
Department: Support netCDF
Priority: Normal
Status: Closed