[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: netCDF/HDF much slower than netCDF?

> Organization: U.S. Geological Survey
> Keywords: 199406281738.AA11776

Hi Rich,

> I just built HDF/netCDF (the mfhdf directory in HDF3.3r3) and 
> ran NCTEST on my DEC Alpha OSF box.  NCTEST completed 
> successfully, but took over 3 minutes, while NCTEST from the 
> standard Unidata Netcdf (v 2.3.2) took only 2 seconds!!
> 
> When I did "ps" during the lengthy NCTEST run with HDF/netCDF,
> it reveals that NCTEST is spending nearly all of it's time as
> a uninterruptible sleeping process.  
> 
> 
> Is this normal behavior for NCTEST with HDF/netCDF?  Has someone
> run NCTEST with HDF/netCDF and obtained decent results?

I don't think nctest is a very good benchmark for timing the two
implementations of the netCDF interface, since it spends much more time
testing error returns and redefining and renaming things than a typical
application would do.  I developed a benchmark for reading and writing
multi-dimensional data on which the HDF-based implementation did better than
our XDR-based implementation in some cases and worse in others.  The
stand-alone benchmark is available as

    ftp://ftp.unidata.ucar.edu/pub/netcdf/nctime.c

On my SPARCstation 10 under Solaris 2.3, the results vary widely depending
on the kind of cross-section of data you are getting.  The results show that
our netCDF implementation that uses XDR is anywhere from 50 times faster to
20 times slower, depending on the operation.  These benchmark times vary
somewhat from run to run, so they aren't terribly accurate, but they show
places where both implementations could probably be tuned for better
performance.  These results may be highly platform dependent and may vary
widely with other sizes of variables; I haven't tried "nctime" on anything
else.

--
Russ Rew                                                   UCAR Unidata Program
address@hidden                                             P.O. Box 3000
http://www.unidata.ucar.edu/                             Boulder, CO 80307-3000