[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #ZFB-587742]: Reading variable with strides very slow

Patrick,

> On the netcdf-3 vs netcdf-4 issue I can at the moment
> only speculate. As a rule, reading small quantities of data
> with netcdf-4 is always slower than netcdf-3 because the
> underlying HDF5 file format is based on b-trees rather than the
> linear disk layout of netcdf-3. Since vars reads a single
> element at a time, that overhead can, I suspect, be significant.
> I am, however surprised that it is as large as you show.

In this case, no b-trees are involved, because the data storage is
contiguous, not chunked (according to ncdump -h -s).  So I'm
surprised how slow the strided netCDF access is, and suspect there
might be a performance bug in how netCDF-4 uses the HDF5 API for
strided access.

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: ZFB-587742
Department: Support netCDF
Priority: Normal
Status: Closed