[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #ZFB-587742]: Reading variable with strides very slow



Patrick,

> On the netcdf-3 vs netcdf-4 issue I can at the moment
> only speculate. As a rule, reading small quantities of data
> with netcdf-4 is always slower than netcdf-3 because the
> underlying HDF5 file format is based on b-trees rather than the
> linear disk layout of netcdf-3. Since vars reads a single
> element at a time, that overhead can, I suspect, be significant.
> I am, however surprised that it is as large as you show.

In this case, no b-trees are involved, because the data storage is
contiguous, not chunked (according to ncdump -h -s).  So I'm
surprised how slow the strided netCDF access is, and suspect there
might be a performance bug in how netCDF-4 uses the HDF5 API for
strided access.

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: ZFB-587742
Department: Support netCDF
Priority: Normal
Status: Closed


NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.