Re: another chunking question - when should chunking NOT be used

NOTE: The netcdf-hdf mailing list is no longer active. The list archives are made available for historical reasons.

Muqun (Kent) Yang wrote:

At 02:23 PM 12/16/2003 -0700, John Caron wrote:

Ed Hartnett wrote:

Howdy all!

Another question relating to chunking - if we don't need it (i.e. for
a dataset with no unlimited dimensions), do we still chunk it?

Or is it better to leave it contiguous?

(With the mental reservation that only chunked datasets will be able
to take advantage of compression, when we get to that feature.)



i would be inclined to say that the default should be the old netcdf layout (continuous) just because that is optimal for sequential processing, which is a reasonnable assumption. but the previously suggested 3 modes could still be used, regardless of whether unlimited dimesnions are used or not.

HDF5 supports compression features only when chunking is turned on. So with only contiguous storage, the compression can not be done. That should be a factor to consider.


good point. so compression is a fourth mode.

  • 2003 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-hdf archives: