Re: requirements for handling chunking...

NOTE: The netcdf-hdf mailing list is no longer active. The list archives are made available for historical reasons.

Hi Ed,
    Seems reasonable.  (We _really_ need to increase the default chunk cache
size... :-)


> Howdy all!
> Based on recent emails and discussions, here's an attempt at
> requirements relating to the setting of chunksizes. This is from the
> requirements at
>     * Chunking is required in any dataset with one or more unlimited
>       dimension in HDF5. NetCDF-4 supports setting chunk parameters at
>       variable creation with the following new function:
> int nc_def_var_x(int ncid, const char *name, nc_type xtype, int ndims, 
>         const int *dimidsp, int *varidp, int chunkalg, int *chunksizes);
>       Where chunksize is a pointer to an array of size ndims, with the
>       chunksize in each dimension. If chunksizes is NULL, the user can
>       select a chunking algorithm by setting chunkalg to NC_CHUNK_SEQ
>       (to optimize for sequential access), NC_CHUNK_SUB (for chunk
>       sizes set to favor equally subsetting in any dimension.
>       When the (netcdf-3) function nc_def_var is used, a sequential
>       chunking algorithm will be used. (Just as if the var had been
>       created with NC_CHUNK_SEQ).
>       The sequential chunking algorithm sets a chunksize of 1 all
>       unlimited dimensions, and all other chunksizes to the size of
>       that dimension, unless the resulting chunksize is greater than
>       250 KB, in which case subsequent dimensions will be set to 1
>       until the chunksize is less than 250 KB (one quarter of the
>       default chunk cache size).
>       The subsetting chunking algorithm sets the chunksize in each
>       dimension to the nth root of (desired chunksize/product of n
>       dimsizes).

  • 2003 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-hdf archives: