Re: no chunking or deflate for scalar values in HDF5?

[Quincey Koziol <koziol@xxxxxxxxxxxx>]

Hi Ed,

On Jul 12, 2007, at 6:42 AM, Ed Hartnett wrote:

> Howdy all HDF5ers!
>
> I have been asked by a user whether the zlib deflate can be used on
> scalar datasets.
>
> I believe from my experiments in HDF5 that it cannot, because there
> is no way to set the chunking for a scalar. Is this correct?
>
> I don't believe this is discussed in the reference manual - am I
> missing it?
>
> The code below, for example, fails when I try to set the chunking.
>
> Is this the correct conclusion, or am I missing something?

	Interesting - not something I thought about trying, since it's not  
very useful in general.  I suppose it could be beneficial if the  
datatype for the dataset was large enough that compression saved some  
space (although those gains would be offset by the B-tree overhead  
for indexing the chunks).

	It is not currently supported, since a scalar dataspace corresponds  
to a 0-dimensional "point", and chunks must have dimensions >=1.  If  
this is critically important, I can work on the modifications to the  
code to allow this to work, but it would require changes in a number  
of places and I'd rather not go that direction right now...  :-)

	Quincey

> Thanks!
>
> Ed
>
>    printf("*** Checking HDF5 scalar variable compession...");
>
> #define MAX_NAME_LEN 50
> #define DEFLATE_LEVEL 3
> #define SIMPLE_VAR_NAME1 "punches"
>    {
>       hid_t fapl_id, fcpl_id;
>       hid_t datasetid;
>       hid_t fileid, grpid, spaceid, plistid;
>       int data_in, data_out = 42;
>       hsize_t chunksize = 1;
>
>       /* Create file, setting latest_format in access propertly list
>        * and H5P_CRT_ORDER_TRACKED in the creation property list. */
>       if ((fapl_id = H5Pcreate(H5P_FILE_ACCESS)) < 0) ERR;
>       if (H5Pset_latest_format(fapl_id, 1) < 0) ERR;
>       if ((fcpl_id = H5Pcreate(H5P_FILE_CREATE)) < 0) ERR;
>       if (H5Pset_link_creation_order(fcpl_id, H5P_CRT_ORDER_TRACKED| 
> H5P_CRT_ORDER_INDEXED) < 0) ERR;
>       if ((fileid = H5Fcreate(FILE_NAME, H5F_ACC_TRUNC, fcpl_id,  
> fapl_id)) < 0) ERR;
>
>       if ((grpid = H5Gopen(fileid, "/")) < 0) ERR;
>
>       if ((spaceid = H5Screate(H5S_SCALAR)) < 0) ERR;
>
>       /* Create property lust. */
>       if ((plistid = H5Pcreate(H5P_DATASET_CREATE)) < 0) ERR;
>
>       if (H5Pset_chunk(plistid, 1, &chunksize) < 0)ERR;
>
>       /* Set up compression. */
>       if (H5Pset_deflate(plistid, DEFLATE_LEVEL) < 0) ERR;
>
>       /* Create the variable. */
>       if ((datasetid = H5Dcreate(grpid, SIMPLE_VAR_NAME1,  
> H5T_NATIVE_INT,
> 				 spaceid, plistid)) < 0) ERR;
>
>       /* Write the data. */
>       if (H5Dwrite(datasetid, H5T_NATIVE_INT, H5S_ALL, H5S_ALL,
> 		   H5P_DEFAULT, &data_out) < 0) ERR;
>
>       if (H5Dclose(datasetid) < 0) ERR;
>       if (H5Pclose(fapl_id) < 0 ||
> 	  H5Sclose(spaceid) < 0 ||
> 	  H5Gclose(grpid) < 0 ||
> 	  H5Fclose(fileid) < 0)
> 	 ERR;
>
>       /* Now reopen the file and check. */
>       if ((fapl_id = H5Pcreate(H5P_FILE_ACCESS)) < 0) ERR;
>       if (H5Pset_latest_format(fapl_id, 1) < 0) ERR;
>       if ((fileid = H5Fopen(FILE_NAME, H5F_ACC_RDONLY, fapl_id)) <  
> 0) ERR;
>       if ((grpid = H5Gopen(fileid, "/")) < 0) ERR;
>
>       if ((datasetid = H5Dopen(grpid, SIMPLE_VAR_NAME1)) < 0) ERR;
>       if ((spaceid = H5Dget_space(datasetid)) < 0)
>       if (H5Dread(datasetid, H5T_NATIVE_INT, H5S_ALL,
> 		  spaceid, H5P_DEFAULT, &data_in) < 0) ERR;
>
>       /* Check the data. */
>       if (data_in != data_out) ERR;
>
>       if (H5Pclose(fapl_id) < 0 ||
> 	  H5Dclose(datasetid) < 0 ||
> 	  H5Gclose(grpid) < 0 ||
> 	  H5Fclose(fileid) < 0)
> 	 ERR;
>    }
>    SUMMARIZE_ERR;
>
>
> --
> Ed Hartnett  -- ed@xxxxxxxxxxxxxxxx
>
> ====================================================================== 
> =========
> To unsubscribe netcdf-hdf, visit:
> http://www.unidata.ucar.edu/mailing-list-delete-form.html
> ====================================================================== 
> =========

Attachment: smime.p7s
Description: S/MIME cryptographic signature