Re: [netcdf-hdf] what the heck is fletcher32 all about, anyway?

Ed,


At 10:39 AM -0700 2/21/08, Ed Hartnett wrote:
Elena Pourmal <epourmal@xxxxxxxxxxxx> writes:

 Ed,

Here is some documentation on Fletcher32 http://www.hdfgroup.uiuc.edu/RFC/HDF5/ARCHIVE/H5checksum/

 This document describes the final design and implementation

 http://www.hdfgroup.uiuc.edu/RFC/HDF5/ARCHIVE/H5checksum/EDC_spec.htm

Checksum is ON by default, i.e read operation will fail, but there is an API that allows you to disable checksum and get data back.

 Elena


OK, thanks!

Is the following correct:

Turning on the fletcher32 checksum filter causes a 4 byte-checksum to
be written with each chunk.

Yes, this is correct.

(What happens to contiguous data?)

Any filter in HDF5 requires chunking storage; one cannot (at least now) apply checksum to contiguous data.

When reading data with fletcher32 checksum enabled, any corruption of
the chunk will cause the read operation to fail.

There is a significant performance penalty when reading checksummed
data.

Is this all correct?

Yes, it is correct, but I believe "significant " will vary from system to system.

Elena

Thanks!

Ed


--
Ed Hartnett  -- ed@xxxxxxxxxxxxxxxx


--

------------------------------------------------------------
Elena Pourmal
The HDF Group
1901 So First ST.
Suite C-2
Champaign, IL 61820

epourmal@xxxxxxxxxxxx
(217)333-0238 (office)
(217)333-9049 (fax)
------------------------------------------------------------