Re: [netcdf-hdf] what the heck is fletcher32 all about, anyway?


At 10:39 AM -0700 2/21/08, Ed Hartnett wrote:
Elena Pourmal <epourmal@xxxxxxxxxxxx> writes:


Here is some documentation on Fletcher32

 This document describes the final design and implementation

Checksum is ON by default, i.e read operation will fail, but there is an API that allows you to disable checksum and get data back.


OK, thanks!

Is the following correct:

Turning on the fletcher32 checksum filter causes a 4 byte-checksum to
be written with each chunk.

Yes, this is correct.

(What happens to contiguous data?)

Any filter in HDF5 requires chunking storage; one cannot (at least now) apply checksum to contiguous data.

When reading data with fletcher32 checksum enabled, any corruption of
the chunk will cause the read operation to fail.

There is a significant performance penalty when reading checksummed

Is this all correct?

Yes, it is correct, but I believe "significant " will vary from system to system.




Ed Hartnett  -- ed@xxxxxxxxxxxxxxxx


Elena Pourmal
The HDF Group
1901 So First ST.
Suite C-2
Champaign, IL 61820

(217)333-0238 (office)
(217)333-9049 (fax)