Bzip2 Compression

There was some question as to how well Bzip2 would compress floating point data. We just did a test on a large NetCDF file, containing a years worth of global, 1-degree six-hourly pressure fields, so that the data is floating point. here are the sizes from a Unix ls command (i.e in 512 byte units).

raw.nc        380,551,616

raw.nc.Z      143,716,211

raw.nc.bz2     55,633,062



Bzip2 gave about the same 7:1 compression that Russ reported, and roughly 3 times better that Unix compress.

-Roy Mendelssohn
--
Roy Mendelssohn
Pacific Fisheries Environmental Group
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: rmendels@xxxxxxxxxxxxx
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/
*** NOTE NEW AREA CODE ************

"Old age and treachery will overcome youth and skill."

  • 2000 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: