Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.

Bzip2 Compression

There was some question as to how well Bzip2 would compress floating point data. We just did a test on a large NetCDF file, containing a years worth of global, 1-degree six-hourly pressure fields, so that the data is floating point. here are the sizes from a Unix ls command (i.e in 512 byte units). 

raw.nc        380,551,616

raw.nc.Z      143,716,211

raw.nc.bz2     55,633,062
Bzip2 gave about the same 7:1 compression that Russ reported, and roughly 3 times better that Unix compress. 

-Roy Mendelssohn
--
Roy Mendelssohn
Pacific Fisheries Environmental Group
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097

e-mail: rmendels@xxxxxxxxxxxxx
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/
*** NOTE NEW AREA CODE ************

"Old age and treachery will overcome youth and skill."
  • 2000 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: