Unidata - To provide the data services, tools, and cyberinfrastructure leadership that advance Earth system science, enhance educational opportunities, and broaden participation. Unidata
         
  advanced  
 

Compressed netCDF and sparse data

Greetings!

I recently noted with interest the April announcement
that a user had achieved transparent compression within
the netCDF library by patching the library and linking
with the zlib compression library. This might provide
a solution to a sparse data storage problem I am currently
looking at.

I have been unable to successfully access the source
of the netCDF 3.31 library patch via the link provided
at http://snow.cit.cornell.edu/noon/z_netcdf.html.
(The file download URL is
http://snow.cit.cornell.edu/noon/znetcdf.tar.gz)

I can successfully download the file with a shift-click,
however the resulting file cannot be uncompressed with
WinZip (undoes the gz OK but reports a corrupted .tar file),
nor can I get to the inner tar file by using gzip, which
reports "invalid compressed data -- CRC error". Am I getting
garbled by the Windows linefeed convention? I would expect
that everything is binary except the innermost files. We have
been unable to get to the file via anonymous FTP to related
sites. I have been unable to locate an email address for
Bill Noon.

Has anybody else been able to get to the patch code?
I am also unable to locate any source at the unidata
FTP site under the "contrib" subdirectory.

Has any consideration been given to adopting compressed
netCDF as a standard part of the netCDF package?

Another option we had considered for the sparse data
problem would involve using a netCDF file as a more generalized
"header" to store/recover access points within an XDR file, as a 
means of getting around the fixed record size limitation.

Thanks sincerely,

Jeremy Beal
jeremyb@xxxxxxxxxxxxx
Scientific Software-Intercomp, Inc.

 
 
  Contact Us     Site Map     Search     Terms and Conditions     Privacy Policy     Participation Policy
 
National Science Foundation (NSF) UCAR Community Programs   Unidata is a member of the UCAR Community Programs, is managed by the University Corporation for Atmospheric Research, and is sponsored by the National Science Foundation.
P.O. Box 3000     Boulder, CO 80307-3000 USA     Tel: 303-497-8643     Fax: 303-497-8690