Hi Yucheng, > I would like to read a variable out from a netcdf file and apply compression > on that then write everything back, > Do you have a sample code to do that? It is a bit messy to read other > variables out, so I would prefer not reading > Other variables if possible. We don't have sample code for that, but you can use the "nccopy" utility that comes with netCDF version 4.2 to compress all the variables in a netCDF file and write the result to another file, like this nccopy -d 1 input.nc output.nc which will apply compression level 1 to the variables in input.nc and write to output.nc. --Russ > -----Original Message----- > From: Unidata netCDF Support [mailto:address@hidden > Sent: Wednesday, February 15, 2012 5:47 PM > To: Song, Yucheng > Cc: address@hidden > Subject: [netCDF #PRD-617628]: Large sized netcdf-4 file > > Hi, > > > Also, I couldn't find an example to show how to or classic_format and > > hdf5 to create the > > Netcdf-4 classic format. > > nf90_create(FILE_NAME, (nf90_classic_model|nf90_hdf5) doesn't work. > > Sorry, the documentation should be clearer. You can use: > > nf90_create(FILE_NAME, IOR(nf90_netcdf4, nf90_classic_model), ncid) > > > The confusion from netcdf is that it is vague from the online document > > on how to generate a > > Netcdf4 classic - when compiling the library for this, does one need hdf5 > > lib at all? > > Yes, the hdf5 library is needed becasue a netCDF-4 classic model file is > really an HDF5 file with an artifact to make sure it is readable by old > programs lijnked to the netCDF-4 library. > For an explanation, see this FAQ, and maybe some of the subsequent questions > and answers: > > http://www.unidata.ucar.edu/netcdf/docs/faq.html#fv1 > > > Using your online example pres_temp_4D_wr.f90, I tried to modify the > > line into call check( nf90_create(FILE_NAME, nf90_hdf5, ncid) ) > > > > now when I look at the file sizes, it is significanly larger (7 times), I > > thought hdf5 should be smaller, anything wrong? > > > > -rw-rw-r-- 16867 Feb 10 10:55 pres_temp_4D.nc > > -rw-rw-r-- 2784 Feb 10 10:38 pres_temp_4D.nc_nocompress > > netCDF-4 files are only smaller if compression is used when writing the data. > > Here's an answer to an FAQ on that: > > http://www.unidata.ucar.edu/netcdf/docs/faq.html#fv8 > > Compression is done on a variable-by-variable basis, so it's possible to use > different compression levels on different variables. But if you just want to > compress all variables in the file using the same compression level, the > easiest way is using the nccopy utility, as in > > nccopy -d1 foo.nc foo-compressed.nc > > to use "deflation level" 1. > > Note that for small files, the compressed netCDF-4 files will actually be > larger than uncompressed netCDF-3 (classic format) files, because the > underlying HDF5 format has some fixed-size overhead. But for large datasets, > compression will save space, at the expense of taking more time to read and > write the data. > > I just tested the 4.1.3 nccopy on an example file for compression and got > this: > > $ nccopy -d1 llnl.nc llnl-compressed.nc && ls -l llnl*.nc > -rw-rw-r-- 1 russ ustaff 13802 Feb 15 14:58 llnl-compressed.nc > -rw-rw-r-- 1 russ ustaff 32864 Oct 17 11:05 llnl.nc > > --Russ > > > Russ Rew UCAR Unidata Program > address@hidden http://www.unidata.ucar.edu > > > > Ticket Details > =================== > Ticket ID: PRD-617628 > Department: Support netCDF > Priority: Normal > Status: Closed > > > Russ Rew UCAR Unidata Program address@hidden http://www.unidata.ucar.edu Ticket Details =================== Ticket ID: PRD-617628 Department: Support netCDF Priority: High Status: Closed
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.