Version 5.3.9 of the netCDF Operators (NCO) has been released. NCO is an Open Source package that consists of a dozen standalone, command-line programs that take netCDF files as input, then operate (e.g., derive new data, average, print, hyperslab, manipulate metadata) and output the results to screen or files in text, binary, or netCDF formats.

The NCO project is coordinated by Professor Charlie Zender of the Department of Earth System Science, University of California, Irvine. More information about the project, along with binary and source downloads, are available on the SourceForge project page.

From the release message:

Version 5.3.9 contains one major improvement and one bugfix. The default buffer size for all operators on classic files is 32 MiB. This will increase I/O speed in most cases. ncremap had a syntax error in a rarely used branch of code that caused Bash to emit scary messages even when the branch was not executed.

Users who want more speed without have to change any commands should upgrade to 5.3.9.

New Features

  1. The default buffer size for operating on classic files is 32 MiB. Formerly NCO used the netCDF default which was 8 KiB unless the filesystem explicitly set a blocksize in the stat->st_blksize element, in which case netCDF would set the buffersize to twice the blocksize. Experience shows that too many filesystems do not set the blocksize element and thus the blocksize remained (often) 8 KiB, which is much too small for efficient I/O on most ESM datasets. Hence we changed it to 32 MiB, the same buffer size that netCDF4 starts with. To return to the netCDF default buffer size, use --bfr_sz=0. To change the buffer size set it to your favorite number. This buffer size affects classic files only, it is ignored and harmless when set for netCDF4 files. Performance can significantly improve with sizes up to 1 GiB, though it is dependent on the filesystem, RAM, and the sizes of the variables being processed.

    ncks in.nc out.nc # NCO default 32 MiB buffer size
ncks --bfr_sz=0 in.nc out.nc # Use netCDF default size
ncclimo --bfr_sz=1073741824 in.nc out.nc # Request 1 GiB buffer size

  2. ncks -k (or --prn_fmt, --prn_fl_fmt) now prints the file format. This is similar (by intention) to the ncdump -k command. The first word ncks prints in response is identical to the ncdump -k response. In addition, ncks also prints all the synonyms for that file type (since clear naming is not a quality of netCDF filetypes), and the capacity of that filetype to hold variables and types:

    % ncks -k in.nc
classic = NC_FORMAT_CLASSIC = CDF1. This is the earliest ...

    http://nco.sf.net/nco.html#prn_fmt

Additional details are available in the ChangeLog.