Version 5.0.3 of the netCDF Operators (NCO) has been released. NCO is an Open Source package that consists of a dozen standalone, command-line programs that take netCDF files as input, then operate (e.g., derive new data, average, print, hyperslab, manipulate metadata) and output the results to screen or files in text, binary, or netCDF formats.
The NCO project is coordinated by Professor Charlie Zender of the Department of Earth System Science, University of California, Irvine. More information about the project, along with binary and source downloads, are available on the SourceForge project page.
From the release message:
Version 5.0.3 contains significant improvements to the quantization algorithms and fixes to MOAB/mbtempest regridding. NCO now supports the Digit Round and Granular BitGroom quantization algorithms. The failed self-test problems that prevented Conda from building NCO 5.0.2 have been fixed, and a 5.0.3 Conda package will soon be available.
Lossy quantization algorithms have been refactored. First, the
attribute name that store the level of quantization are now
QuantizeBitGroomNumberOfSignificantDigits. Previously, NCO stored the NSD attribute
number_of_significant_digits. However, this was deemed too ambiguous, given the increasing number of supported quantization methods. The new attribute names better disambiguate which algorithm was used to quantize the variable. Second, the default algorithm has been changed from BitGroom with BitRound masks from R. Kouznetsov (2021), to what we call Granular BitGroom (GBG). GBG combines features of BitGroom, BitRound, and DigitRound by Delaunay et al. (2019). GBG improves compression ratios by ~20% relative to BitGroom for
NSD=3on our benchmark 1 GB climate model output dataset.
ncks -7 -L 1 --ppc default=3 in.nc out.nc # GranularBitGroom ncks -7 -L 1 --baa=0 --ppc default=3 in.nc out.nc # BitGroom ncks -7 -L 1 --baa=1 --ppc default=3 in.nc out.nc # BitShave ncks -7 -L 1 --baa=2 --ppc default=3 in.nc out.nc # BitSet ncks -7 -L 1 --baa=3 --ppc default=3 in.nc out.nc # DigitRound ncks -7 -L 1 --baa=4 --ppc default=3 in.nc out.nc # GranularBitGroom ncks -7 -L 1 --baa=5 --ppc default=3 in.nc out.nc # BitRound ncks -7 -L 1 --baa=6 --ppc default=3 in.nc out.nc # HalfShave ncks -7 -L 1 --baa=7 --ppc default=3 in.nc out.nc # BruteForceDocumentation on Bit-Adjustment-Algorithms (BAA) 3-7 will be improved in the future. Bit-Groom/Shave/Set are described here:
Additional details are available in the ChangeLog.