« Successful Python-Fo... | Main | Unidata Update:... »

Version 4.7.7 of the netCDF Operators (NCO) has been released. NCO is an Open Source package that consists of a dozen standalone, command-line programs that take netCDF files as input, then operate (e.g., derive new data, average, print, hyperslab, manipulate metadata) and output the results to screen or files in text, binary, or netCDF formats.

The NCO project is coordinated by Professor Charlie Zender of the Department of Earth System Science, University of California, Irvine. More information about the project, along with binary and source downloads, are available on the SourceForge project page .

From the release message:

Version 4.7.7 has modest new features everywhere: Improved forward and backward compatibility with newer and older versions of ESMF_RegridWeightGen; Generation and inferral of grids running north->south (à la ECMWF); exact symmetry for Gaussian grid interfaces; an ncpdq compression map that converts doubles to floats; production of "skeleton files" for new grids; equivalent treatment of MPAS and CESM grid variables in arithmetic; and reversion of the filename whitelist security.

New Features

1. ncpdq has a new packing map: dbl_flt
   The map converts to double-precision to single-precision, so it is technically a precision change, not packing. It is in ncpdq because it reduces space by ~50%, like packing. Like packing the precision loss is irreversible. Unlike packing, no attributes are created, modified, or deleted.
   

   ncpdq -M dbl_flt in.nc out.nc

   http://nco.sf.net/nco.html#dbl_flt
   http://nco.sf.net/nco.html#ncpdq
2. Filename sanitization via character whitelists, first introduced in NCO 4.7.3 to increase security against malicious filenames, has been turned-off, possibly permanently. Points passionately raised by a user led us to turn-off this feature until lack of it is shown to lead to a real-world exploit.
   https://github.com/nco/nco/issues/104
   http://nco.sf.net/nco.html#wht_lst
3. The CCM/CAM/EAM family of atmospheric models does not output a "bounds" variable or attribute corresponding to its "lev" coordinate. This prevents NCO from activating some desirable CF machinery. To workaround this, NCO now outputs the ilev coordinate (and hyai, hybi) whenever the lev coordinate is also output.
   

   ncks -v T in.nc out.nc

   http://nco.sf.net/nco.html#bnd
4. Equivalent treatment of MPAS and CESM grid variables. NCO does not arithmetically alter grid-related variables. Who wants to subtract latitude coordinates from each other? Previously it only skipped arithmetic on CESM'ish grid variables. Now NCO has an extensive (pun intended!) list of MPAS and other grid variables that it will not subtract. For the most part, ncremap will not regrid these variables either.
   http://nco.sf.net/nco.html#cnv_CCSM
   http://nco.sf.net/nco.html#cnv_MPAS
   http://nco.sf.net/nco.html#ncremap
5. I greatly expanded the field manual to regridding with ncremap. The documentation is currently available to E3SM users at:
   https://acme-climate.atlassian.net/wiki/spaces/SIM/pages/754286611/Regridding+E3SM+Data+with+ncremap and to others via email request (it will be in Users Guide soon).
6. ncremap adds --ignore_degenerate to default ESMF_RegridWeightGen (ERWG) options when ERWG version >= 7.0. This is done to preserve backwards compatibility since ERWG 7.1.0r and later require --ignore_degenerate to successfully regrid some datasets (e.g., CICE) that previous ERWG versions handle fine.
   

   ncremap -m map.nc in.nc out.nc

   http://nco.sf.net/nco.html#wgt_opt
7. Not only do the British drive on the wrong side of the road, they prefer North-to-South (n2s) grids over South-to-North (s2n) grids. NCO now implements a latitude direction keyword lat_drc. Grid generation implements n2s ordering via --rgr lat_drc=n2s. Grid diagnosis now recognizes n2s ordering for all rectangular grids.
   http://nco.sf.net/nco.html#lat_drc
8. ncremap now generates exactly symmetric interface latitudes for Gaussian grids. To our knowledge only NCO generates accurate Gaussian interfaces (via a Newton-Raphson iteration of boundaries to enclose the area indicated by the Gaussian weight). However, previously the interface latitudes could accumulate rounding errors during Newton-Raphson iteration differ between the hemispheres by ~1.0e-14. Now symmetry properties are used to ensure exact symmetry.
   http://nco.sf.net/nco.html#gss
9. ncremap now produces a "skeleton file" of grids it generates when requested with --skl_fl.

   ncremap -G latlon=2560,5136#lat_typ=gss#lon_typ=grn_ctr#lat_drc=n2s -g grd_ecmwf.nc --skl=skl_ecmwf.nc

   http://nco.sf.net/nco.html#skl_fl
   http://nco.sf.net/nco.html#skeleton

Additional details are available in the ChangeLog.