Version 4.9.0 of the netCDF Operators (NCO) has been released. NCO is an Open Source package that consists of a dozen standalone, command-line programs that take netCDF files as input, then operate (e.g., derive new data, average, print, hyperslab, manipulate metadata) and output the results to screen or files in text, binary, or netCDF formats.
The NCO project is coordinated by Professor Charlie Zender of the Department of Earth System Science, University of California, Irvine. More information about the project, along with binary and source downloads, are available on the SourceForge project page .
From the release message:
Version 4.9.0 includes new regridder features, JSON, and
Notably, this version simplifies the weight-application interface,
ncremap in stand-alone mode, and completely revamps,
improves, and accelerates the sub-gridscale weight application
The NCO conservative weight-generator algorithm also debuts.
ncremap always used ESMF or TempestRemap to make weights.
'ncremap -a nco' to get NCO's conservative weight generation.
NCO will always support ESMF and TR which are more time-tested.
However, NCO has some unique features, like an option to output the
overlap mesh in SCRIP format, and it performs best for some grids.
ncremaphas been refactored for full parallelism. Previously the standalone
ncremapcould only run parallelized when invoked with pure weight application requests. Special processing like CLM/ELM sub-gridscale regridding or MPAS juggling only worked with single file invocations (which is what ncclimo always requests). Now the standalone
ncremapis parallelized and adheres to
--job_nbrcontrol for all regridding in standalone invocations, meaning that regridding lists of files offline with MPI or background parallelism is now much quicker:
ls cam/*.nc | ncremap --job_nbr=4 --par_typ=mpi -m map.nc ls mpas/*.nc | ncremap -P mpas -j 4 -p mpi -m map.nc ls elm/*.nc | ncremap --sgs_frc=landfrac -j 16 -m map.nchttp://nco.sf.net/nco.html#ncremap
ncremaphas a new API for sub-gridscale (SGS) regridding. Previously
ncremapemployed a convoluted and slow procedure to remap data with sub-gridscale fractions, e.g., CLM/ELM, CICE, and MPAS-Seaice data. That procedure and ncremap API used to require specification of the source and destination grids. The new, faster
ncremapSGS procedure requires only the map-file and the name of the sub-gridscale fraction field. The
-P sgsoption is no longer required, thought is still accepted for back-compatibility.
ncremap --sgs_frc=landfrac -m map.nc # CLM/ELM ncremap --sgs_frc=frc.nc/landfrac -m map.nc # CLM/ELM external ncremap -P elm -m map.nc # CLM/ELM alternative ncremap --sgs_frc=aice --sgs_msk=tmask -m map.nc # CICE ncremap -P cice -m map.nc # CICE alternative ncremap --sgs_frc=timeMonthly_avg_iceAreaCell -m map.nc # MPAS-Seaice ncremap -P mpasseaice -m map.nc # MPAS-Seaice alternativehttp://nco.sf.net/nco.html#ncremap
ncremapsupports the new NCO weight generation algorithm. The NCO algorithm is first-order conservative and compares well in accuracy with other FV algorithms such as ERWG and TR. Special thanks to Henry Butowsky for his dedication to this. We evaluated ERWG "conserve", NCO "nco", and TR "mono" with
--chk_map(see below) on the E3SMv1 lo-res and hi-res grids. The three algorithms generally agree to 10-13 significant digits. ESMF won for atm->atm remapping, and NCO for ocn->atm remapping, both by razor-thin margins. YMMV. We hope users will try the NCO algorithm and send feedback. NCO weight-generation is threaded and scales well to 3-4 threads.
ncremap -a nco -s grd_src.nc -d grd_dst.nc -m map.nc in.nc out.nc ncremap -t 4 -a nco -s grd_src.nc -d grd_dst.nc -m map.nchttp://nco.sf.net/nco.html#ncremap
nckssupports a new option
--chk_mapto evaluate the quality of regridding weights. This option works with map-files (not grid-files) in ESMF/CMIP6-compliant format (i.e., sparse matrix S and coordinates
[xy][ab]_[cv]). When invoked with the additional
--area_wgtoption (also new), the evaluation statistics are area-weighted and thus reflect exactly the global-mean/min/max/ mebs/rms/sdn biases expected when regridding globally uniform fields. This tool makes it easier to objectively assess weight-generation algorithms, and will hopefully assist in their improvement.
ncks --chk_map map.nc ncks --chk_map --area_wgt map.nchttp://nco.sf.net/nco.html#chk_map http://nco.sf.net/nco.html#area_wgt
ncremapworks with newer MPAS Ocean files that contain BGC dimensions like R3 and FOUR.
ncremap -P mpas -m map_bilin.nc in.nc out.nchttp://nco.sf.net/nco.html#ncremap
ncremapnow propagates all netCDF formats except CDF5 to TempestRemap (TR) (which does not yet accept or produce CDF5 files AFAICT).
ncremapomits the TR
fv2se_sttmaps. TempestRemap author Paul Ullrich recommends this to keep
fv2se_sttmaps consistent (producing output in same range) with
ncremap -a fv2se_stt -s grd_src.nc -d grd_dst.nc -m map.nc in.nc out.nchttp://nco.sf.net/nco.html#ncremap
ncremapallows simultaneous invocation of horizontal and vertical regridding. Yes, now
ncremapcan automagically regrid any file to any new horizontal and vertical grid at the same time. Simply supply both a map-file and a vertical grid-file:
ncremap -m map.nc --vrt=vrt_grd.nc in.nc out.nchttp://nco.sf.net/nco.html#vrt
ncremap/ncclimohave deprecated support for options named for the short-lived ALM land surface model. Use
ncremap -P elm -m map.nchttp://nco.sf.net/nco.html#ncremap
NCO supports flexible options to specify non-default names for
vertical grid coordinates in input and output (vertically
interpolated) data files.
ncks --rgr plev_nm=vrt_nm --vrt=vrt_grd.nc in.nc out.nc ncremap -n '--rgr plev_nm=vrt_nm' --vrt=vrt_grd.nc in.nc out.nchttp://nco.sf.net/nco.html#lev_nm
nckssupports a new option
--chk_nanto quickly find the location of any NaN values in a dataset. Thanks to Matthew Thompson of NASA for this suggestion.
ncks --chk_nan in.nchttp://nco.sf.net/nco.html#chk_nan
Additional details are available in the ChangeLog.