Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
The netCDF operators NCO version 4.0.4 are ready. http://nco.sf.net (Homepage) http://dust.ess.uci.edu/nco (Homepage "mirror") This version contains several improvements and a few fixes. New stuff: Chunking is now supported on all relevant operators, i.e., all except ncrename and ncatted. ncap2, moreover, now propagates deflation, and tries to propagate chunking options, through derived expressions. ncpdq now tries to avoid packing grid properties like coordinates. Metadata printed by ncks is now re-formatted, expanded, and simpler. Scripts that depend on the exact format of ncks output may break. Let us know if this causes you major heartburn and, if so, we will consider implementing an option to restore the previous format. Work on NCO 4.0.5 is underway. Areas of improvement include support for spectral transforms, more chunking rulesets. "New stuff" in 4.0.4 details: A. Chunking supported in all operators. The exception is ncap2, which propagates chunking from fixed and copied variables, though not through algebraic expressions. (This capability may be in the next version.) http://nco.sf.net/nco.html#cnk B. ncecat, and all other operators, now retain the input deflation level, if any, for a variable, as the default for the output file. C. Deflation level (--dfl_lvl) propagates into ncap2 derived variables Thanks to Henry Butowsky. D. ncks -m now always summarizes metadata on chunking, compression, and packing Moreover, it prints metadata with clearer English descriptions E. ncpdq no longer packs grid properties such as coordinate variables, CF "special variables, or weights. Rationale: Coordinate variables are 1D, so packing saves little space, and the resulting files are difficult for humans to read. Similarly, Gaussian and area weights are often used to multiply other variables in re-inflated (unpacked) files. Packing such weights causes a considerable loss of precision in the result. If users wish to pack such variables, we could implement new packing policies that explicitly packs coordinates and weighting variables. We welcome your feedback. A workaround for those needing to pack such variables now, is to use the packing functions in ncap2, or to rename the grid properties prior to calling ncpdq. F. Stricter checks to identify malformed user-input strings. Issue more informative warnings, hints, and errors when detected. G. Prior versions of ncrename -a .old_nm,new_nm in.nc out.nc would, if old_nm did not exist, write the correct file then exit with an error message although no error had occurred. http://nco.sf.net#bug_ncrename_dot H. MacIntosh users who do not use MacPorts can install NCO binaries by clicking on an Apple Disk Image (.dmg) file. Thanks to Chad Cantwell of UCI. http://nco.sf.net#dmg "Sticky" reminders: J. ncks and ncecat support netCDF4 chunking options: These options can improve performance on large datasets. Large file users: Send us suggestions on useful chunking patterns! More useful chunking patterns may be implemented in NCO 4.0.4. ncks -O -4 --cnk_plc=all in.nc out.nc http://nco.sf.net/nco.html#chunking K. Pre-built, up-to-date Debian Sid & Ubuntu Karmic packages: http://nco.sf.net#debian L. Pre-built Fedora and CentOS RPMs: http://nco.sf.net#rpm M. Did you try SWAMP (Script Workflow Analysis for MultiProcessing)? SWAMP efficiently schedules/executes NCO scripts on remote servers: http://swamp.googlecode.com SWAMP can work command-line operator analysis scripts besides NCO. If you must transfer lots of data from a server to your client before you analyze it, then SWAMP will likely speed things up. N. NCO support for netCDF4 features is tracked at http://nco.sf.net/nco.html#nco4 NCO supports netCDF4 atomic data types, compression, and chunking. NCO 4.0.4 with was built and tested with HDF5 hdf5-1.8.4-patch1 and with netCDF4 4.1.2-beta1-snapshot2010080820 NCO may not build with earlier, and should build with later, netCDF4 releases. This is particularly true since NCO 4.0.4 takes advantage of an internal change to the netCDF nc_def_var_chunking() API in June 2009. export NETCDF4_ROOT=/usr/local/netcdf4 # Set netCDF4 location cd ~/nco;./configure --enable-netcdf4 # Configure mechanism -or- cd ~/nco/bld;./make NETCDF4=Y allinone # Old Makefile mechanism O. Have you seen the NCO logo candidates by Tony Freeman, Rich Signell, Rob Hetland, and Andrea Cimatoribus? http://nco.sf.net Tell us what you think... Enjoy, Charlie GSL functions added to ncap2 in 4.0.4: none! (Most desired GSL functions have been merged) -- Charlie Zender, Department of Earth System Science University of California, Irvine (949) 891-2429 :)
netcdfgroup archives: