NetCDF Status

Ed Hartnett, Russ Rew
March 12, 2007


NetCDF and Unidata

Work in maintaining, supporting, and developing the netCDF data model and software is associated with Endeavor 6: Improved scientific data access infrastructure from the Unidata 2008 proposal. NetCDF has become a key infrastructure element for data providers and users of oceanographic and atmospheric science data, as well as data in other geosciences.

Recent netCDF development, both at Unidata and at other institutions, aims at generalizing the netCDF data model, improving interoperability with other representations for scientific data, making the netCDF interface more suitable for use on high-end parallel platforms with high-resolution models, and providing netCDF software on a wider range of platforms.

libcf - The CF Conventions Library

The NetCDF CF Library supports the creation of scientific data files conforming to the CF conventions, using the netCDF API. It assists the user in correctly writing and interpreting the Climate and Forecast Conventions.

The prototype of libcf, a library to help write and read CF conventions files, was designed and developed. With the recent alpha4 release, the base functionality for the CF Convention document was achieved - the entire document has been addressed in some way (not necessarily the best, or final, way).

This is part of the general effort to bring some of the advanced features of the netCDF Java API to the C/Fortran netCDF library. The CF Library will allow users to use geo-located data in a more general way, without having to customize their tools (as much) for each data set.

NetCDF-4/HDF5 Development

NetCDF-4 is a major project to allow users to write and read data in HDF5 format, through the netCDF API. NetCDF-4 also expands the netCDF API to include new constructs, such as groups and user-defined data types.

NetCDF-4 depends on the still unreleased version 1.8 of HDF5, now tentatively scheduled for beta release in April 2007. The delay in releasing HDF5 1.8 continues to be an obstacle in making netCDF-4 functionality available to developers and users.

NetCDF-4 development has continued, with many alpha releases of netCDF-4.0 improving support for new platforms and compilers, performance, functionality, and documentation, and tracking changes in the HDF5 1.8.0 alpha releases. Additionally several bugs have been reported by netCDF-4 alpha version users, and have been fixed.

The netCDF-4 alpha-18 release can be downloaded from the netCDF-4 FTP site: ftp://ftp.unidata.ucar.edu/pub/netcdf/

NetCDF Training Workshop

Two new training workshops, "NetCDF for Developers" and "NetCDF Java" were created and presented twice to a total of about 36 attendees. Feedback from the workshop participants has been positive.

The netCDF workshops will be held again in July or August.

Other NetCDF Developments

In March, 2007, we released netCDF 3.6.2. NetCDF 3.6.2 prepares the netCDF configure and build system for the upcoming 4.0 release, and fixes some platform-specific build problems. It also includes complete example programs demonstrating the C, C++, Fortran-77, and Fortran-90 interfaces in a new netCDF tutorial for developers. netCDF-3.6.2 includes support for shared libraries, support for multiple Fortran compilers on the same platform, C++ interface improvements, and better support for building Windows DLLs.

The nightly testing of the netCDF package at Unidata has been substantially improved. All available platforms now test netCDF with all available compilers, and with many variations of configuration options. The testing generates a daily snapshot of the netCDF distribution, and a daily report on the results of the testing (which can be found at: http://www.unidata.ucar.edu/software/netcdf/builds/snapshot/).

This testing is also being done for the netCDF-4.0 daily snapshot releases, but netCDF-4.0 does not build and test cleanly on as many platforms as netCDF-3.6.2 (yet).

For new users, we made new resources for learning about netCDF available: an online version of the NetCDF for Developers workshop, and a set of progressively more complex example programs available for various programming interfaces, initially C, Fortran-77, Fortran-90, C++, and Java. When we announced these, we also sought equivalent examples for other contributed interfaces, and we have made available the resulting examples for Python, MATLAB, Perl, and IDL.

The netCDF project was nominated for the UCAR 2006 Outstanding Scientific and Technical Advancement Award.

Ray Pierrehumbert, noted climatologist from the University of Chicago, had this to say on realclimate.org:

 I think one mustn't discount a breakthrough of a technological sort in
 AR4 though: The number of model runs exploring more of scenario and
 parameter space is vastly increased, and more importantly, it is
 available in a coherent archive to the full research community for the
 first time. The amount of good science that will be done with this
 archive in the next several years is likely to have a significant
 impact on our understanding of climate. --raypierre 
by Russ Rew