NetCDF Status

Ed Hartnett and Russ Rew
May 2005

NetCDF and Unidata

Work in maintaining, supporting, and developing the netCDF data model and software is associated with Endeavor 6: Improved scientific data access infrastructure from the Unidata 2008 proposal. NetCDF has become a key infrastructure element for producers and consumers of atmospheric science data, as well as data in other geosciences.

Recent netCDF development, both at Unidata and at other institutions, aims at improving interoperability with other representations for scientific data, making the netCDF interface more suitable for use on high-end parallel platforms with high-resolution models, and providing netCDF software on a wider range of desktop platforms.

NetCDF-3 Development

Since the last status report, we released netCDF-3.6.0 and netCDF-3.6.0-p1 in December 2004 and February 2005. The latest software improves large file support, Windows compatibility, ease of installation, and performance using the Fortran-90 interface. We also made binary releases of netCDF-3.6.0-p1 available for 12 platforms, including a Windows DLL.

As a result of previous discussions with ESRI, the primary supplier of GIS software, ESRI announced that ArcGIS 9.2 will support accessing netCDF data that follows specified georeferencing conventions.

NetCDF-4/HDF5 Development

In October 2004, we attended and presented posters on netCDF-4 progress and on a Common Data Model (CDM) for improving interoperability among netCDF, HDF, and OPeNDAP (formerly DODS). We also met with OPeNDAP and HDF5 developers for further CDM discussion and agreements. John Caron presented a Unidata seminar on the CDM in December.

Since the last status report, we have implemented parallel I/O, a netCDF version of HDF5 Groups, support for new data types, and much of support for compound types in netCDF-4. We merged the source for netCDF-3.6.0 with netCDF-4 prototype code to create netCDF-4.0-alpha. Some unanticipated problems were uncovered in attempting to support the old netCDF 2 interface, but we are still on schedule for a summer release. Several alpha releases have been made available to NCSA developers. A teleconference in March settled some issues related to parallel I/O, joint release, scheduling, CDM work, compiler/build problems, and use of a new HDF5 facility for "dimension scales".

Other netCDF developments

Work continued on the development of a new version (2.2) of the Netcdf-Java library that implements the Common Data Model. Early versions of the code and documentation are available. Version 2.2 includes a Java framework for translation of other file formats into netCDF. With this software, for example, it is possible to access GRIB data through the same interface as for netCDF or OPeNDAP data.

We continue to work with researchers at Argonne National Laboratory and Northwestern University who have developed a parallel interface for writing and reading netCDF data, tailored for use on high performance platforms with parallel I/O.

In December, we submitted an unsolicited proposal to NASA for "Implementing a Common Data Model for NetCDF-4, HDF5, and OPeNDAP" that we just received full funding for, supporting 0.75 FTE during the next year to continue improving interoperability among netCDF, HDF5, and OPeNDAP by implementing the scientific data layer of a "Common Data Model" in C. In addition, we reached agreement with some external developers to add netCDF to their Web Coverage Server (WCS) return types for binary data, to make netCDF data available through GDAL and a commercial GIS client.