Work in maintaining, supporting, and developing the netCDF data model and software is associated with Endeavor 6: Improved scientific data access infrastructure from the Unidata 2008 proposal. NetCDF has become a key infrastructure element for producers and consumers of atmospheric science data, as well as data in other geosciences.
Recent netCDF development at Unidata aims at improving interoperability with other representations for scientific data, making the netCDF interface more suitable for use on high-end parallel platforms with high-resolution models, and providing netCDF software on a wider range of desktop platforms.
In February, we released version 3.5.1, which integrates bug fixes, performance enhancements, and portability improvements accumulated since a 2001 release. The netCDF file format is unchanged, so files written with previous versions can be read or written with version 3.5.1. This release netCDF-4 prototype.
We then created and released netCDF 3.6.0-beta1, with 64-bit file offsets for large file support. To test the new 64-bit offset format and to support compatibility with existing code, we added new functions to set and get default file creation format. We also updated the netCDF Windows port to compile under Visual C++.Net. In addition, the new release includes support for Intel compilers on Linux, ifc, icc, and a new version of the Fortran-to-C interface macros.
To facilitate creating documentation for netCDF-4, we converted and updated existing netCDF FrameMaker manuals to texinfo format,comprising a language-neutral guide and four language-specific reference manuals for C, Fortran-77, Fortran-90, and C++. We also added new sections on format guidance, new mode functions, and the new 64-bit offset mode.
Our goal is to implement an enhanced netCDF interface using an enhanced HDF5 (a data model, interface, and format from NCSA) for storage. Implementing netCDF-3 on HDF5 is essentially complete, as are some necessary HDF5 enhancements. What remains are specifications of the new netCDF-4 interfaces, additional enhancements to HDF5, and implementation of netCDF-4.
Recent enhancements to HDF5 to support netCDF-4 include:
NCSA and Unidata developers held several teleconferences to discuss proposed designs for HDF5 enhancements, HDF5 dimension scales, netCDF requirements for shared dimensions, and support for coordinate systems. The NCSA and UCAR lawyers finally settled intellectual property issues for the netCDF/HDF5 merger, leaving the result under an open source license. In addition, we participated in mailing list discussions on Unicode issues and uncovered a problem with HDF5 concurrency, identifying a need to change HDF5 internal caching.
In May 2004, we presented a quarterly review of the NASA-funded project for NASA. The project is still on schedule to meet its milestones. A tentative netCDF release schedule can be found at http://www.unidata.ucar.edu/software/netcdf/release_schedule.html.
Ed Hartnett wrote and submitted a paper "Merging the NetCDF and HDF5 Libraries to Achieve Gains in Performance" for NASA's 2004 Earth Science Technology Conference to be held June 22 - 24 in Palo Alto, California. Russ Rew will also be attending the conference to present the annual report for the project to NASA reviewers,