NetCDF Status

Russ Rew, Ed Hartnett, and Dennis Heimbigner
March 23, 2009

This report updates the last status report of Unidata's netCDF development efforts since the User Committee meeting in October 2008.

NetCDF and Unidata

NetCDF efforts are associated with Theme 2: Advancing data services from the Unidata 2013 proposal. NetCDF continues to serve as a key infrastructure element for data providers and users of climate, atmosphere, and ocean data.

Recent netCDF developments improve client access to remote data services, documentation, portability, ease of use, interoperability with other data representations, and performance on high-end platforms.

NetCDF and libcf Development

Developments since the last User Committee meeting have included for the C-based netCDF 4.0.1, netCDF-Java 4.0, and libcf 1.0 have included:

Presentations and Meetings

Ed met with WRF developers to discuss conversion of WRF output to netCDF-4 classic model files. He also met with NASA Goddard's Global Modeling and Assimilation team to help with a transition to netCDF-4 for assimilating GOES data and using parallel I/O.

In October we presented netCDF-4 information at the annual HDF5 workshop. In November, we conducted a two-day netCDF workshop for 39 attendees, with new sections on netCDF-Java, remote access via DAP protocols, chunking and compression, parallel I/O, and the Common Data Model. At the December AGU meeting in San Francisco Ed presented a poster on "NetCDF-4 and the Weather Research and Forecasting (WRF) Model" and Dennis, Russ, and Ed were coauthors of a poster on "Merging the Data Models of NetCDF and DAP: Design Choices and Benefits". In January, Ed presented "NetCDF-4: Benefits and Advice for Data Providers" at the AMS meeting in Phoenix. In March Ed presented a netCDF parallel I/O tutorial to the LCI Conference on High Performance Computing

NetCDF Use

Following examination by 20 reviewers, NASA's Earth Science Data Systems Standards Process Group has concluded that netCDF classic should be adopted as a recommended standard. NOAA NESDIS has adopted netCDF-4 and CF metadata as the standard intermediate file format for new satellite products from the Center for Satellite Applications and Research operational systems and from the NOAA Office of Satellite Data Processing and Distribution. NetCDF-4 classic model will be used in the CMIP5 archive of climate models for fifth assessment report of the IPCC. A standards package including netCDF, CF conventions, OPeNDAP, and aggregation was reviewed and promoted to a Recommended DMAC Standard at a December meeting of the IOOS DMAC Steering Team. The International Best Track Archive for Climate Stewardship is archiving their data using netCDF. Recently netCDF-4 was also adopted at the UKAEA Fusion Culham Science Centre for storing fusion research data and model output. See also the March CommuniteE-letter for the lead article on "New and noteworthy events in netCDF development".

by Russ Rew