Unidata - To provide the data services, tools, and cyberinfrastructure leadership that advance Earth system science, enhance educational opportunities, and broaden participation. Unidata
         
  advanced  
 

Parallel I/O and NetCDF

In a March 1998 email response to a support question dealing with the general area of parallelism and how netCDF 3.4 could be modified for specific parallel platforms to use parallel I/O, Glenn Davis listed the parts of the library code that would have to be modified.

Subsequently R. K. Owen and others at NERSC implemented a parallel version of netCDF 3.4 for the Cray T3E. That effort is described in R.K's NetCDF pages.

Working with NERSC, the changes necessary to support parallel I/O on the Cray T3E have been integrated into the netCDF 3.5 beta test version, available from ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-beta.tar.Z

To support parallel I/O on the Cray T3E, the functions nc__create_mp(), nc__open_mp(), nc_set_base_pe(), and nc_inq_base_pe() were added to the public interface. The description of these functions, their parameters, and how they are used are documented in the netCDF reference documentation for the C interface under the section MPP Function Descriptions in the NetCDF-3 reference documentation for C.

In 2000, Unidata with researchers at NCSA and NCAR submitted a proposal in response to NSF's ITR program solicitation, to provide efficient parallel I/O on multiprocessor platforms by implementing the netCDF interface on top of the HDF5 library, which uses the Message Passing Interface (MPI) standard for parallel I/O. This proposal was not funded, but indicates one strategy for a parallel implementation of netCDF.

Researchers at Northwestern University and Argonne National Laboratories have begun work on an MPInetcdf parallel library for C. Another parallel netCDF library is under development at NERSC. In 2002, Unidata and NCSA submitted another proposal to the NASA AIST program to fund development of a parallel netCDF library using HDF5, but results of that proposal are still unknown.


This document is maintained by Russ Rew <russ@unidata.ucar.edu>
 
 
  Contact Us     Site Map     Search     Terms and Conditions     Privacy Policy     Participation Policy
 
National Science Foundation (NSF) UCAR Office of Programs University Corporation for Atmospheric Research (UCAR)   Unidata is a member of the UCAR Office of Programs, is managed by the University Corporation for Atmospheric Research, and is sponsored by the National Science Foundation.
P.O. Box 3000     Boulder, CO 80307-3000 USA     Tel: 303-497-8643     Fax: 303-497-8690