Previous: Dispatch layer and netCDF architecture Next: Parallel I/O Table of contents Frames 2012 Unidata NetCDF Workshop

35 Parallel I/O with NetCDF
Using parallel I/O can result in significant performance improvements in high performance computing applications.

35.0  Parallel I/O
Parallel I/O allows each processor in a multi-processor system to read and write data independently.
35.1  Introduction to Parallel I/O
Parallel I/O allows many processes to read/write netCDF data at the same time.
35.2  The pNetCDF Package
The parallel-netcdf (formerly "pnetcdf") package from Argonne and Northwestern can be used for parallel I/O with classic netCDF data.
35.3  Parallel I/O in NetCDF-4
NetCDF-4 exposes the parallel I/O features of HDF5.
35.4  Building NetCDF-4 with Parallel I/O
You must build netCDF-4 properly to take advantage of parallel features.
35.5  Using Parallel I/O in NetCDF-4
How to actually use parallel I/O in netCDF-4.
35.6  Collective and Independent Operations with Parallel I/O in NetCDF-4
Some netCDF operations are collective (must be done by all processes at the same time), and others are independent (can be done by any process at any time.)
35.7  Parallel I/O Example
This example is from nc_test4/tst_parallel.c, and is run by make check on platforms with MPI/IO support.
35.8  Example of Simple Parallel I/O Access
The nc_test4/tst_parallel3.c program provides an example of parallel I/O.
35.9  Example of Simple Parallel I/O Access (part 2)
The nc_test4/tst_parallel3.c program provides an example of parallel I/O.
35.10  Using NetCDF Parallel I/O in Fortran 90
Using netCDF-4 parallel I/O in Fortran 90 is easy.
35.11  Jumpshot Graph of Independent Parallel Data Writes
The jumpshot output shows four processors independently writing netCDF data.
35.12  Jumpshot Graph of Collective Parallel Data Writes
The jumpshot output shows four processors collectively writing netCDF data.

 


Previous: Dispatch layer and netCDF architecture Next: Parallel I/O Table of contents Frames 2012 Unidata NetCDF Workshop