NOTE: The netcdf-hdf mailing list is no longer active. The list archives are made available for historical reasons.
"David Stuebe" <dstuebe@xxxxxxxxxx> writes: > <br><br>Hi NETCDF folks<br><br>I work on an unstructured finite volume > coastal ocean model, FVCOM, which is parallel (using MPICH2). The Read Write > is a major slow down for our large cases. On our cluster, we have one large > storage device, an emc raid array. The network is infini-band - the network > is much faster than the raid array. > <br><br>For our model we need to read large initial condition data sets, and > single frames of forcing data while running. We also need to write single > frames of data for output (frequently), and large restart files (less > frequently). > <br><br>I am considering two options for recoding the IO from the model. One > is based around the future F90 netcdf 4 parallel interface which would allow > a symmetric code- every processor does the same thing. The other option is to > use netcdf 3, let the master processor read/write the data and distribute it > to each node, -an asymmetric coding. > <br><br>What I need to know- are netcdf 4 parallel IO operations > blocking? <br><br>The problem - the order of cells and nodes in our data set > does not allow for a simple start, count read format. A data array might have > dimensions (time,layers,cells). As an example, in a 2 processor case > with 8 cells, proc1 has cells(1 2 5 7) while proc2 has cells (3 4 6 8) - > write operations would have to be in a do loop to write each cell > individually from the processor that owns it. > <br><br>For a model with 300,000 cells on 30 processors, this would be 10,000 > calls to NF90_PUT_VAR on each processor. Even if the calls are non-blocking > this seems dangerous.<br><br>Any thoughts?<br><br>David<br><br><br> > <br><br> Howdy David! Are you using unlimited dimensions for this test, and writing? There was a bug in netCDF-4 which caused metadata to be written every time a record variable was expanded along the unlimited dimension. This would cause a slowdown of parallel I/O performance, because blocking would occur on every write operation, as the metadata were updated. This is now fixed on the netcdf-4 snapshot: http://www.unidata.ucar.edu/software/netcdf/builds/snapshot/index_4.html Other than this bug, I believe that netCDF-4 will yield the same performance as the underlying HDF5 API, so the comments of the HDF5 programmers are very relevant. But before you test again, get the netCDF-4 snapshot to make sure it's not the netCDF-4 metadata bug which was causing your problems. Thanks! Ed -- Ed Hartnett -- ed@xxxxxxxxxxxxxxxx ============================================================================== To unsubscribe netcdf-hdf, visit: http://www.unidata.ucar.edu/mailing-list-delete-form.html ==============================================================================
netcdf-hdf archives: