Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
To clarify: the way I see it, you can do parallel I/O in three different ways. The first is to reserve a process which will only deal with I/O and other process will exchange data to read/write with it. -- In a sense, this is not a parallel IO. It is using the sequential IO to handle the parallel applications. The second is to have each process read/write independantly. -- You may talk about the independent IO here. The third is to aggregate the I/O for several processes to improve performances. -- You may mean collective IO here. So my question was: in practice, which approach does parallel netCDF use ? -- You can do both independent IO and collective IO with parallel HDF5. Definitely collective IO for parallel NetCDF-3(argonne's parallel NetCDF) and very also very possible independent IO. > in strict performance terms -- which in the end is not really the > be-all end all -- Argonne-Northwestern Parallel-NetCDF will be hard to > beat, unless you are working with record variables. Do you speak from personal experience ? I would be very interested in seeing some data or benchmark about it. -- There should be a paper that listed the flash benchmark comparison between parallel NetCDF from Northwest(or parallel netcdf-3) and parallel HDF5. However, it is an unfair comparison. It used collective IO for parallel NetCDF-3 but independent IO for parallel HDF5. You can find more detailed about the fair comparison with the collective IO for these two packages from http://www.spscicomp.org/ScicomP12/Presentations/User/Yang.pdf Be aware this was also a bit old. Don't know what's the current status between these two packages. Kent -- Alexis Praga _________________________________________________________ Ph.D Student Aviation et Environnement CERFACS alexis.praga@xxxxxxxxxx (33) 05 61 19 31 74
netcdfgroup archives: