netcdf-hdf mailing list is no longer active. The list archives are made available for historical reasons.
John:It seems that we are going to a review of MEAD project. MEAD is an NSF expedition project back to 4 years ago. At first, we didn't committed to provide either an operational parallel or a sequential IO module.s. What we did is to do a kind of expedition, and further more to investigate the performance of HDF5. However, at the end of the project, John Mialakes would like to have our parallel version into WRF release. Since I have been spending limited time maintaining the module even after the project ended a year ago. Sequential version has never been into WRF release since WRF has already had several sequential IO modules. I did remember I checked the output carefully about dim. scale at that time; I should realize the reverse order of dim-rank. Somehow it just slipped away from my finger. In the future I need to review them again, I just don't have time now. As for parallelzation, WRF does parallelism in spatially(domain decompositions). Time won't affect parallelzation.
As for the last few questions you ask: We can "extend" the data this way but just not use chunking storage in WRF.I am not sure your last few questions. The bottom line is to store data in contiguous storage won't affect parallelzation at all. It just add a little bit more meta-data. If you want to know more about this, you can go to http://hdf.ncsa.uiuc.edu/apps/WRF-ROMS to read more report about this work; especially the article at http://hdf.ncsa.uiuc.edu/apps/WRF-ROMS/performance-report.pdf may help answer your questions.
Kent At 09:32 AM 3/31/2005, you wrote:
Hi Kent:So do you think no one will use this particular format? If so, I will wait for new version..I don't know. Somebody from OU contacted with me about using WRF-PHDF5. But I didn't hear from them about this issue.since its also in the sequential version, i guess the question is, do you know if WRF is planning on using either version?BTW, in the parallel output (but not the sequential), you put each time step in its own group. was this to optimize parallel performance or something ?The reason is: we are using contiguous storage for parallel HDF5 because of performance issue. So that's why we put each time step in to its own group. I hope that in the future I can have time or maybe a little funding to modify the WRF-HDF5 IO modules with 1.8 release.so you use contiguous, not chunked storage. so you cant extend the data ? so you create a new group as each time step completes ? how do you parellelize the tasks? does each thread have its own seperate region of the array?