Re: question about HDF5 parallel use of unlimited dimensions...

NOTE: The netcdf-hdf mailing list is no longer active. The list archives are made available for historical reasons.

"Muqun (Kent) Yang" <ymuqun@xxxxxxxxxxxxx> writes:

> Ed:
> On Wed, 29 Jun 2005, Ed Hartnett wrote:
>> Howdy all!
>> I have a question about parallel programming with HDF5. Do parallel
>> programmers just not use unlimited dimensions?
> You can use unlimited dimensions for parallel programming.
>> I noticed that H5Dextend is a collective operation. This would make it
>> difficult for programs to add a record at a time.
> I don't think it is difficult as long as all processors participate.

But doesn't that waste a lot of cycles, wait to sync up every time a
new record is written?

As I understand it, using fixed-length data, my different processes
could write their data and go on their merry way, without waiting for

Maybe this just doesn't matter much because it's not the bottleneck.

> I may miss something. Parallel NetCDF tests I wrote include unlimited 
> dimension dataset tests for both collective and independent modes. Tests 
> passed.
> Although H5Dextend is collective, I remember that I noticed that doesn't 
> matter for netCDF4 users. I may be wrong. Maybe other people can give 
> some insights.

As currently implemented, H5Dextend is called when needed as you write
the data in netcdf. That is, if you are writing a record at a time,
H5Dextend is called for each record.


Ed Hartnett  -- ed@xxxxxxxxxxxxxxxx

  • 2005 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-hdf archives: