Re: question about HDF5 parallel use of unlimited dimensions...

"Muqun (Kent) Yang" <ymuqun@xxxxxxxxxxxxx> writes:

> Ed:
> On Wed, 29 Jun 2005, Ed Hartnett wrote:
>
>> Howdy all!
>> 
>> I have a question about parallel programming with HDF5. Do parallel
>> programmers just not use unlimited dimensions?
>> 
> You can use unlimited dimensions for parallel programming.
>> I noticed that H5Dextend is a collective operation. This would make it
>> difficult for programs to add a record at a time.
>> 
>
> I don't think it is difficult as long as all processors participate.

But doesn't that waste a lot of cycles, wait to sync up every time a
new record is written?

As I understand it, using fixed-length data, my different processes
could write their data and go on their merry way, without waiting for
anything.

Maybe this just doesn't matter much because it's not the bottleneck.

>
> I may miss something. Parallel NetCDF tests I wrote include unlimited 
> dimension dataset tests for both collective and independent modes. Tests 
> passed.
>
> Although H5Dextend is collective, I remember that I noticed that doesn't 
> matter for netCDF4 users. I may be wrong. Maybe other people can give 
> some insights.
>

As currently implemented, H5Dextend is called when needed as you write
the data in netcdf. That is, if you are writing a record at a time,
H5Dextend is called for each record.

Thanks,

Ed
-- 
Ed Hartnett  -- ed@xxxxxxxxxxxxxxxx