[netcdf-hdf] using POSIX calls when configured for MPI-IO

[Quincey Koziol]

Hi Rob,

On Sep 17, 2007, at 12:47 PM, Robert Latham wrote:

> On Sat, Sep 15, 2007 at 07:03:06PM -0500, Quincey Koziol wrote:
>> I think you might want the "MPI-POSIX" driver in HDF5 (look at
>> H5Pset_fapl_mpiposix), but I don't know if the netCDF-4 library is
>> exporting a way to use that in netCDF-4 applications.  Hmm, but re-
>> reading your note again, I'm not certain.  Can you take a look at
>> the  MPI-POSIX file driver and see if that's what you want?
>
> Hi Quincey
>
> That's close, but
>
> H5FD_mpio_read actually uses MPI_File_* calls, whereas
> H5FD_mpiposix_read uses fread and such.
>
> What I'm asking for is an all MPI-IO path for applications to use.
> This is already possible in HDF5.
>
> There are two reasons for this.  First, MPI_File_open does not
> necessarily take a POSIX-style file name:
>
> http://www.mpi-forum.org/docs/mpi-20-html/node175.htm#Node175
>
>   "The format for specifying the file name in the filename argument is
>   implementation dependent and must be documented by the
>   implementation."
>
> Second, by using all MPI-IO routines, there are opportunities for
> performance tuning (through collective I/O or MPI_Info hints)
> unavailable in the POSIX I/O case.
>
> I'm quite interested in an all MPI-IO path, because I made extensive
> use  of this feature of HDF5 to track down bugs found by the HDF5
> parallel tests last year.   Also, an all MPI-IO path allows us to make
> full use of all the ROMIO optimizations we've worked on.
>
> You guys are so close to this, and that's why I'm sort of pushing.  It
> looks like just a few tweaks to nc_get_file_version() and
> nc_check_for_hdf5() get you there.

	Ah, OK, that'll be in Russ & Ed's court then, I think.

		Quincey

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2448 bytes
Desc: not available
Url : http://mailman.unidata.ucar.edu/mailing_lists/archives/netcdf-hdf/attachments/20070917/f6e0cdfc/attachment.bin