"Hernan G. Arango" <arango@xxxxxxxxxxxxxxxxxx> writes:
> Hi All,
>
> I implemented the parallel I/O in ROMS (http://www.myroms.org) using
> the latest snapshot of the NetCDF-4/HDF5 libraries.
>
> I have two questions:
>
> (1) The documentation for "nf90_open_par" says that it is only
> possible to open a NetCDF4 format file for parallel processing.
> This makes sense if the file is open for writing. However,
> I think that it should be able to open a classic format file
> (NetCDF 3.x) for just reading since the library is backward
> compatible. I tested this in ROMS and in fact it is reading
> a classic format in parallel as far as I can see in the
> debugger. I am not going inside the libraries. So the
> question is this actually happening? Is the library reading
> a classic format in parallel? If so, perhaps the documentation
> needs to be revised.
>
>
http://www.unidata.ucar.edu/software/netcdf/docs_snapshot/netcdf-f90.html#NF90_005fOPEN_005fPAR
This is a very good question.
Parallel reads with netCDF classic files work because of the fseek
parallelism that work in parallel situation. Since the classic file is
laid out in a simple and strict way, each process ends up calculating
their offsets separately, and when the netCDF library reads the file,
it is able to do so in parallel.
I have added the following to the documentation:
Since the API for netCDF-4/HDF5 files is the same as the API for
classic files, parallel I/O code written for netCDF-4/HDF5 will
also work for classic files, when reading the file.
It is not possible to create a classic or 64-bit offset file with
parallel access, or write to classic or 64-bit files in parallel
with the netCDF library.
This is in the NetCDF Users' Guide:
http://www.unidata.ucar.edu/software/netcdf/docs_snapshot/netcdf.html#Parallel-Access
> (2) Why doesn't the library create identical files with the
> same executable? Are there any time makings or other execution
> information hidden in the created NetCDF-4 file. I tried to run
> the same execute twice and a binary difference tells me
> that the files are different:
>
> > diff new.nc old.nc
> Binary files new.nc and old.nc differ
>
> However, if I dump the data into ASCII files, using "ncdump", the
> files are identical.
>
> I need to understand this issue because this is an important
> parallel partition test. Regardless of the parallel partition
> or number of MPI nodes, we should get identical solutions.
> Otherwise, we have a parallel bug.
>
> If there is an internal marking, how should we deactivate it?
I do not believe you can deactivate this; it is a characteristic of
HDF5.
I have forwarded your question to the HDF5 programmers for
discussion.
Thanks,
Ed
--
Ed Hartnett -- ed@xxxxxxxxxxxxxxxx
