Christopher, > thank you for your answer. > I just wanted to mention that I have found a bug in the NetCDF-4 library. > I have raised an issue on the GitHub repository > (see https://github.com/Unidata/netcdf-c/issues/15), but received > no reply until today. Thanks, I tested and committed your bug fix. --Russ > Zitat von Unidata netCDF Support <address@hidden>: > > > Hi Christopher, > > > >> I am currently analysing the performance of NetCDF-4 using parallel I/O > >> with > >> Lustre as underlying parallel filesystem. > >> I was wondering, why the library does not provide the alignment > >> feature, which aligns the data section to a particular byte boundary. > > > > The netCDF-3 library does permit specifying alignment, in the nc__enddef() > > function call: > > > > > > http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-c.html#nc_005f_005fenddef > > > > but the benefits of this feature were apparently overlooked in the netCDF-4 > > implementation. You're right, we should take advantage of alignment in > > netCDF-4 for filesystems like Lustre. > > > > Also, the parallel-netCDF library based on netCDF-3 has a default > > "nc_var_align_size" > > alignment hint of 512 bytes, but can use other values set with a call to > > MPI_Info_set(). The alignment setting is not stored in the file, so > > has to be reset > > when open iscalled. > > > >> Parallel I/O performance with Lustre improves significantly when the > >> I/O accesses are aligned to the stripe boundaries. > >> > >> I have added the call > >> > >> if( H5Pset_alignment(fapl_id, 1024*1024,1024*1024) < 0 ) > >> BAIL(NC_EHDFERR); > >> > >> in line 346 in nc4file.c , > >> recompiled the library, and observed that the bandwith improved > >> significantly > >> (stripe size : 1 MiB ). > >> > >> Could you tell me why alignment is not provided in the NetCDF-4 library? > > > > Lack of resources, mostly. I've added a Jira ticket to add this capability > > in a future release: > > > > https://bugtracking.unidata.ucar.edu/browse/NCF-289 > > > > You've demonstrated that it should be relatively easy; we will need to do it > > portably, so the library will work for any file system. > > > > Thanks for the valuable suggestion! > > > > --Russ > > > > Russ Rew UCAR Unidata Program > > address@hidden http://www.unidata.ucar.edu > > > > > > > > Ticket Details > > =================== > > Ticket ID: VUQ-958690 > > Department: Support netCDF > > Priority: High > > Status: Closed > > > > > > > Russ Rew UCAR Unidata Program address@hidden http://www.unidata.ucar.edu Ticket Details =================== Ticket ID: VUQ-958690 Department: Support netCDF Priority: High Status: Closed
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.