[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #VUQ-958690]: Alignment of data section in NetCDF-4

Hi Christopher,

> I am currently analysing the performance of NetCDF-4 using parallel I/O with
> Lustre as underlying parallel filesystem.
> I was wondering, why the library does not provide the alignment
> feature, which aligns the data section to a particular byte boundary.

The netCDF-3 library does permit specifying alignment, in the nc__enddef()
function call:

  
http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-c.html#nc_005f_005fenddef

but the benefits of this feature were apparently overlooked in the netCDF-4
implementation.  You're right, we should take advantage of alignment in
netCDF-4 for filesystems like Lustre.

Also, the parallel-netCDF library based on netCDF-3 has a default 
"nc_var_align_size" 
alignment hint of 512 bytes, but can use other values set with a call to 
MPI_Info_set().  The alignment setting is not stored in the file, so has to be 
reset 
when open iscalled. 

> Parallel I/O performance with Lustre improves significantly when the
> I/O accesses are aligned to the stripe boundaries.
> 
> I have added the call
> 
> if( H5Pset_alignment(fapl_id, 1024*1024,1024*1024) < 0 )
> BAIL(NC_EHDFERR);
> 
> in line 346 in nc4file.c ,
> recompiled the library,  and observed that the bandwith improved significantly
> (stripe size : 1 MiB ).
> 
> Could you tell me why alignment is not provided in the NetCDF-4 library?

Lack of resources, mostly.  I've added a Jira ticket to add this capability
in a future release:

  https://bugtracking.unidata.ucar.edu/browse/NCF-289

You've demonstrated that it should be relatively easy; we will need to do it 
portably, so the library will work for any file system.  

Thanks for the valuable suggestion!

--Russ

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: VUQ-958690
Department: Support netCDF
Priority: High
Status: Closed