Hi Jesse, > I have an HPC cluster using lustre as our backend file systems. The > cluster serves primarily weather models, such as the WRF and GFS. > > One thing we observed is that netcdf writes can often be very slow on > lustre. Do you have any recommended tuning procedures for netcdf on lustre? No, sorry, we don't currently test on lustre. However, if you have configured lustre with a large disk block size and are writing netCDF files with lots of records and lots of record variables (variables that use an unlimited dimension) then you could be seeing a problem with writing such data a variable at at time instead of a record at a time: https://www.unidata.ucar.edu/jira/browse/NCF-142 You haven't said what version of the library you're using, but the fix above is in the nccopy utility in version 4.2, and in some of the utilities in the most recent release of NCO (the NetCDF Operators software from UC Irvine). Also, are you using parallel I/O? Use of parallel-netcdf may be a solution worth looking at if you're writing classic-format files, or the HDF5-based parallel I/O in netCDF-4 otherwise. If you have a small example that demonstrates the bad performance, we could try to reproduce it and diagnose the problem. --Russ --Russ Russ Rew UCAR Unidata Program address@hidden http://www.unidata.ucar.edu Ticket Details =================== Ticket ID: TCU-710461 Department: Support netCDF Priority: Normal Status: Closed
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata inquiry tracking system and then made publicly available through the web. If you do not want to have your interactions made available in this way, you must let us know in each email you send to us.