Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Hi I am running some tests on an HPC cluster, altering the size of reads to test the performance of the file system. I am using python, and for sequential reads not using netCDF4 the read rate is pretty constant across different read sizes. However, when I introduce the netCDF4 library the smaller and larger reads see a dip in performance with a peak on the medium sized reads (creating a hill-like profile). The peak in the netCDF4 performance is at about the same read rate as the non-netCDF4 reads. The peak is at reads of about 1MB. We think this could be to do with buffering somewhere in the NetCDF library. Does anyone know of such buffering that we should be aware of? Many thanks Matt ---------------------------------------- Matthew Jones PhD Student Atmosphere, Oceans and Climate Department of Meteorology, University of Reading Room 288, ESSC, Harry Pitt Building, 3 Earley Gate, Reading, RG6 6AL, UK Ext: 5214 https://www.linkedin.com/pub/matthew-jones/8b/b81/25a http://www.met.reading.ac.uk/users/users/1887
netcdfgroup archives: