Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
Due to the current gap in continued funding from the U.S. National Science Foundation (NSF), the NSF Unidata Program Center has temporarily paused most operations. See NSF Unidata Pause in Most Operations for details.
reading other much larger arrays is much much faster, eg reading lon array with 1152 values only takes 7.7e-5 sec. I don't get it.
I'd like some advise on this. On my workstation the read times of time values on the same dataset ranges between 1.9e-5 and 4.7e-5 sec. Thus something seems very wrong with the performance on the cray. btw this is on the lustre scratch2 file system and files are 433 MB each and striped across 24 ost's with a stripe size of 1MB.
Burlen here is the raw data gathered from edison$/usr/common/graphics/teca/builds/TECA/bin/bin/teca_metadata_probe --input_regex /scratch2/scratchdirs/prabhat/TCHero/data/cam5_1_amip_run2'\.cam2\.h2.2005-09-[0-9][0-9]-10800.nc$'
scan_files = 1.1513 open_file = 0.0967366 x_axis_metadata = 7.4109e-05 y_axis_metadata = 7.732e-06 z_axis_metadata = 3.84e-07 t_axis_metadata = 5.418e-06 variables = 0.000824452 read_x_axis = 5.9976e-05 read_y_axis = 1.0444e-05 read_z_axis = 4.294e-06 read_t_axis = 0.368079 open_file = 0.122236 t_axis_metadata = 4.906e-05 read_t_axis = 0.431482 open_file = 0.0953903 t_axis_metadata = 3.4205e-05 read_t_axis = 0.3815 open_file = 0.0853393 t_axis_metadata = 2.9607e-05 read_t_axis = 0.396472 open_file = 0.0664037 t_axis_metadata = 2.8239e-05 read_t_axis = 0.351707 open_file = 0.748844 t_axis_metadata = 3.3047e-05 read_t_axis = 1.03634 open_file = 0.161732 t_axis_metadata = 2.6955e-05 read_t_axis = 0.377919 open_file = 0.0820469 t_axis_metadata = 3.1613e-05 read_t_axis = 0.363014 open_file = 0.0903407 t_axis_metadata = 3.1844e-05 read_t_axis = 0.370521 open_file = 0.092586 t_axis_metadata = 2.9547e-05 read_t_axis = 0.37146 open_file = 0.083997 t_axis_metadata = 4.0095e-05 read_t_axis = 0.396375 open_file = 0.0799897 t_axis_metadata = 2.9833e-05 read_t_axis = 0.386237 open_file = 0.105688 t_axis_metadata = 0.000124456 read_t_axis = 0.481453 open_file = 0.0816038 t_axis_metadata = 3.0969e-05 read_t_axis = 0.355051 open_file = 0.101204 t_axis_metadata = 2.5927e-05 read_t_axis = 0.408186 open_file = 0.108662 t_axis_metadata = 2.6313e-05 read_t_axis = 0.314209 open_file = 0.0916682 t_axis_metadata = 2.9697e-05 read_t_axis = 0.319686 open_file = 0.0812744 t_axis_metadata = 2.7813e-05 read_t_axis = 0.391357 open_file = 0.101898 t_axis_metadata = 2.8607e-05 read_t_axis = 0.305515 open_file = 0.0748274 t_axis_metadata = 1.7247e-05 read_t_axis = 0.69522 open_file = 0.100151 t_axis_metadata = 1.7887e-05 read_t_axis = 0.511695 open_file = 0.0700686 t_axis_metadata = 1.8225e-05 read_t_axis = 2.27155 open_file = 0.121193 t_axis_metadata = 1.9918e-05 read_t_axis = 0.497648 open_file = 0.0734922 t_axis_metadata = 2.1134e-05 read_t_axis = 1.70601 open_file = 0.312652 t_axis_metadata = 1.888e-05 read_t_axis = 0.468489 open_file = 0.0906366 t_axis_metadata = 1.9165e-05 read_t_axis = 0.348474 open_file = 0.0776435 t_axis_metadata = 2.3417e-05 read_t_axis = 0.387956 open_file = 0.157729 t_axis_metadata = 4.9765e-05 read_t_axis = 0.344205 open_file = 0.114566 t_axis_metadata = 2.8431e-05 read_t_axis = 0.374615 open_file = 0.0788776 t_axis_metadata = 2.7865e-05 read_t_axis = 0.381856 collect_t_axis = 19.47 total = 20.72 A total of 240 steps available in 30 files. Using the noleap calendar. Timesare specified in units of days since 1979-01-01 00:00:00. The available times
range from 2005-9-1 3:0:0 (9733.12) to 2005-10-1 0:0:0 (9763).
netcdfgroup archives: