•         advanced

4.8 UNICOS Optimization

It should be noted that no UNICOS platform has been available at Unidata for netCDF testing for some years. The following information is left here for historical reasons.

As was mentioned in the previous section, it is possible to replace the I/O layer in order to increase I/O efficiency. This has been done for UNICOS, the operating system of Cray computers similar to the Cray Y-MP.

Additionally, it is possible for the user to obtain even greater I/O efficiency through appropriate setting of the NETCDF_FFIOSPEC environment variable. This variable specifies the Flexible File I/O buffers for netCDF I/O when executing under the UNICOS operating system (the variable is ignored on other operating systems). An appropriate specification can greatly increase the efficiency of netCDF I/O–to the extent that it can surpass default FORTRAN binary I/O. Possible specifications include the following:

bufa:336:2

2, asynchronous, I/O buffers of 336 blocks each (i.e., double buffering). This is the default specification and favors sequential I/O.

cache:256:8

8, synchronous, 256-block buffers. This favors larger random accesses.

cachea:256:8:2

8, asynchronous, 256-block buffers with a 2 block read-ahead/write-behind factor. This also favors larger random accesses.

cachea:8:256:0

256, asynchronous, 8-block buffers without read-ahead/write-behind. This favors many smaller pages without read-ahead for more random accesses as typified by slicing netCDF arrays.

cache:8:256,cachea.sds:1024:4:1

This is a two layer cache. The first (synchronous) layer is composed of 256 8-block buffers in memory, the second (asynchronous) layer is composed of 4 1024-block buffers on the SSD. This scheme works well when accesses proceed through the dataset in random waves roughly 2x1024-blocks wide.

All of the options/configurations supported in CRI's FFIO library are available through this mechanism. We recommend that you look at CRI's I/O optimization guide for information on using FFIO to it's fullest. This mechanism is also compatible with CRI's EIE I/O library.

Tuning the NETCDF_FFIOSPEC variable to a program's I/O pattern can dramatically improve performance. Speedups of two orders of magnitude have been seen.

Contact Us   •   Site Map   •   Search   •   Terms and Conditions   •   Privacy Policy   •   Participation Policy

Unidata is a member of the UCAR Office of Programs, is managed by the University Corporation for Atmospheric Research, and is sponsored by the National Science Foundation. P.O. Box 3000   •   Boulder, CO 80307-3000 USA   •   Tel: 303-497-8643   •   Fax: 303-497-8690