Server Performance

Operating System Configuration

Setting the number of file handles

The OS typically limits the number of open file handles per process. To check this value on Unix, use:

 ulimit -n

If you are using the default TDS configuration values, this value should be 1024 or greater. Otherwise you can tune this number based on your own settings.

Java Virtual Machine

We recommend the latest version of JDK 1.6. This seems to have performance improvements over 1.5, especially with the -server option.

To do so, set the Java runtime parameters with the environmental variable JAVA_OPTS. For example, add the following to ${tomcat_home}/bin/startup.sh :

starts Tomcat with the Server JVM, and gives it 256 Mbytes of memory to start with, and allows a maximum of 1024 Mbytes. These are suggested numbers, use more if you can. The maximum you can give seems to be around 1500m. You need to restart Tomcat for these options to take effect.

Tomcat

Version

We recommend the latest stable version of Tomcat 6.0, which supposedly has better stability and performance than Tomcat 5.x. This requires JDK 1.5 or above.

Automatic Startup

In a production environment, Tomcat should be automatically restarted when the machine starts. How to do this depends on what OS you are running. This FAQ has a bit of info.

Miscellaneous

Once thredds.war is expanded, manually copy everything in ${tomcat_home}/webapps/thredds/initialContent/root/ to ${tomcat_home}/webapps/ROOT/ .

Resources

Thredds Data Server

File Handles and Caching

The TDS caches file handles to minimize OS overhead. Currently this defaults to allow 200 - 400 open files for OPeNDAP and WCS, and 25 - 40 for HTTP file serving. This means that your OS must allow at least that many file handles per process, else you can get a "out of file handles" message, and the server may freeze up.

These numbers limit performance, but not functionality. For example, the number of files in an aggregation is not limited by these file handle limits.

Consolidate cache / temporary directories

The TDS writes temporary files and caches files. By default these are stored under ${tomcat_home}/content/thredds/. These directories can get large. You might want to relocate them to another place, for example if ${tomcat_home} has limited space. Also, theres no need to backup the cache directories, so they can be placed on a disk that is not backup up. To move these directories, see TDS Configuration with threddsConfig.xml.

OPeNDAP Memory Use

The OPeNDAP-Java layer of the server currently has to read the entire data request into memory before sending it to the client (we hope to get a streaming I/O solution working eventually). Generally clients only request subsets of large files, but if you need to support large data requests, make sure that the -Xmx parameter above is set accordingly.

Pre-indexing GRIB files

If you are serving GRIB files through any of the subsetting services (OPENDAP, WCS, etc), the CDM must write an index the first time it tries to read it. This can take several minutes for very large GRIB files. By indexing GRIB files before they are accessed, users get much faster response time.

Data File Disk Access

Generally the TDS will be I/O bound, and its speed will be determined by the speed of its disk access. Typically disk access is faster on a local drive than on an NFS mounted drive. High performance disk subsystems like RAID or SANs can also significantly improve TDS throughput. However, its best to measure performance of alternate hardware configurations.