TDS Tutorial: Getting Started

System requirements

• Sun Java 1.6
• Apache Tomcat 5.5 or above (we recommend the latest version of Tomcat 6.0)

While there are different distributors of Java and servlet containers, Unidata develops, uses and tests the THREDDS Data Server using Sun Java and the Apache Tomcat servlet container.

Installing Java

1. Download current Java SE Developer Kit (JDK) from Sun.

We will be using the Linux x32-bit, non-rpm, self-extracting file for this workshop:

jdk-6u15-linux-i586.bin

2. Install the JDK as per the Sun installation instructions.

Copy the self-extracting file to the installation directory (our home directory is the installation directory in this example):

$ cp jdk-6u15-linux-i586.bin ~
$ cd

Change the permissions on the file to be executable:

$ ls -l jdk-6u15-linux-i586.bin 
rw-r--r-- 1 thredds workshop 81771117 2009-08-03 15:56 jdk-6u15-linux-i586.bin

$ chmod 755 jdk-6u15-linux-i586.bin  

$ ls -l jdk-6u15-linux-i586.bin 
-rwxr-xr-x 1 thredds workshop 81771117 2009-08-03 15:56 jdk-6u15-linux-i586.bin

Run self-extracting file and agree to the terms of the code license:

$ ./jdk-6u15-linux-i586.bin

3. Set the $JAVA_HOME environment variable.

$ set JAVA_HOME=/home/thredds/jdk1.6.0_15  
$ echo $JAVA_HOME 
/home/thredds/jdk1.6.0_15 

Installing Tomcat

1. Download current version of the Tomcat servlet container.

We will be using the core binary tar.gz file for this workshop:

apache-tomcat-6.0.20.tar.gz

2. Install Tomcat as per the Apache Tomcat installation instructions.

Copy the binary tar.gz file to the installation directory (our home directory is the installation directory in this example):

$ cp apache-tomcat-6.0.20.tar.gz ~
$ cd

Unpack the archive to create Tomcat directory:

$ tar xvzf apache-tomcat-6.0.20.tar.gz

$ ls -ld apache-tomcat-6.0.20
drwxr-xr-x 9 thredds workshop    4096 2009-08-03 16:04 apache-tomcat-6.0.20

3. Set the $TOMCAT_HOME environment variable.

$ set TOMCAT_HOME=/home/thredds/apache-tomcat-6.0.20
$ echo $TOMCAT_HOME
/home/thredds/apache-tomcat-6.0.20

Exploring $TOMCAT_HOME

1. Examine the Tomcat directory structure.

Move into the new Tomcat directory and do a long listing:

$ cd apache-tomcat-6.0.20
$  ls -l 
total 88
drwxr-xr-x 2 thredds workshop  4096 2009-08-03 16:04 bin
drwxr-xr-x 2 thredds workshop  4096 2009-05-13 17:15 conf
drwxr-xr-x 2 thredds workshop  4096 2009-08-03 16:04 lib
-rw-r--r-- 1 thredds workshop 37951 2009-05-13 17:15 LICENSE
drwxr-xr-x 2 thredds workshop  4096 2009-05-13 17:15 logs
-rw-r--r-- 1 thredds workshop   556 2009-05-13 17:15 NOTICE
-rw-r--r-- 1 thredds workshop  7317 2009-05-13 17:15 RELEASE-NOTES
-rw-r--r-- 1 thredds workshop  6368 2009-05-13 17:15 RUNNING.txt
drwxr-xr-x 2 thredds workshop  4096 2009-08-03 16:04 temp
drwxr-xr-x 7 thredds workshop  4096 2009-05-13 17:15 webapps
drwxr-xr-x 2 thredds workshop  4096 2009-05-13 17:15 work

2. Familiarize yourself with the following important directories.

$TOMCAT_HOME/bin

• Contains startup.sh, shutdown.sh and other scripts/programs.
• The *.sh files (for Unix systems) are functional duplicates of the *.bat files (for Windows systems).

$TOMCAT_HOME/conf

• Server-wide Tomcat configuration.
• You will modify server.xml and tomcat-users.xml to adjust logging, authentication and access control, enable SSL, etc.
• Web applications can override some server-wide settings in their own configuration files (more about that later).

$TOMCAT_HOME/logs

• Tomcat log files are here by default.
• This is one of the directories you will be looking for log files (the TDS logs elsewhere by default).
• The log files should be your first stop for troubleshooting Tomcat and TDS issues. (Hint, hint.)
• Logs files may contain useful information for assessing the security of your system.
• You will become very familiar with the Tomcat-generated catalina.out and access.yyyy-mm-dd.log files by the end of this workshop.

$TOMCAT_HOME/webapps

• Contains web applications directories and WAR files.
• This is where we will be putting the TDS web application.
• You will also be using the manager application that comes with Tomcat during this workshop.

Starting & stopping Tomcat

1. Tomcat isn't currently running so we need to start it up.

Move into the $TOMCAT_HOME/bin/ directory and run the startup.sh script:

$ cd $TOMCAT_HOME/bin
$  ./startup.sh

2. Verify Tomcat is running.

Look and see if you have a Tomcat process running:

$ ps -ef | grep tomcat
thredds   4443     1 21 16:08 pts/0    00:00:03 /usr/bin/java -Djava.util.logging.config.file=/home/thredds/apache-tomcat-6.0.20/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/home/thredds/apache-tomcat-6.0.20/endorsed -classpath :/home/thredds/apache-tomcat-6.0.20/bin/bootstrap.jar -Dcatalina.base=/home/thredds/apache-tomcat-6.0.20 -Dcatalina.home=/home/thredds/apache-tomcat-6.0.20 -Djava.io.tmpdir=/home/thredds/apache-tomcat-6.0.20/temp org.apache.catalina.startup.Bootstrap start

Open a new browser window/tab and go to http://localhost:8080/ to verify Tomcat is running:



3. See if you can shutdown Tomcat.

Run the $TOMCAT_HOME/bin/shutdown.sh script:

$ $TOMCAT_HOME/bin/shutdown.sh

Setting $JAVA_HOME, $JAVA_OPTS and $CATALINA_HOME

We are going to create a file called setenv.sh in the $TOMCAT_HOME/bin directory to:

• allow Tomcat to reference/find the location of $JAVA_HOME and $TOMCAT_HOME (aka $CATALINA_HOME) during startup and shutdown; and
• increase the amount of memory allocated to the JVM to enhance performance by setting $JAVA_OPTS.

1. Create setenv.sh file.

Using your favorite text editor (gedit, vi, emacs, nano, etc.), create a new file called setenv.sh in $TOMCAT_HOME/bin:

$  pwd 
/home/thredds/apache-tomcat-6.0.20/bin/
$ vi setenv.sh

Add the following information and save setenv.sh:

#!/bin/sh
#
# ENVARS for Tomcat and TDS environment
#
JAVA_HOME="/home/thredds/jdk1.6.0_15"
export JAVA_HOME
JAVA_OPTS="-Xmx1500m -Xms512m -server -Djava.awt.headless=true"
export JAVA_OPTS
CATALINA_HOME="/home/thredds/apache-tomcat-6.0.20"
export CATALINA_HOME

Settings

Whenever possible, Unidata recommends -Xmx1500m for 32-bit systems, and -Xmx2048m - -Xmx4096m for 64-bit systems.

The parameters we pass to $JAVA_OPTS:

• -Xms is the initial allocated memory of the JVM (for performance).
• -Xmx the maximum allocated memory of the JVM (for performance).
• -server tells the Hostspot compiler to run the JVM in "server" mode.
• -Djava.awt.headless=true is needed to prevent graphics rendering code to assume that a graphics console exists. Without this, WMS code will crash the server in some circumstances.
2. Restart Tomcat to implement your changes.

Restart Tomcat and examine the output generated to the terminal window by the startup script:

$ $TOMCAT_HOME/bin/startup.sh
sing CATALINA_BASE:   /home/thredds/apache-tomcat-6.0.20
Using CATALINA_HOME:   /home/thredds/apache-tomcat-6.0.20
Using CATALINA_TMPDIR: /home/thredds/apache-tomcat-6.0.20/temp
Using JRE_HOME:       /home/thredds/jdk1.6.0_15

Did you notice any difference in the what is being reported to STDOUT during startup?

Take a look at the running Tomcat process to see the new $JAVA_OPTS settings:

$ ps -ef | grep tomcat
dds   4606     1 99 16:15 pts/0    00:00:03 /home/thredds/jdk1.6.0_15/bin/java -Djava.util.logging.config.file=/home/thredds/apache-tomcat-6.0.20/conf/logging.properties -Xmx256m -Xms256m -server -Djava.awt.headless=true -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/home/thredds/apache-tomcat-6.0.20/endorsed -classpath :/home/thredds/apache-tomcat-6.0.20/bin/bootstrap.jar -Dcatalina.base=/home/thredds/apache-tomcat-6.0.20 -Dcatalina.home=/home/thredds/apache-tomcat-6.0.20 -Djava.io.tmpdir=/home/thredds/apache-tomcat-6.0.20/temp org.apache.catalina.startup.Bootstrap start
thredds   4626  4207  0 16:15 pts/0    00:00:00 grep tomcat

What allows us to create the setenv.sh file and have its contents read? (Hint: have a look at the $TOMCAT_HOME/bin/catalina.sh starting around line 110).

Troubleshooting tips

• Do not forget include the 'm' in your -Xms and -Xmx settings.
• You may have allocated too much memory for the JVM settings if Tomcat fails to start and you get the following error reported in the Tomcat log catalina.out:

Error occurred during initialization of VM
Could not reserve enough space for object heap

• Likewise, if there is an error with your JVM memory allocation syntax in the setenv.sh file, it will be reported to catalina.out:

Error occurred during initialization of VM
Incompatible minimum and maximum heap sizes specified

About WAR files

• WAR is short for Web ARchive.
• By default, Tomcat will automatically unpack the WAR distribution into directory of the same name upon deployment.
• Note: the unpacked directory is overwritten each time a new WAR file is deployed.

Downloading & deploying thredds.war

1. Download the TDS WAR file from Unidata's web site.

We will be using the current TDS 4.0 (stable) version for this workshop:

thredds.war

2. Deploy the TDS in Tomcat.

Put thredds.war in the Tomcat webapps/ directory:

$ cp thredds.war $TOMCAT_HOME/webapps
$ cd $TOMCAT_HOME/webapps

$ ls -l
total 22820
drwxr-xr-x 10 thredds workshop     4096 2009-08-03 16:04 docs
drwxr-xr-x  5 thredds workshop     4096 2009-08-03 16:04 examples
drwxr-xr-x  5 thredds workshop     4096 2009-08-03 16:04 host-manager
drwxr-xr-x  5 thredds workshop     4096 2009-08-03 16:04 manager
drwxr-xr-x  3 thredds workshop     4096 2009-08-03 16:04 ROOT
-rw-r--r--  1 thredds workshop 23317222 2009-08-03 16:22 thredds.war

3. Confirm the TDS has been deployed.

If Tomcat is already running, wait a couple of seconds after placing the WAR file in webapps/ and then verify the thredds.war file was unpacked:

$ $TOMCAT_HOME/bin/startup.sh

$ ls -l
total 22824
drwxr-xr-x 10 thredds workshop     4096 2009-08-03 16:04 docs
drwxr-xr-x  5 thredds workshop     4096 2009-08-03 16:04 examples
drwxr-xr-x  5 thredds workshop     4096 2009-08-03 16:04 host-manager
drwxr-xr-x  5 thredds workshop     4096 2009-08-03 16:04 manager
drwxr-xr-x  3 thredds workshop     4096 2009-08-03 16:04 ROOT
drwxr-xr-x  5 thredds workshop     4096 2009-08-03 16:23 thredds
-rw-r--r--  1 thredds workshop 23317222 2009-08-03 16:22 thredds.war

Go to http://localhost:8080/thredds/ in your browser to verify the TDS has been deployed:

$TOMCAT_HOME/logs

1. Look at the different types of log files being generated in $TOMCAT_HOME/logs.

Move into the logs directory to see the type of information being logged:

$ cd $TOMCAT_HOME/logs
$ ls -l
total 12
-rw-r--r-- 1 thredds workshop 2952 2009-08-03 16:23 catalina.2009-08-03.log
-rw-r--r-- 1 thredds workshop 3331 2009-08-03 16:23 catalina.out
-rw-r--r-- 1 thredds workshop    0 2009-08-03 16:08 host-manager.2009-08-03.log
-rw-r--r-- 1 thredds workshop 2030 2009-08-03 16:23 localhost.2009-08-03.log
-rw-r--r-- 1 thredds workshop    0 2009-08-03 16:08 manager.2009-08-03.log

Do you see a correspondence between some of the web applications in the $TOMCAT_HOME/webapps directory and the naming of certain log files?

Is there a difference in the information being logged to catalina.out versus catalina.yyyy-mm-dd.log?

Are some log files more verbose than others?

2. Examining catalina.out.

Open another terminal window (hereafter referred to as terminal #2) and run the following command in the new terminal:

$ tail -f /home/thredds/apache-tomcat-6.0.20/logs/catalina.out

In your original terminal window, start/stop and start Tomcat and watch what is being logged to catalina.out in the terminal #2 window.

Is it only errors messages being reported to catalina.out?

What messages in catalina.out are from the TDS?

Things to know about catalina.out

• Tomcat System.out and System.err gets appended to catalina.out.
• catalina.out can quickly grow large if the hosted web applications are not specifically catching and logging System.out and System.err to designated files.
• catalina.out is not automatically rotated in Tomcat.
• You will need to employ an outside log rotation program (e.g., logadm or logrotate) to rotate catalina.out.