DEV Community

Jesús G. Calderín
Jesús G. Calderín

Posted on

Setup Tomcat cluster in Linux

#devops #tutorial #linux #java

Prerequisites

Install Java

And set JAVA_HOME variable accordingly in ~/.bashrc:

export JAVA_HOME=/opt/jdk1.8.0_451
Enter fullscreen mode Exit fullscreen mode

Install Tomcat

And set CATALINA_HOME accordingly in ~/.bashrc:

export CATALINA_HOME: /opt/apache-tomcat-9.0.109
Enter fullscreen mode Exit fullscreen mode

Create two instances of our Tomcat

(Kudos to Goran Vasic)

They will be the two nodes of our cluster.

First, we create one empty directory for our first tomcat:

$ mkdir -p /apps/myapp/clus1/tomcat1/
Enter fullscreen mode Exit fullscreen mode

Then, we copy inside the following folders from CATALINA_HOME:

  • /conf
  • /webapps 
$ cp -R $CATALINA_HOME/conf    /apps/myapp/clus1/tomcat1/
$ cp -R $CATALINA_HOME/webapps /apps/myapp/clus1/tomcat1/
Enter fullscreen mode Exit fullscreen mode

We will also need to create an empty logs folder.

$ mkdir -p /apps/myapp/clus1/tomcat1/logs
Enter fullscreen mode Exit fullscreen mode

Finally, we create start and stop scripts that set our CATALINA_BASE accordingly:

$ cat /apps/myapp/clus1/tomcat1/startup.sh
export CATALINA_BASE="/apps/myapp/clus1/tomcat1"
cd $CATALINA_HOME/bin
TITLE=tomcat1
./startup.sh $TITLE

$ cat /apps/myapp/clus1/tomcat1/shutdown.sh
export CATALINA_BASE="/apps/myapp/clus1/tomcat1"
cd $CATALINA_HOME/bin
./shutdown.sh
Enter fullscreen mode Exit fullscreen mode

We now turn the files executable:

$ chmod u+x /apps/myapp/clus1/tomcat1/startup.sh
$ chmod u+x /apps/myapp/clus1/tomcat1/shutdown.sh
Enter fullscreen mode Exit fullscreen mode

And test them:

$ /apps/myapp/clus1/tomcat1/startup.sh
Using CATALINA_BASE:   /apps/myapp/clus1/tomcat1
Using CATALINA_HOME:   /opt/apache-tomcat-9.0.109
Using CATALINA_TMPDIR: /apps/myapp/clus1/tomcat1/temp
Using JRE_HOME:        /opt/jdk1.8.0_451
Using CLASSPATH:       /opt/apache-tomcat-9.0.109/bin/bootstrap.jar:/opt/apache-tomcat-9.0.109/bin/tomcat-juli.jar
Using CATALINA_OPTS:
Tomcat started.

$ /apps/myapp/clus1/tomcat1/shutdown.sh
Using CATALINA_BASE:   /apps/myapp/clus1/tomcat1
Using CATALINA_HOME:   /opt/apache-tomcat-9.0.109
Using CATALINA_TMPDIR: /apps/myapp/clus1/tomcat1/temp
Using JRE_HOME:        /opt/jdk1.8.0_451
Using CLASSPATH:       /opt/apache-tomcat-9.0.109/bin/bootstrap.jar:/opt/apache-tomcat-9.0.109/bin/tomcat-juli.jar
Using CATALINA_OPTS:
Enter fullscreen mode Exit fullscreen mode

We now need to create the same start/shutdown scripts for tomcat2, but bear in mind that since we are running our tomcats on the same machine, we need to use different listening and shutdown ports in each of them, otherwise the start and stop will fail. In my case, I'm going to replace default 8080 by 9080 in tomcat2's $CATALINA_BASE/conf/server.xml listening port:

<Connector port="9080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443"
maxParameterCount="1000"
/>

, and replace 8005 by 9005 in shutdown's port:

<Server port="9005" shutdown="SHUTDOWN">
<Listener className="org.apache.catalina.startup.VersionLoggerListener" />...

After starting both tomcats, we can see the ports they are using:

$ sudo netstat -nlp|grep -E '8080|9080'
tcp6       0      0 :::8080  :::*   LISTEN      20941/java
tcp6       0      0 :::9080  :::*   LISTEN      21014/java

$ lsof -i -P |grep 20941
java    20941 TCP *:8080 (LISTEN)
java    20941 TCP localhost:8005 (LISTEN)

$ lsof -i -P |grep 21014
java    21014  TCP *:9080 (LISTEN)
java    21014  TCP localhost:9005 (LISTEN)
Enter fullscreen mode Exit fullscreen mode

Activate the cluster

We just need to uncomment the Cluster element under Engine in both tomcat's $CATALINA_BASE/conf/server.xml file:

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>

, and restart.
This will make them multicast their membership to default 228.0.0.4 and port 45564, and will listen on the first port available between 4000-4100:

$ sudo netstat -nlp|grep 45564
udp6       0      0 :::45564    :::*  13411/java
udp6       0      0 :::45564    :::*  13273/java
$ lsof -i -P |grep 13411
java    13411 TCP *:9080 (LISTEN)
java    13411 TCP DESKTOP-IUMEID0.:4001 (LISTEN)
java    13411 UDP *:45564
java    13411 TCP localhost:42728->DESKTOP-IUMEID0.:4000 (ESTABLISHED)
java    13411 TCP DESKTOP-IUMEID0.:4001->localhost:36104 (ESTABLISHED)
java    13411 TCP localhost:9005 (LISTEN)
$ lsof -i -P |grep 13273
java    13273  TCP *:8080 (LISTEN)
java    13273  TCP DESKTOP-IUMEID0.:4000 (LISTEN)
java    13273  UDP *:45564
java    13273  TCP localhost:8005 (LISTEN)
java    13273  TCP DESKTOP-IUMEID0.:4000->localhost:42728 (ESTABLISHED)
java    13273  TCP localhost:36104->DESKTOP-IUMEID0.:4001 (ESTABLISHED)
Enter fullscreen mode Exit fullscreen mode

In the output above, we can see that both tomcats are communicating with each other via the ports 4000 and 4001 respectively:

java    13411 TCP localhost:42728->DESKTOP-IUMEID0.:4000 (ESTABLISHED)
java    13411 TCP DESKTOP-IUMEID0.:4001->localhost:36104 (ESTABLISHED)
java    13273  TCP DESKTOP-IUMEID0.:4000->localhost:42728 (ESTABLISHED)
java    13273  TCP localhost:36104->DESKTOP-IUMEID0.:4001 (ESTABLISHED)
Enter fullscreen mode Exit fullscreen mode

N.B: DESKTOP-IUMEID0 is just the name of my computer.

So that's it. Both tomcats now belong to the same cluster.

Enable session replication

The cluster is not very useful unless its nodes can replicate sessions between them. This would allow a session that was established in one node to be recoverable on the other node when the former dies out.
To test session replication, I will use the examples webapp that comes with out-of-the-box tomcat.

The session replication is enabled by simply adding

<distributable/>

under element web-app of $CATALINA_BASE/webapps/examples/WEB-INF/web.xml in both tomcats:

<web-app xmlns="http://xmln..."
  xmlns:xsi="http://www.w3...."
  xsi:schemaLocation="http:..."

  version="4.0"
  metadata-complete="true">

    <distributable/>
    ......
Enter fullscreen mode Exit fullscreen mode

Test session replication

After restarting we can use curl to establish a session in tomcat1 by sending a request to servlet SessionExample:

$ curl http://localhost:8080/examples/servlets/servlet/SessionExample

<!DOCTYPE html>
<html>
....
<h3>Sessions Example</h3>
Session ID: 104F520BB401ED856048700D302D255A
...
The following data is in your session:<br>
....
</html>
Enter fullscreen mode Exit fullscreen mode

We can now use the cookie JESSIONID to verify that the session also exists in tomcat2:

$ curl --cookie "JSESSIONID=104F520BB401ED856048700D302D255A" \
http://localhost:9080/examples/servlets/servlet/SessionExample

<!DOCTYPE html>
<html>
....
<h3>Sessions Example</h3>
Session ID: 104F520BB401ED856048700D302D255A
...
The following data is in your session:<br>
....
</html>
Enter fullscreen mode Exit fullscreen mode

We will now put some data into the session via tomcat2 and verify if it is replicated in tomcat1:

#update the session via TOMCAT 2 (note that port=9080)
$ curl --data "dataname=james&datavalue=bond" \
--cookie "JSESSIONID=104F520BB401ED856048700D302D255A" \
http://localhost:9080/examples/servlets/servlet/SessionExample

<!DOCTYPE html>
<html>
....
<h3>Sessions Example</h3>
Session ID: 104F520BB401ED856048700D302D255A
...
The following data is in your session:<br>
james = bond
...
</html>

#recover the updated session via TOMCAT 1 (note that port=8080)
$ curl --cookie "JSESSIONID=104F520BB401ED856048700D302D255A" \ 
http://localhost:8080/examples/servlets/servlet/SessionExample

<!DOCTYPE html>
<html>
....
<h3>Sessions Example</h3>
Session ID: 104F520BB401ED856048700D302D255A
...
The following data is in your session:<br>
james = bond
...
</html>
Enter fullscreen mode Exit fullscreen mode

So now we have a cluster of 2 tomcats that can replicate sessions between them. Not very useful, though, if we have to manually change the URL's and the JESSIONID cookie whenever one tomcat dies out or is responding slowly.

I'll soon cover how to front these tomcats with a web server that will provide us with new important functionalities:

  • request forwarding: aka as reverse-proxy
  • load balancing: it will distribute the requests between the tomcats according, for example, to how busy they are
  • sticky sessions: it will always forward the requests pertaining to the same server session to the node that initiated it
  • fail-over: it will detect when one tomcat is down and refrain from sending it any requests until it is up again

Top comments (0)