Remotely monitor Tomcat clusters using MC4J

Add the power of JMX to Tomcat

J2SE 5.0's monitoring and management support is comprehensive, with remote monitoring capabilities for the Java platform and for applications that run on it. Included in J2SE 5.0 is Java Management Extensions (JMX) remote monitoring, a new addition to the list of monitoring capabilities on a JVM. Java Specification Request 160 is the specification for JMX remote monitoring. It focuses on the mechanism for remote access by building a remote client API for a JMX-based agent.

Tomcat 5.5 is designed to leverage J2SE 5.0's built-in JMX capabilities. Version 5.5 (which was branched based on Tomcat 5.0.27) implements the latest Servlet (2.4) and JavaServer Pages (2.0) specifications and is the result of an extensive redesign and refactoring of the Tomcat server architecture. It proves more stable and improves upon Tomcat 4.x with enhanced performance, scalability and reliability, JMX monitoring, integrated session clustering, and application deployment.

With the latest version of Tomcat and J2SE 5.0's JMX capabilities, we can make the servlet container's attributes and methods available via JMX while simultaneously reducing the complexity of JMX-related code. In this article, I discuss how to launch the Tomcat servlet container with remote JMX monitoring enabled. We also look at a sample Web application that runs in a Tomcat cluster with session replication turned on. And, finally, we use a JMX client to view the details of cluster elements and HTTP sessions. But before we delve into the discussion of remote JMX monitoring, let's look at the components that make up a Tomcat cluster.

Cluster elements

A Tomcat cluster consists of six main components: cluster, membership, sender, receiver, replication valve, and deployer.

Table 1 shows how each component makes the clustering and session replication work in Tomcat.

Table 1. Tomcat cluster elements

Cluster elementClass nameDescription
Clusterorg.apache.catalina.cluster.tcp.SimpleTcpClusterMain element in the cluster configuration. Cluster creates the ClusterManager for all the Web contexts that have distributable tags in the web.xml.
Membershiporg.apache.catalina.cluster.mcast.McastServiceCluster membership is established by the Tomcat instances sending IP multicast pings. A multicast broadcast message contains the IP address and the server's TCP listen port. If an instance has not received the message within a given time frame, the member is considered unavailable. The attributes starting with mcastXXX are for the membership multicast ping.
Senderorg.apache.catalina.cluster.tcp.ReplicationTransmitterWhen session replication is requested, a Tomcat instance will use the host and port information and establish a TCP socket. Using this socket, the instance sends over the session information as serialized data. There are three modes of session replication in Tomcat 5.5: asynchronous, synchronous, and pooled.
Receiverorg.apache.catalina.cluster.tcp.ReplicationListenerThis element is responsible for the actual session replication. It listens on a specified IP address and port number for TCP cluster requests.
Replication valveorg.apache.catalina.cluster.tcp.ReplicationValveThe replication valve parses the HTTP requests and filters the files that do not need to be replicated. We don't want the session replication to trigger for static files such as HTML, JavaScript, Cascading Stylesheets, PDF, and images. We can filter these file types by specifying the list of file extensions that we don't want to replicate across the cluster.
Deployerorg.apache.catalina.cluster.deploy.FarmWarDeployerThis is a new element added to Tomcat cluster in the later versions of 5.0.x. It can be used for cluster-wide deployment of Web applications.

Fore more details on the clustering elements, refer to Tomcat 5.5's clustering documentation.

Tomcat 5.5 provides several improvements over its predecessors (Tomcat versions 4.1 and 5.0) in terms of clustering, session replication, and server monitoring and manageability. In Tomcat 4, several server components (such as host, engine, and service) could be monitored using MBeans (managed beans). But in Tomcat 5.5, Yoav Shapira, Filip Hanik, and the other Tomcat developers wrote JMX implementations for the cluster elements.

Now that we know the function of each element in a Tomcat cluster, let's look at various JMX client tools used for connecting to a Tomcat server cluster and monitoring the cluster details.

JMX clients

A JMX client is a graphical user interface (client/server or thin client) used to connect to a JMX agent (running on a local or remote machine). An ideal JMX client should have the following features to monitor an application server effectively without incurring any additional overhead:

  • Low overhead on system and network resources
  • The ability to maintain system stability and performance
  • Minimum or no special configuration needed (if some settings must be configured, they should be configured declaratively rather than programmatically)
  • Good reporting features

J2SE 5.0 comes with a JMX client tool called JConsole that can be used to look into the runtime JVM details. Tomcat installation includes a JMX servlet called JMXProxyServlet that can view and update Tomcat MBean attributes. It is a lightweight proxy for viewing and manipulating the MBeans running in a Tomcat container and proves helpful in integrating command line scripts for monitoring and changing Tomcat internals. JMX Query and Set commands can be used to query the MBeans and modify their attributes and operations respectively.

Other than these two tools, several third-party open source JMX client applications are available (links to which are available in Resources):

  • XMOJO
  • jManage
  • MX4J
  • Spring JMX
  • JMX-HTML adaptor
  • MC4J JMX Console

In this article, I discuss how to install and configure MC4J to remotely connect to a Tomcat servlet container and monitor all the MBean components in a server cluster.

JMX remote monitoring and management using MC4J

The MC4J console provides the following capabilities:

  • Tree view of MBeans, their attributes, operations, and notifications
  • Ability to set values on MBean attributes
  • Ability to execute MBeans operations
  • Connections to multiple J2EE application servers
  • Visual graphs of MBean attributes

I used MC4J 1.2 Beta 9 for this article's sample application. To install MC4J, download the executable (MC4J-12b9-Windows.exe) from SourceForge. Double-click on the exe file and select JDK Home and the MC4J installation directory (c:\dev\tools in the sample setup) when prompted during the installation process.

Set system properties for JMX

You must specify some system properties to see the remote JMX monitoring in action. These properties are specified in a properties file called management.properties located in the JAVA_HOME/lib/management directory. Table 2 shows the properties that must be set for JMX monitoring.

Table 2. System properties to enable JMX monitoring

Property nameValue
com.sun.management.jmxremote.port8999
com.sun.management.jmxremote.password.fileDefault location (JAVA_HOME/lib/management/jmxremote.password)
com.sun.management.jmxremote.access.fileDefault location (JAVA_HOME/lib/management/jmxremote.access)
com.sun.management.jmxremote.sslfalse

Setting the password file is very important especially in a multi-user environment. I set up the password file in the JAVA_HOME/lib/management directory as follows:

  • Copy the password template file (called jmxremote.password.template) into a file called jmxremote.password
  • Set file permissions so that only you can read and write the password file
  • Add passwords for the roles, such as monitorRole and controlRole
  • Set this system property when you start the JVM as shown in Table 2

Tomcat cluster configuration

I installed Tomcat 5.5.9 to set up the Tomcat cluster. The server cluster used in this article includes two Tomcat instances sharing the session state and a load balancer to distribute the requests between the server nodes. I used SimpleTcpCluster and DeltaManager (which are default options) in the cluster configuration. I explain the cluster setup in more detail in my series "Clustering and Load Balancing in Tomcat" (ONJava.com, 2004).

To enable remote JMX monitoring when starting Tomcat server, modify the Tomcat startup script (catalina.bat or catalina.sh) located in the CATALINA_HOME/bin directory as shown below:

 set JAVA_OPTS=%JAVA_OPTS% -Dcom.sun.management.jmxremote.port=8999

Note: You must use a different port for the jmxremote port option on the second Tomcat instance (9999).

Figure 1 shows the cluster topology details.

Figure 1. Tomcat cluster architecture diagram. Click on thumbnail to view full-sized image.

Table 3 lists various configuration parameters used in setting up the Tomcat cluster.

Table 3. Cluster configuration details

ParameterCluster Node 1Cluster Node 2
Directoryc:\dev\tomcat-5.5.9-instance1c:\dev\tomcat-5.5.9-instance2
Server port80059005
Connector port80809080
Coyote/JK2 AJP connector port80099009
JMX port89999999
Multicast IP address228.0.0.4228.0.0.4
Multicast port4556445564
TCP listener IP address127.0.0.1127.0.0.1
TCP listener port40004001
JDK directoryC:\dev\java\jdk1.5.0C:\dev\java\jdk1.5.0

Connect to the JMX agent programmatically

Before we dive into the details of Tomcat monitoring using MC4J, let's briefly discuss a sample Java application that connects to a remote JMX agent (in this case, a Tomcat servlet container) using the remote JMX API. Begin by starting Tomcat with JMX monitoring enabled.

The sample JMX remote client is called RemoteJMXClient, which is basically a standalone Java application that acts as a JMX connector. This Java class is located in the sample Web application's src\com\remotejmx\client directory. Add the jmx-remote.jar and jmxri.jar files in the classpath when running the Java application. The following steps explain how to remotely connect to the JMX server:

  1. Start by creating JMXServiceURL with the protocol, host name, remote JMX port number, and credentials HashMap (with username and password):

     

    JMXServiceURL url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://localhost:8999/jmxrmi");

    Map map = new HashMap(); String[] credentials = new String[] { "monitorRole" , "QED" }; map.put("jmx.remote.credentials", credentials);

  2. Create a JMXConnector using the URL and credentials HashMap defined in Step 1. Once you obtain a reference to the JMX connector, call the getConnectionId() method to ensure you have a valid connection ID:

     

    JMXConnector conn = JMXConnectorFactory.connect(url, map); System.out.println("JMXConnector="+conn.toString());

    String id = conn.getConnectionId(); System.out.println("Connection Id=" + id);

  3. Next, get an MBeanServerConnection from the JMXConnector object:

      mbsc = conn.getMBeanServerConnection();
       String domains[] = mbsc.getDomains();
       System.out.println("# of domains="+domains.length);
       for (int i = 0; i < domains.length; i++) { 
          System.out.println("Domain[" + i + "] = " + domains[i]); 
       }
    
    
  4. Once you have the MBeanServerConnection object, call the MBeans-related methods just as you would call on an MBeanServer if connected to the JMX server locally (within the same JVM). You can check the number of domains and their types in the JMX server. You can also get the number of MBeans registered in the server and their attributes and operations. The code snippet below shows these steps:

     

    mbsc = conn.getMBeanServerConnection(); String domains[] = mbsc.getDomains(); System.out.println("# of domains="+domains.length); for (int i = 0; i < domains.length; i++) { System.out.println("Domain[" + i + "] = " + domains[i]); }

    // Get MBean count Integer mbeanCount = mbsc.getMBeanCount(); System.out.println("mbeanCount : " + mbeanCount.intValue());

  5. Next, query the MBeans in the server and display their attributes and operations. Also retrieve the MBeans details for Cluster's object type, as shown in the code below. We look at the same cluster details using MC4J in the following section.

     

    Set mBeanSet = mbsc.queryMBeans(null, null); System.out.println("mBeanSet.size() : " + mBeanSet.size());

    Iterator mBeanSetIterator = mBeanSet.iterator(); while (mBeanSetIterator.hasNext()) { ObjectInstance objectInstance = (ObjectInstance)mBeanSetIterator.next(); ObjectName objectName = objectInstance.getObjectName(); String canonicalName = objectName.getCanonicalName(); System.out.println("canonicalName : " + canonicalName);

    if (canonicalName.equals("Catalina:host=localhost,type=Cluster")) { // Get details of cluster MBean System.out.println("Cluster MBean Details:"); System.out.println("========================================="); getMBeanDetails(canonicalName); }

    String canonicalKeyPropList = objectName.getCanonicalKeyPropertyListString(); }

  6. Finally, close the JMX MBean connection to clean up the resources:

      conn.close();
    
    

Sample Web application setup

This section explains the sample Web application used to test the failover and session replication in the Tomcat cluster. I deployed the Web application on both cluster nodes and wrote a test client program to run a load test by creating and modifying HTTP sessions in the servlet container.

The following steps get the server cluster and load balancer up and running:

1 2 Page
Recommended
Join the discussion
Be the first to comment on this article. Our Commenting Policies
See more