Systematically shut down enterprise Java Web applications

Tools and techniques for ensuring an orderly shutdown of Web applications

Many software developers do not pay enough attention to the shutdown (also referred to as undeploy) of the applications they write. In fact, the most popular shutdown mechanism is to execute Ctl+C or a kill -9. However, if you are writing an application that coexists with other applications on the same application server, then a kill of the application server kills all the resident applications. You may need the ability to shut down your application, ensure that all resources consumed by it are released, and clear its memory footprint while leaving the other deployed applications running. In this article, I provide code samples for how to do that, and identify a set of checks and techniques.

Besides releasing resources, another important aspect of the shutdown process is ensuring your system is in a consistent state and mission-critical data is not lost. If your application processes transactions via Web service calls or batch loads, it is important that you bring those ongoing transactions to a halt smoothly; this article examines some techniques that can help you achieve that goal. Further, I discuss the usage of tools like JConsole and Borland Optmizeit to ensure a clean shutdown.

I will desist from presenting a formal definition of Web application shutdown. But, for practical purposes, one can say that an application has shut down when all the threads either started by it or servicing it have terminated their processing, and when memory used by the application during its lifetime, including object instances and class definitions, becomes unreachable from any of the currently live threads. Effectively, the memory used by the application during its lifetime becomes a candidate for garbage collection.

Shutdown event propagation

Application server front ends are available that allow you to deploy and undeploy Web applications. In the case of JBoss and Tomcat, the shutdown front ends are based on Java Management Extensions (JMX): when your application is deployed, the application server creates a managed bean (MBean) to manage its lifecycle, and when you undeploy an application, that MBean's undeploy method is invoked. You can also invoke such MBean operations from command line utilities. In addition to the JBoss JMX Console, JBoss supplies a utility called twiddle that invokes MBean operations from the command line.

When you undeploy a Web application, the container (through MBeans, in the case of JBoss) ultimately calls the destroy method on all active servlets. Applications typically have a singleton launcher servlet that performs the initialization and is placed as the first servlet definition in the web.xml file. Servlets are initialized in the order in which they are defined in web.xml and destroyed in reverse order. Cleanup code can be written in the launcher servlet's destroy method.

In standalone applications, you can notify your code to initiate an orderly shutdown through JVM hooks.

In this article, I take the case of a simple Web application that helps a database operations administrator request for, analyze, or truncate tables via a Web UI, and illustrate how shutdown is implemented. The operations requested via this application can take a long time to complete and are handled asynchronously by a thread pool, as illustrated in the code below:

 

public class LauncherServlet extends HttpServlet { static final String COMMAND = "command"; static final String TABLE_NAME = "table_name"; static final String ANALYZE_TABLES = "analyze_table"; static final String TRUNCATE_TEMP_TABLE = "truncate_temp_table"; private ThreadPoolManager m_threadPoolManager = null; private List<Statement> m_statementRegistry = null; private List<Connection> m_connectionRegistry = null; public enum ApplicationState { Unitialized, Running, ShutdownRequested, Shutdown }; static ApplicationState appState = ApplicationState.Unitialized; public LauncherServlet() { } /** * Container calls init during servlet load. */ public void init(ServletConfig servletConfig) throws ServletException { super.init(servletConfig); //Initialize your Thread pool. m_threadPoolManager = new ThreadPoolManager(); //Application initialization code. this.m_statementRegistry = new Vector<Statement>(); this.m_connectionRegistry = new Vector<Connection>(); setAppState(ApplicationState.Running); } /**

* Get */ public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { String command = request.getParameter(COMMAND); String table = request.getParameter(TABLE_NAME); String query = null; if (command != null && command.equals(ANALYZE_TABLES) ) { query = "ANALYZE TABLE " + table + " COMPUTE STATISTICS"; } else if (command != null && command.equals(TRUNCATE_TEMP_TABLE)) { query = "TRUNCATE TABLE " + table; } else { throw new ServletException("Command:" + command + " not recognized"); } SQLExecutor sqlExec = new SQLExecutor(this.m_connectionRegistry, this.m_statementRegistry, query); m_threadPoolManager.submitTaskForExecution(sqlExec); //In a real world application, an ID would be passed back to the client to track the status of the request. response.getOutputStream().println("<html><BODY><B> Request Submitted for Execution <B> <BODY></HTML>"); response.getOutputStream().flush(); } /** * Post */ public void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { doGet(request,response); } /** * Container calls destroy at the end of the Servlet lifecycle. */ public void destroy() { //Set the application state. setAppState(ApplicationState.ShutdownRequested); //First shut down the thread pool, this call will attempt to //kill all the worker threads that are performing tasks. Be aware that //there is no guarantee that all the threads will die after this method call. try { m_threadPoolManager.shutdown(); } catch (InterruptedException e) { //The current thread that is calling destroy itself //may be interrupted by the container. In this case, //we want to do as much cleanup as possible so we will consume //this exception and not propagate it. System.out.println("Thread " + Thread.currentThread().getName() + " received interrupt, thread pool destroy was not complete"); } //Clean up all the resources. //Doing so, also increases the likelihood that any live threads at this point will die. //In our example, the resources are JDBC Connection and Statement objects. for (Statement stmt : this.m_statementRegistry) { try { //Statement.cancel will cancel the statement and will cause //the thread that is using this statement to get an SQLException. stmt.cancel(); stmt.close(); } catch (SQLException e) { ; //Move on and try to clean up other Statement objects. } } for (Connection conn : this.m_connectionRegistry) { try { //Good idea to rollback as some database systems will commit on close; //an attempt to commit might run into locking problems in some situations. conn.rollback(); conn.close(); } catch (SQLException e) { ; //Move on and try to clean up other Connection objects. } } //Unregister any components (MBeans for example) that you may have registered //with the container. setAppState(ApplicationState.Shutdown); System.out.println("Shutdown Complete"); }

public synchronized static ApplicationState getAppState() { return appState; }

public synchronized static void setAppState(ApplicationState appState) { LauncherServlet.appState = appState; } }

Shutdown sequence

The shutdown sequence can be implemented in two phases. In the first phase, set the application state to indicate that shutdown has been initiated and proceed to execute Thread.interrupt on all the threads your application has started (the subsequent section examines the details)—waiting a brief amount of time for the threads to die. However, you cannot be sure that they will die in the specified amount of time; thus, the next phase will help here.

In the second phase, clean up and undeploy all the resources that your application used, e.g., Java Database Connectivity (JDBC) connection pools, Java Message Service (JMS) session pools, or sockets. At this stage, any threads that are still alive and using resources should receive exceptions and die. Finally, unregister components that may have been registered with the container such as MBeans. Figure 1 illustrates the sequence diagram for shutdown.

Figure 1. Shutdown sequence diagram. Click on thumbnail to view full-sized image.

Thread termination

Thread termination is the first task to initiate as part of the shutdown sequence. This may also be the most challenging task especially if you are using a set of third-party tools that may spawn threads carelessly. Keep in mind that the Enterprise JavaBeans specification prohibits enterprise beans from starting threads; however, many non-EJB Web applications create threads that perform asynchronous processing of requests like the sample database administrator application. Instead of creating threads in your application, use a thread pool or a thread factory class that has a handle to all the threads your application has started. If you use J2SE 5.0 or later, use the java.util.concurrent.ThreadPoolExecutor as your thread pool. During shutdown, you can then call ThreadPoolExecutor.shutdownNow as shown in the following code listing:

 public class ThreadPoolManager {
  
  ThreadPoolExecutor m_threadExecutor;
  
  public ThreadPoolManager() {
//  ThreadPoolExecutor(int corePoolSize, 
//  int maximumPoolSize, 
//  long keepAliveTime, 
//  TimeUnit unit, 
//  BlockingQueue<Runnable> workQueue) 
//  You can also choose to provide your own ThreadFactory.
//  If ThreadFactory is not specified as in this case, 
//  a defaultThreadFactory will be created.
    this.m_threadExecutor = new ThreadPoolExecutor(2, 5, 5000, TimeUnit.MILLISECONDS, new ArrayBlockingQueue<Runnable>(100));
  }
  
    /**
   * Submit a Task to the ThreadPool for execution.
   * @param task Runnable object that the ThreadPool will execute.
   */
  public void submitTaskForExecution(Runnable task) {
    m_threadExecutor.execute(task);
  }
  
    /**
   * Shut down the thread pool.
   * @return true if all the threads terminated at the time of return,
   *         else false.
   */
  public boolean shutdown() throws InterruptedException {
    //Alternative is shutdown(), which will wait for
    //all executing tasks to complete while stopping new tasks.
    m_threadExecutor.shutdownNow();
    //Put the current thread to sleep for a few seconds
    //to allow for all executing threads to complete.
    m_threadExecutor.awaitTermination(10000, TimeUnit.MILLISECONDS);
    return m_threadExecutor.isTerminated();
  }
}

If you are using a pre-5.0 JDK, you can use the EDU.oswego.cs.dl.util.concurrent.PooledExecutor class from the open source concurrent processing utilities library concurrent.jar from Doug Lea (the predecessor to the java.util.concurrent package).

Note that a thread cannot be stopped or destroyed on demand. You may only interrupt the thread and write the associated Runnable object's run() method or the subclassed Thread object's run() method in such a way that frequent checks verify whether an interrupt has been set and that any InterruptedExceptions are not ignored. This is true no matter which approach you use.

Some notes that will be helpful in writing your run() method logic:

  • Never consume the InterruptedException in your run() method. If a shutdown has been requested, always propagate it.
  • If you call interrupt() on a thread, it will run into an InterruptedException only if it is in a wait or sleep state. If the thread is active, it will not receive an exception; nevertheless, the interrupt flag will be set on the thread, which can be checked using Thread.isInterrupted().
  • If you have threads that process data for long periods of time or threads that are in an eternal loop, you should perform frequent checks to see if the interrupt flag has been set.
  • Not all interrupts mean shutdown; sometimes threads can receive interrupt signals due to other reasons. Therefore, you must check whether the interrupt was the result of a shutdown before you make the thread exit the run() method. Perform this check by looking at the application state.
  • Further, if a thread has not yet started, but has just been constructed, calling an interrupt on it will not set the interrupted flag. You can work around this case by checking for the application state up-front in your run() method.

The following code listing shows the implementation of a simple Runnable object that performs database queries and handles interrupts and exceptions properly to ensure a smooth shutdown:

Related:
1 2 Page 1