I have a very large web application, running on three tomcats, which has been running for many years.
There was a change made to the web application overnight, to add the following jar files to the webapp/WEB-INF/lib directory:
avalon-framework-cvs-20020806.jar batik.jar crimson_1_1_3.jar icu4j_2_6.jar jacob.jar jaxen-full.jar saxon_6_5_3.jar saxpath.jar
Today the web application started exhibiting behaviour I have never seen before. Some classes, in unrelated areas of the application are getting lost by Tomcat. It seems to be classes that are not used often, so my theory is that the class (a servlet, or another class used by the servlet or JSP) is loaded, and used, then a period of time goes by during which it is not used. The next time an attempt to use the same class then Tomcat gives this type of response:
Error: javax.servlet.ServletException: Wrapper cannot find servlet class XYZ or a class it depends on at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1076) at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:791) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:127) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151) at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200) at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283) at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773) at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703) at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:595)
Root Stack Trace: java.lang.ClassNotFoundException: XYZ at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1359) at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1205) at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1068) at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:791) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:127) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151) at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200) at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283) at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773) at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703) at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:595)
-------------
Then the next time the page is requested the response is a 404 error.
The three load-balanced tomcat servers all have the same setup, and by getting one of the other servers you would sometimes get the page that was reported as missing.
It seems that Tomcat is unloading the classes, but having read up about unloading classes it doesn't seem possible unless the class loader that loaded the class is also unloaded. The reloadble option in server.xml is NOT set to true.
Any ideas?
|