Writing good unit tests, Part 2: Follow your nose

Assertions, exceptions, and refactoring in test code

1 2 3 4 5 Page 4
Page 4 of 5

Exceptions and failures in callback methods

In general, a thread can only catch exceptions that are thrown inside its run() method. Therefore, if you're running JUnit or TestNG in one thread and creating assertion failures (or exceptions) in another, the test framework will not able to catch them. For your listener tests, that means that you will need some kind of synchronization and communication between the listener thread and your main test thread, because the callback method is executed asynchronously on a different thread.

This article's source code includes the class javaworld.junit4.example.ListenerExampleTest_JUnit4, which gives you ideas about how to deal with assertions and exceptions that occur in callback methods. The code uses JUnit 4 as the test framework, but you can use the same mechanisms when running JUnit 3.8.x or TestNG. The simulated listener problems (one assertion failure and one exception, occurring in the update() method of the callback listener) should be reported as if they had occurred in the JUnit main thread. As a result, you will get red bars in your IDE (or a corresponding problem message reported in your daily build JUnit-HTML file), together with the corresponding stack trace (!). Note that both the failure and the division-by-zero exception will be mapped onto a JUnit AssertionFailedError, as Figures 1 and 2 illustrate.

JUnit failure stack trace for callback problem
Figure 1. JUnit failure stack trace for callback problem (click to enlarge)
JUnit exception stack trace for callback problem
Figure 2. JUnit exception stack trace for callback problem (click to enlarge)

To get this running, I used a CoundDownLatch from java.util.concurrent for synchronization, plus some other features available since Java 5; thus, you will need JDK 1.5+ to play with the sample code. At the heart of the code example are some Thread- and StackTrace-related configurations, shown in Listing 9.

Listing 9. Catching failures and exceptions in callback routine testing

/*
 * Because callback routines are executed in a thread different from the
 * JUnit main thread we will catch all exceptions and errors that could be
 * thrown in such threads. To save problem related information for later
 * analysis we cache the problem message and stacktrace. A 'problem' can
 * either be an AssertionError thrown by JUnit or an unexpected exception
 * thrown by the listener.
 */
private void prepareForAsynchronousFailureHandling(final Thread junitMainThread) {
  Thread.setDefaultUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() {
    @Override
    public void uncaughtException(Thread t, Throwable e) {
      if (!t.equals(junitMainThread)) {
        synchronized (lock) {
          problemMessage = e.getMessage();
          problemStackTrace = e.getStackTrace();
        }
      }
    }
  });
}

/**
 * Analyze cached stacktrace to find out if we are processing a 'real'
 * listener problem.
 * 
 * @param clazz the listener class to be checked for failures/errors
 * @param methodName the listener method name where we expect problems
 */
private void reportPossibleProblemsInListenerWithMethod(final Class<?> clazz, final String methodName) {
  synchronized (lock) {
    if (problemMessage != null) {
      if (isProblemInClassMethod(clazz, methodName, problemStackTrace)) {
        final AssertionFailedError ae = new AssertionFailedError(problemMessage);
        ae.setStackTrace(problemStackTrace); // save stacktrace for reporting by JUnit main thread
        throw ae;
      }
    }
  }
}
1 2 3 4 5 Page 4
Page 4 of 5