JUnit best practices

Techniques for building resilient, relocatable, multithreaded JUnit tests

Page 2 of 3
  • HttpServletRequest can be subclassed to allow the test class to specify the header, method, path info, and other data
  • HttpServletResponse can be subclassed to return an output stream that stores the servlets' responses in a string for later checking

A simpler solution is to use HttpUnit to test your servlets. HttpUnit provides a DOM view of a request's results, which makes it relatively simple to compare actual data with expected results.

You can avoid visual inspection in many ways. However, sometimes it is more cost-effective to use visual inspection or a more specialized testing tool. For example, testing a UI's dynamic behavior within JUnit is complicated, but possible. It may be a better idea to purchase one of the many UI record/playback testing tools available, or to perform some visual inspection as part of testing. However, that doesn't mean the general rule -- don't visually inspect -- should be ignored.

Keep tests small and fast

Executing every test for the entire system shouldn't take hours. Indeed, developers will more consistently run tests that execute quickly. Without regularly running the full set of tests, it will be difficult to validate the entire system when changes are made. Errors will start to creep back in, and the benefits of unit testing will be lost. This means stress tests and load tests for single classes or small frameworks of classes shouldn't be run as part of the unit test suite; they should be executed separately.

Use the reflection-driven JUnit API

Allowing TestSuite to populate itself with test cases using reflection reduces maintenance time. Reflection ensures that you don't need to update the suite() implementation whenever a new test is added.

Build a test case for the entire system

It is important to build a test case for the entire system. If one test case exercises the whole system, then developers can test the impact their changes will have on every class in the system. This increases the chance of errors resulting from unanticipated side effects being caught earlier. Without a universal test case, developers tend to test only the class they have modified. Also, running all the tests for the system becomes a painstaking manual process.

If we built a test case for the entire system, it would consist of all the other test cases, already defined. The test case would define the suite() method, which would add all test cases defined in the system to a TestSuite. This test suite would then be returned from the suite() method. If you had many test cases, building such a test suite would be time-consuming. In addition, you would have to update the universal test case when new test cases were added or existing test cases were renamed or deleted. Instead of manually building and maintaining the test suite, build a test case that automatically builds a TestSuite from all of your system's test cases. Here is an outline of the requirements for such a test case:

  • It should not be self-loading; that would cause recursion. As such, we need to mark test cases as not loadable.
  • It should not load classes derived from TestCases that are meant to be subclasses, and not directly executed.
  • It should distinguish between unit tests and other tests, like load or stress tests. That will let different tests run at different times.
  • It should recurse down a directory structure, looking for test cases to add to the test suite.

We can use the Java type system to determine what sort of test a test case represents. We can have test cases extend classes like UnitTest, StressTest, LoadTest, and so on. However, this would make test case classes difficult to reuse between test types, because the test type decision is made near the root of the inheritance hierarchy; it should be made at each leaf instead. As an alternative, we can distinguish tests using a field: public static final String TEST_ALL_TEST_TYPE. Test cases will be loaded if they have this field declared with a value matching a string that the automatic test case has been configured with. To build this, we'll implement three classes:

  • ClassFinder recursively searches a directory tree for classfiles. Each classfile is loaded and the class's full class name is extracted. That class name is added to a list for later loading.
  • TestCaseLoader loads each class in the list found by ClassFinder and determines if it is a test case. If it is, it is added to a list.
  • TestAll is a subclass of TestCase with an implementation of suite() that will load in a set of test cases by TestCaseLoader.

Let's look at each class in turn.

ClassFinder

ClassFinder locates the classes within the system to be tested. It is constructed with the directory that holds the system's classes. ClassFinder then finds all the classes in the directory tree and stores them for later use. The first part of ClassFinder's implementation is below:

public class ClassFinder {
   // The cumulative list of classes found.
   final private Vector classNameList = new Vector ();
   /**
    * Find all classes stored in classfiles in classPathRoot
    * Inner classes are not supported.
    */
   public ClassFinder(final File classPathRoot) throws IOException {
    findAndStoreTestClasses (classPathRoot);
   }
   /**
    * Recursive method that adds all class names related to classfiles it finds in
    * the currentDirectory (and below).
    */
   private void findAndStoreTestClasses (final File currentDirectory) throws IOException {
      String files[] = currentDirectory.list();
      for(int i = 0;i < files.length;i++) {
         File file = new File(currentDirectory, files[i]);
         String fileBase = file.getName ();
         int idx = fileBase.indexOf(".class");
         final int CLASS_EXTENSION_LENGTH = 6;
         if(idx != -1 && (fileBase.length() - idx) == CLASS_EXTENSION_LENGTH) {

In the code above, we iterate over all the files in a directory. If a filename has a ".class" extension, we determine the fully qualified class name of the class stored in the classfile, as seen here:

            JcfClassInputStream inputStream = new JcfClassInputStream(new FileInputStream (file));
            JcfClassFile classFile = new JcfClassFile (inputStream);
            System.out.println ("Processing: " + classFile.getFullName ().replace ('/','.'));
            classNameList.add (classFile.getFullName ().replace ('/','.'));

This code uses the JCF package to load the classfile and determine the name of the class stored within it. The JCF package is a set of utility classes for loading and examining classfiles. (See Resources for more information.) The JCF package allows us to find each class's full class name. We could infer the class name from the directory name, but that doesn't work well for build systems that don't store classes according to this structure. Nor does it work for inner classes.

Lastly, we check to see if the file is actually a directory. (See the code snippet below.) If it is, we recurse into it. This allows us to discover all the classes in a directory tree:

         } else if(file.isDirectory()) {
            findAndStoreTestClasses (file);
         }
      }
   }
/**
 * Return an iterator over the collection of classnames (Strings)
 */
public Iterator getClasses () {
   return classNameList.iterator ();
}
}

TestCaseLoader

TestCaseLoader finds the test cases among the class names from ClassFinder. This code snippet shows the top-level method for adding a class that represents a TestCase to the list of test cases:

public class TestCaseLoader {
   final private Vector classList = new Vector ();
   final private String requiredType;
   /**
    * Adds testCaseClass to the list of classdes
    * if the class is a test case we wish to load. Calls
    * shouldLoadTestCase () to determine that.
    */
   private void addClassIfTestCase (final Class testCaseClass) {
      if (shouldAddTestCase (testCaseClass)) {
         classList.add (testCaseClass);
      }
   }
   /**
    * Determine if we should load this test case. Calls isATestCaseOfTheCorrectType
    * to determine if the test case should be
    * added to the class list.
    */
   private boolean shouldAddTestCase (final Class testCaseClass) {
      return isATestCaseOfTheCorrectType (testCaseClass);
   }

You'll find the meat of the class in the isATestCaseOfTheCorrectType() method, listed below. For each class being considered, it:

  • Determines whether it is derived from TestCase. If not, it is not a test case.
  • Determines whether the field public final static TEST_ALL_TEST_TYPE has a value matching that specified in the member field requiredType.

Here's the code:

   private boolean isATestCaseOfTheCorrectType (final Class testCaseClass) {
      boolean isOfTheCorrectType = false;
      if (TestCase.class.isAssignableFrom(testCaseClass)) {
         try {
            Field testAllIgnoreThisField = testCaseClass.getDeclaredField("TEST_ALL_TEST_TYPE");
            final int EXPECTED_MODIFIERS = Modifier.STATIC | Modifier.PUBLIC | Modifier.FINAL;
            if (((testAllIgnoreThisField.getModifiers() & EXPECTED_MODIFIERS) != EXPECTED_MODIFIERS) ||
               (testAllIgnoreThisField.getType() != String.class)) {
               throw new IllegalArgumentException ("TEST_ALL_TEST_TYPE should be static private final String");
            }
            String testType = (String)testAllIgnoreThisField.get(testCaseClass);
            isOfTheCorrectType = requiredType.equals (testType);
         } catch (NoSuchFieldException e) {
         } catch (IllegalAccessException e) {
            throw new IllegalArgumentException ("The field " + testCaseClass.getName () + ".TEST_ALL_TEST_TYPE is not accessible.");
         }
      }
      return isOfTheCorrectType;
   }

Next, the loadTestCases() method examines each class name. It loads the class (if it can be loaded); if the class is a test case and of the required type, the method adds the class to its list of test cases:

   public void loadTestCases (final Iterator classNamesIterator) {
      while (classNamesIterator.hasNext ()) {
         String className = (String)classNamesIterator.next ();
         try {
            Class candidateClass = Class.forName (className);
            addClassIfTestCase (candidateClass);
         } catch (ClassNotFoundException e) {
            System.err.println ("Cannot load class: " + className);
         }
      }
   }
   /**
   * Construct this instance. Load all the test cases possible that derive
   * from baseClass and cannot be ignored.
   * @param classNamesIterator An iterator over a collection of fully qualified class names
   */
  public TestCaseLoader(final String requiredType) {
      if (requiredType == null) throw new IllegalArgumentException ("requiredType is null");
      this.requiredType = requiredType;
   }
   /**
   * Obtain an iterator over the collection of test case classes loaded by loadTestCases
   */
   public Iterator getClasses () {
      return classList.iterator ();
   }

TestAll

TestCall pulls everything together. It uses the aforementioned classes to build a list of test cases defined in the system. It adds those test cases to a TestSuite and returns the TestSuite as part of its implementation of the suite() method. The result: a test case that automatically extracts every defined test case in the system, ready for execution by JUnit.

public class TestAll extends TestCase {

The addAllTests() method iterates over the classes loaded by the TestCaseLoader and adds them to the test suite:

   private static int addAllTests(final TestSuite suite, final Iterator classIterator)
   throws java.io.IOException {
      int testClassCount = 0;
      while (classIterator.hasNext ()) {
         Class testCaseClass = (Class)classIterator.next ();
         suite.addTest (new TestSuite (testCaseClass));
         System.out.println ("Loaded test case: " + testCaseClass.getName ());
         testClassCount++;
      }
      return testClassCount;
   }

With suite(), the test cases are added to the TestSuite, then returned to JUnit for execution. It obtains, from the system property "class_root", the directory where the classes are stored. It obtains, from the system property "test_type", the type of test cases to load. It uses the ClassFinder to find all the classes, and the TestCaseLoader to load all the appropriate test cases. It then adds these to a new TestSuite:

   public static Test suite()
   throws Throwable {
      try {
         String classRootString = System.getProperty("class_root");
         if (classRootString == null) throw new IllegalArgumentException ("System property class_root must be set.");
         String testType = System.getProperty("test_type");
         if (testType == null) throw new IllegalArgumentException ("System property test_type must be set.");
         File classRoot = new File(classRootString);
         ClassFinder classFinder = new ClassFinder (classRoot);
         TestCaseLoader testCaseLoader = new TestCaseLoader (testType);
         testCaseLoader.loadTestCases (classFinder.getClasses ());
         TestSuite suite = new TestSuite();
         int numberOfTests = addAllTests (suite, testCaseLoader.getClasses ());
         System.out.println("Number of test classes found: " + numberOfTests);
         return suite;
      } catch (Throwable t) {
         // This ensures we have extra information. Otherwise we get a "Could not invoke the suite method." message.
         t.printStackTrace ();
         throw t;
      }
   }
  /**
   * Basic constructor - called by the test runners.
   */
   public TestAll(String s) {
      super(s);
   }
}

To test an entire system using these classes, execute the following command (in a Windows command shell):

| 1 2 3 Page 2
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.