Writing good unit tests, Part 2: Follow your nose

Assertions, exceptions, and refactoring in test code

Klaus Berg continues his in-depth investigation of tools and best practices for programming with GUTs. Get tips for writing cleaner and more efficient assertions with the help of a matching library (Hamcrest) and handling checked and unchecked exceptions in JUnit 3 and 4 and TestNG. The remainder of the article focuses on refactoring test code, first explaining why it's important and then revealing the three layers of code smell that indicate something's rotten. This article is packed with tips and trivia for the code quality addict. Level: Intermediate

The first half of this article presented best practices for writing test code, both from a technical angle and from a process-oriented one. Code coverage tools can help you measure how well your tests perform with respect to executing the different statements and branches of your system under test. In this article I continue the discussion with a look at two foundations of quality test code: writing proper assertions (e.g., with Hamcrest), and handling checked and unchecked exceptions (in JUnit 3.8.x, JUnit 4, and TestNG). I'll also present an example class that should give you ideas for dealing with assertions and exceptions in callback methods.

The second half of the article will focus on refactoring test code, from answering the question "Why refactor test code?" to outlining the various code smells you're likely to encounter in your unit tests. I'll also discuss tools that are specifically helpful for detecting code smells (most of them introduced in the first half of this article) and will conclude with some code quality standards for knowing when your test code is "good enough."

About the sample code

The source code package that accompanies this article contains an Eclipse project with source code and corresponding test code that demonstrates some of the points to be discussed. If you download the code, you'll find the structure below the root directory Eclipse_Example as follows:

  • The src directory contains example source code and a JUnit3 extension called AssertExceptions that can be used to test exceptions in an object-oriented fashion with JUnit 3.8.x.
  • The test directory contains the corresponding test code written with JUnit 4 and JUnit 3. Some test code makes use of the Hamcrest assertion library and contains a customized Hamcrest assertion file (FileMatchers.java).
  • build.xml contains an Ant script to create an Emma test coverage HTML report in the reports directory.
  • ant_junit.xml contains an Ant script to run the unit tests and to create an HTML JUnit 4 report in the junit directory.
  • XmlConfigBuilder.java, located in the src/javaworld/junit4/example/log4j directory, is a Log4j XML config file merger that merges arbitrary combinations of profiles but ignores their appender part. The appender part comes from a master XML file that contains appender definitions only. This file can be created using the following command:
    new XmlConfigBuilder(..).createAppenderFile()
    
    By running XmlConfigBuilderAPITest, you can see different configurations merged into one final config file. If you have different log-level settings, the highest one will survive.
  • The resources/templates directory contains a set of Log4J XML configurations that can be merged. appenders.xml, located in this directory, contains all available appenders.

The code was created and tested on Windows XP SP3 with JDK 6, Ant 1.7, and Eclipse 3.4.

Assertions in JUnit and TestNG

Assertions are a cornerstone of unit testing, enabling developers to test assumptions about code as it's being written. Each assertion contains a boolean expression that you believe will be true when the assertion executes. If it evaluates to false, the system or your test framework will throw an error. So, assertions are used to determine and "report" a test case verdict. As old as they are -- dating back to the early days of JUnit -- assertions aren't perfect, and they're not always written as well as they could be. On the other hand, writing good assertions isn't rocket science. (Well, unless of course you're NASA.) So, in this section, we'll look at simple ways to improve your assertions in JUnit and TestNG, with the help of a smart library called Hamcrest.

In JUnit 3 (the first widely used open source testing framework) assertions are built using the Assert class. Depending on the version of JUnit you use, Assert is either located in the package junit.framework (JUnit 3) or in org.junit (JUnit 4). You don't need to make explicit use of the classname Assert in either version; JUnit 3's TestCase extends Assert, and in JUnit 4 you can use Java 5's static import capabilities, like so:

import static org.junit.Assert.*;

TestNG allows you to write assertions by means of the assert statement that has been available since JDK 1.4. So you can write code like this:

myInstance.doSomething(1,2) : "comment if the test fails"

TestNG also lets you use JUnit's asserts, which are included in the class org.testng.AssertJUnit. Because JUnit 3.x asserts are hard to read (the parameter order is counterintuitive for English-speakers), TestNG introduces another class, Assert, in its org.testng package. The only way this is different from JUnit's class is that the message-string parameter is always the last parameter, and the expected and actual values are reversed, as shown in Listing 1.

Listing 1. Different Assert classes in TestNG

@Test
public void checkAccounts() {
   ...
    // JUnit Assert
   org.testng.AssertJUnit.assertEquals("The two accounts should be the same", expected, actual);

    // TestNG Assert
   org.testng.Assert.assertEquals(actual, expected, "The two accounts should be the same");
}

If you do not provide an explicit error message, the default problem report handling in either JUnit or TestNG is quite limited. You will only get a message telling you that an AssertionError has occurred. How can you do better? The best way is to use a kind of DSL to write proper and easily readable assertions. The Hamcrest library, now hosted by the Google code repository, can assist you with this goal.

Assertion matching with Hamcrest

In May 2005, Joe Walnes posted an article on his blog about using jMock constraints to improve assertion methods in JUnit. A simple JUnit assertion method with a constraint was an elegant replacement for all the other assertion methods typically cluttering test code:

Since using this one assert method, I've found my tests to be much easier to understand because of lack of noise, and I've spent a lot less time creating "yet another assertion" method for specific cases. And in most cases I never need to write a custom failure message, as the failures are self describing.

Walnes named his new assert method assertThat(); its syntax looked something like what you see in Listing 2:

Listing 2. assertThat() with flexible conditions

assertThat(x, is(3));
assertThat(x, is(not(4)));
assertThat(responseString, either(containsString("color")).or(containsString("colour")));
assertThat(myList, hasItem("3"));

The Hamcrest library, born of assertThat(), is a collection of matcher objects (also known as constraints or predicates) that allow match rules to be defined declaratively, and to be used in other frameworks.

Origins of Hamcrest

Hamcrest was originally spawned from the Constraints code in jMock1, which itself came from the Predicates code in DynaMock. The latest version of jMock (jMock2) uses Hamcrest directly.

You can use Hamcrest in testing frameworks, mocking libraries, or UI validation rules. But note that Hamcrest is not a testing library; it just happens that matchers are very useful for testing. You can include Hamcrest in any testing framework, but JUnit developers in particular have realized its potential, and it has been part of JUnit since version 4.4. Using Hamcrest matchers makes assert statements more readable and natural, and will also result in more informative, concrete default error messages (that is, the ones that you get without an explicit message parameter).

According to the Hamcrest tutorial, the most important Hamcrest matchers are as follows:

  • Core:
    • anything(): Always matches, useful if you don't care what the object under test is
    • describedAs(): Decorator to add custom failure description
    • is(): Decorator to improve readability
  • Logical:
    • allOf(): Matches if all matchers match, then short circuits (like Java &&)
    • anyOf(): Matches if any matchers match, then short circuits (like Java ||)
    • not(): Matches if the wrapped matcher doesn't match, and vice versa
  • Object:
    • equalTo(): Tests object equality using Object.equals()
    • hasToString(): Tests Object.toString()
    • instanceOf(), isCompatibleType(): Test type
    • notNullValue(), nullValue(): Test for null
    • sameInstance(): Tests object identity
  • Beans:
    • hasProperty(): Tests JavaBeans properties
  • Collections:
    • array(): Tests an array's elements against an array of matchers
    • hasEntry(), hasKey(), hasValue(): Test to see if a map contains an entry, key, or value
    • hasItem(), hasItems(): Test to see if a collection contains elements
    • hasItemInArray(): Tests to see if an array contains an element
  • Number:
    • closeTo(): Tests to see if floating point values are close to a given value
    • greaterThan(), greaterThanOrEqualTo(), lessThan(), lessThanOrEqualTo(): Test ordering
  • Text:
    • equalToIgnoringCase(): Tests string equality, ignoring case
    • equalToIgnoringWhiteSpace(): Tests string equality, ignoring differences in runs of whitespace
    • containsString(), endsWith(), startsWith(): Test string matching

Hamcrest also provides operators like is(), which act as syntactic sugar. This operator allows for elegant solutions like the one you see in Listing 3.

Listing 3. The Hamcrest is() operator adds some syntactic sugar

assertThat(color, equalTo("red"));
// or even better:
assertThat(color, is(equalTo("red")));
// optimal:
assertThat(color, is("red")); //allowed since 'is(T value)' is overloaded to return 'is(equalTo(value))'.

On its own, assertThat() has a powerful, de-cluttering effect on test code. You can also extend the library and even write custom matchers, such as the one I created for file-related checks. My custom matcher is shown in part in Listing 4, and you can find the complete code in this article's source code.

Listing 4. Extending Hamcrest for file-related checks

assertThat(generatedFile, exists());
assertThat(generatedFile, is(readable()));
assertThat(generatedFile, is(sized(equalTo(LOG4J_APPENDER_FILE_LENGTH))));

Exception testing -- the tester's notebook

When writing unit tests for a method (or a constructor) it's important that you cover all boundary cases and possible inputs. Some inputs will cause the method to throw an exception, and you'll want to verify that output. There are two types of exceptions: checked and unchecked. Both are of interest when it comes to properly unit testing a method.

Checked exceptions are for expected errors and should be dealt with programmatically. Unchecked exceptions are mainly for programming errors, and are derived from the JDK RuntimeException class. Such exceptions should be allowed to brachiate all the way up the class hierarchy and terminate the test, because they usually indicate situations from which the program is not meant to recover. However, some RuntimeException subclasses, like IllegalArgumentException or NullPointerException, may indicate errors in method parameters handed over to the callee. Of course, if such exceptions are part of the API contract, they have to be checked with a unit test.

In the next sections I'll highlight some of the idiosyncrasies of exception testing with JUnit 3.8.x, JUnit 4, and TestNG, and also offer tips for resolving exceptions and failures in callback methods, based on JUnit 4.

Exception testing with JUnit 3.8.x

If you are still using good old JUnit 3.8.x, or even previous versions, you might notice that the framework acknowledges the difference between test failures and errors. Failures are the result of failed programmed assertions. If, however, your tested objects throw an exception that is ultimately caught by the JUnit framework itself, it is reported as a test error. (JUnit 4.x has dropped the distinction between test failures and test errors -- a step backward, in my opinion.) A test error means that the test did not run to successful completion -- that is, the set of assertions that the test tried to make was not even executed. Listing 5 illustrates a common coding pattern for dealing with expected exceptions when testing with JUnit 3.8.x.

Listing 5. Exception testing pattern for use with JUnit 3.8.x

public void testNegativeAmount() {
   try {
      final double NEGATIVE_AMOUNT = -2.00;
      new Euro(NEGATIVE_AMOUNT);
      fail("Should have raised an IllegalArgumentException");
   } catch (IllegalArgumentException expected) {
      ; // expected (or sometimes even better: assertNotNull(expected.getMessage());)
   }
}

But you can do even better by taking an object-oriented approach, demonstrated in Listing 6. From a conceptual point of view, this is along the lines of Groovy's closures.

Listing 6. Exception testing with JUnit 3.8.x, the object-oriented way

import javaworld.junit.util.AssertExceptions;
...
/**
 * Here we use a custom assert statement to check if an expected exception is thrown and if 
 * the expected exception has a proper toString() format.
 * Throws an 'AssertionFailedError' if the exception is not thrown or if the
 * exception's toString() format is wrong. </p>
 * Use this assert method if you want your JUnit test to proceed if the expected exception is thrown. 
 * This method is more "object oriented" than writing again and again try/catch blocks in your test cases.
 *
 * Furthermore, by using this kind of assert you clearly signal your test case intention to the reader.
 */
public void testNegativeAmount() {
   final Block actionBlock = new AssertExceptions.Block() {
      public void execute() throws Exception {
         final double NEGATIVE_AMOUNT = -2.00;
         new Euro(NEGATIVE_AMOUNT);
      }
   };
   assertThrows(IllegalArgumentException.class, actionBlock);
}

Listing 6 contains a custom subclass of JUnit's TestCase, called AssertExceptions, that provides assert statements specific to exception handling. Using the assertThrows() methods contained therein, you can easily have more than one exception-checking assertion in your test without writing try/catch scenarios again and again. (The AssertExceptions class is provided in the source code package for this article.)

Exception testing with JUnit 4

JUnit 4, like TestNG, is based on annotations. The @Test annotation has an optional parameter, expected, that takes as values subclasses of Throwable. If you want to verify that your code under test throws the correct exception, you would use a pattern like the one shown in Listing 7.

Listing 7. Exception testing pattern used by JUnit 4

@Test(expected=IllegalArgumentException.class) public void checkForNegativeAmountException() {
   final double NEGATIVE_AMOUNT = -2.00;
   new Euro(NEGATIVE_AMOUNT);
}

Exception testing with TestNG

TestNG allows you to check for the occurrence of an exception in a way that's quite similar to the technique you'd use with JUnit 4. (Perhaps it's more fair to say it the other way around: if you look at both testing frameworks' evolution, TestNG was the trendsetter, so JUnit 4 may have borrowed from TestNG's way of doing things.) With TestNG, you use the @ExpectedExceptions annotation, as illustrated in Listing 8. This annotation specifies that the raising of an IllegalArgumentException is tolerated by the framework and therefore should not be considered a failure. In other words, your test will pass only if that particular type of exception is raised inside the test method.

Listing 8. Exception testing pattern used by TestNG

@ExpectedExceptions(IllegalArgumentException.class)
public void checkForNegativeAmountException() {
   final double NEGATIVE_AMOUNT = -2.00;
   new Euro(NEGATIVE_AMOUNT);
}

Note that @ExpectedExceptions takes a list of exceptions that a test method is expected to throw. If an exception is thrown that's not on this list, or no exception is thrown at all, this test will be marked a failure. This is different from JUnit 4, where you have exactly one exception class.

Exceptions and failures in callback methods

In general, a thread can only catch exceptions that are thrown inside its run() method. Therefore, if you're running JUnit or TestNG in one thread and creating assertion failures (or exceptions) in another, the test framework will not able to catch them. For your listener tests, that means that you will need some kind of synchronization and communication between the listener thread and your main test thread, because the callback method is executed asynchronously on a different thread.

This article's source code includes the class javaworld.junit4.example.ListenerExampleTest_JUnit4, which gives you ideas about how to deal with assertions and exceptions that occur in callback methods. The code uses JUnit 4 as the test framework, but you can use the same mechanisms when running JUnit 3.8.x or TestNG. The simulated listener problems (one assertion failure and one exception, occurring in the update() method of the callback listener) should be reported as if they had occurred in the JUnit main thread. As a result, you will get red bars in your IDE (or a corresponding problem message reported in your daily build JUnit-HTML file), together with the corresponding stack trace (!). Note that both the failure and the division-by-zero exception will be mapped onto a JUnit AssertionFailedError, as Figures 1 and 2 illustrate.

Figure 1. JUnit failure stack trace for callback problem (click to enlarge)
Figure 2. JUnit exception stack trace for callback problem (click to enlarge)

To get this running, I used a CoundDownLatch from java.util.concurrent for synchronization, plus some other features available since Java 5; thus, you will need JDK 1.5+ to play with the sample code. At the heart of the code example are some Thread- and StackTrace-related configurations, shown in Listing 9.

Listing 9. Catching failures and exceptions in callback routine testing

/*
 * Because callback routines are executed in a thread different from the
 * JUnit main thread we will catch all exceptions and errors that could be
 * thrown in such threads. To save problem related information for later
 * analysis we cache the problem message and stacktrace. A 'problem' can
 * either be an AssertionError thrown by JUnit or an unexpected exception
 * thrown by the listener.
 */
private void prepareForAsynchronousFailureHandling(final Thread junitMainThread) {
  Thread.setDefaultUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() {
    @Override
    public void uncaughtException(Thread t, Throwable e) {
      if (!t.equals(junitMainThread)) {
        synchronized (lock) {
          problemMessage = e.getMessage();
          problemStackTrace = e.getStackTrace();
        }
      }
    }
  });
}

/**
 * Analyze cached stacktrace to find out if we are processing a 'real'
 * listener problem.
 * 
 * @param clazz the listener class to be checked for failures/errors
 * @param methodName the listener method name where we expect problems
 */
private void reportPossibleProblemsInListenerWithMethod(final Class<?> clazz, final String methodName) {
  synchronized (lock) {
    if (problemMessage != null) {
      if (isProblemInClassMethod(clazz, methodName, problemStackTrace)) {
        final AssertionFailedError ae = new AssertionFailedError(problemMessage);
        ae.setStackTrace(problemStackTrace); // save stacktrace for reporting by JUnit main thread
        throw ae;
      }
    }
  }
}

Refactoring: It starts with broken windows

Test code has to be written in the same manner as real production code. That means that you should include a refactoring step in your test implementation process. Your unit tests should evolve via the following steps:

  1. Use case analysis to understand the requirements.
  2. Test design as a pre-step to implement the expected unit behavior testing.
  3. Test code implementation using the test framework of your choice.
  4. Test automation with support from tools like Ant and Maven 2.
  5. Refactoring your test code to improve the quality of both the tests and your code.

The question that arises for many developers is, when is it better to refactor, versus leaving well enough alone? Well, you may have heard the phrase "Broken window, broken house"; it formed the basis for Rudolph Giuliani's policing strategy when he was mayor of New York in the 1990s. If there is one tip Giuliani can share with Java developers and testers, it's the power and simplicity of the broken windows theory. Every time you find a "broken window" in your code -- like massive nested if-else statements, code duplications across classes, or dead code in your tests -- your warning bells should start ringing. Why? They're all signs of software entropy -- a condition to avoid.

Refactoring Java code

Refactoring is basically the object-oriented variant of restructuring. As described in a doctoral thesis by W. F. Opdyke, refactoring is "the process of changing a[n object-oriented] software system in such a way that it does not alter the external behavior of the code, yet improves its internal structure." The key idea here is to properly rename, shorten, or redistribute classes, variables, and methods across the class hierarchy to facilitate future adaptations and extensions. In the context of software maintenance and evolution, the goal of refactoring is to improve the quality of software.

When to refactor

Refactoring is a powerful technique, but it needs to be done carefully. When done improperly, some quality metrics may become even worse. The main danger of refactoring is that it can inadvertently introduce errors, especially when it's is done by hand. Refactoring test code is also different from refactoring production code. You have no unit tests as a safety net to check your test code, so you have to be a more careful. Refactoring test code is a matter of making small steps (perhaps with peer reviews) and thinking twice before making larger ones.

Static code analyzers, which you learned about in the first half of this article, are among the tools that can help you find and judge possible problems awaiting refactoring. They can detect and measure weak spots by indicating problems using simple length metrics (long methods, long classes, long parameter lists), reporting violations of naming conventions, or indicating needed design modifications.

In addition to testing tools, the most powerful tool for deciding whether your test code needs to be refactored is your nose, which can be trained to quickly detect what has become known as code smell.

Code smell

Code smell was first described by Martin Fowler in his famous book, Refactoring: Improving the Design of Existing Code. Code smells listed in Fowler's book include:

  • Comments. There's a fine line between comments that illuminate and comments that obscure.
  • Uncommunicative or inconsistent names.
  • Long methods.
  • Long parameter lists. If you use a test framework that allows for parameter-driven test methods, then you should watch out for long parameter lists.
  • Duplicated code.
  • Conditional complexity.
  • Large classes.
  • Data clumps.
  • Dead code, including unused imports, variables, and constants.
  • Speculative generality.

Testers in any agile process should recognize these smells as they maintain test code. Code smells typically affect the maintenance cost of tests, but they may also be early warnings signs of behavior smells to follow.

You should look for code smells as you write your test classes using the code inspection capabilities of your IDE. Code inspectors are tools designed to assist in maintaining and cleaning up your Java code. They accomplish this by searching for dead code, bugs, or inconsistencies, and then suggesting solutions for any problem found. Eclipse, NetBeans, and IntelliJ's IDEA, for example, have built-in support for code inspection. Mark code about which you are uncertain with a "TODO" label, and look for such markers later on with your code inspector to clean up ugly code blocks.

Learning from a wise grandma

As co-author of Refactoring: Improving the Design of Existing Code (see Resources), Kent Beck apparently took inspiration from his grandmother's child-rearing motto: "If it stinks, change it!" Likewise, you will know when to change your test code, assuming you are familiar with test code smells.

When your test code package is compiled and cleaned up this way, you can perform further analysis using FindBugs, an open source tool that seeks out bug patterns. It's quite smart and can even discover synchronization problems in multithreaded programs. That's especially useful when you're writing multithreaded unit tests to check concurrency of SUTs.

If you do not want to invest much money in static analyzers, you can continue with PMD and Checkstyle. These tools are open source and easy to customize to satisfy your needs, and they detect a wide range of possible problems.

Test code smells

Once you've sniffed out and fixed the typical code smells in Fowler's book, you'll want to turn your attention to test code-specific smells. In xUnit Test Patterns, Gerard Meszaros describes the following test code smells:

  • Hard-coded test data. Does your test include lots of "magic numbers" or Strings used when creating objects? Such code is likely to result in an unrepeatable test.

  • Test code duplication. Do identical or near-identical code sequences appear many times in many tests? This makes for more code to modify when something changes, which results in fragile tests.

  • Mystery guest. When a test uses external resources, such as a file containing test data, it becomes hard to tell what the test is really verifying. In such cases, either the setup or the verification of the outcome is often external to the test.

  • Complex test code. Does your test include too much test code or conditional test logic? That makes it hard to verify correctness, and means that your tests are more likely to have bugs.

  • Can't see the forest for the trees. Is there so much test code that it obscures what the test is verifying? Such tests do not act as a specification, because they take too long to understand.

  • Conditional test logic. Do your tests contain conditional logic (if statements or loops)? If so, how do you verify that the conditional logic is correct? Does it always test the same thing? Do you have untested test code?

  • Complex undo logic. Do your tests include complex fixture teardown code? Such code is more likely to leave the test environment corrupted by not cleaning up after itself properly; it results in data leaks that may later cause this or other tests to fail for no apparent reason.

There are also a number of behavior smells -- that is, smells that jump out at you while you are running your tests:

  • Fragile tests. Do your tests fail, or fail to compile, or fail every time you change the SUT? With such tests, you need to modify much of them to get things "green" again. This greatly increases the cost of maintaining the system. Contributing code smells include test code duplication and hard-coded test data.

  • Fragile fixture. Do your tests start failing when a shared fixture is modified -- when new records are put into a database, for instance? This could be a result of tests making assumptions about the contents of that shared fixture. A contributing code smell is the mystery guest.

  • Interdependent tests. When one test fails, a number of other tests may also fail for no apparent reason; this may be because they depend on a previously run test's side effects. Tests that cannot be run alone are hard to maintain.

  • Unrepeatable tests. You may find that your tests cannot be run repeatedly without manual intervention. This is often caused by tests not cleaning up after themselves properly and preventing themselves (or other tests) from running again. The root cause of this problem is typically hard-coded test data.

  • Test run war. Do you encounter seemingly random transient test failures? You may note that this occurs when several people are testing simultaneously; if so, it's probably caused by parallel tests interacting with each other through a shared test fixture. This problem can arise with multithreaded tests.

Unfortunately, there are no specific rules that come along with static code analyzers to identify these smells and propose corresponding refactorings (though you are free to write custom rule checkers if you're feeling ambitious). It's up to you to look at your test code very carefully and to check for possible test smells. As the second step, you can try to carry out refactorings as proposed by Gerard Meszaros or Arie van Deursen and his coauthors to clean up your code. Test refactorings are changes to test code that do not add or remove test cases. Because you have no safety net (remember, you are refactoring the test code itself), the production code can be used as a (simple) test case for the refactoring: If a test for a piece of code succeeds before the test's refactoring, it should also succeed after the refactoring. This obviously also means that you should not modify production code while refactoring test code (just as you shouldn't change tests when refactoring production code).

You can also try to implement an even more defensive test coding strategy, by using test patterns, as Meszaros describes in xUnit Test Patterns, or Jon Thomas and others in Java Testing Patterns. With test patterns, you can implement high-quality test code right from the start. Furthermore, you should design your production code to unit test by following the advice given by  Akshay Sharma. He tells us to follow some general design rules, like "program to an interface, not an implementation"; "favor object composition over inheritance"; "avoid unnecessary singletons"; and "leverage the use of dependency frameworks." All these techniques force you to think about unit testing up front.

When are your tests good enough?

To determine when you can stop writing tests, you should ask the following questions (taken from James Bach's "A framework for good enough testing"):

  • Are your tests covering the aspects of the product you need to cover?
  • Are you using a sufficient variety of test techniques or sources of information about quality to eliminate gaps in your test coverage? In other words, do you write good test cases?
  • What is the likelihood that the product could have important problems you do not know about?
  • What problems that your testing should have found are reported through means other than your test process?

Answering the question "When are my tests good enough?" is a lot like knowing when to stop upgrading your personal stereo equipment: you can always buy better amplifiers, better loudspeakers, or better CD players. The problem is to find the right balance between cost and benefit. One last thought I'll leave you with is: look at test code with the eye of an artist or painter -- see if your tests are beautiful in three ways (as described in "Beautiful Tests" by Alberto Savoia):

  • Beautiful in their simplicity. With a few lines of test code, you should be able to document and verify the target code's basic behavior.

  • Beautiful because they reveal ways to make code more elegant, maintainable, and testable. In other words, tests should help make code more beautiful. The process of writing tests often helps you realize not only logical problems, but also structural and design issues with your implementation.

  • Beautiful in their breadth and depth. Very thorough and exhaustive tests boost the developer's confidence that the code functions as expected -- not only in some basic or handpicked cases, but in all cases.

Implementing a good design and architecture for your code (both in testing and in production) has a lot to do with elegance. Quite often, elegance equates to simplicity. To quote Albert Einstein: "Things should be as simple as possible, but not any simpler." This quote could easily refer to test architecture, design, and code. It's this minimalist view that generally yields the best results, and refactoring is the path to this goal. So good test are also beautiful tests, and beautiful tests are simple, insightful, and widely applicable.

Keep the conversation going

If you agree with James Gosling, who once said "I don't think anybody tests enough of anything," it's obvious that we need more tests of better quality. Of course, not everybody can follow Google's testing on the toilet initiative. Google posted flyers about everything from dependency injection to code coverage in the bathrooms of its various headquarters -- almost 500 stalls worldwide -- to educate employees about testing. The rest of us, however, may have to learn about testing in more standard ways.

Even so, I hope the best practices in this article will help you improve your unit tests. If you come across examples of good or bad tests, I'd like to hear about them. That includes my own test code examples -- if you have suggestions for improvement, I have an open mind for constructive criticism. Learning together, we may someday end up with "GUTs all over the place" (to quote Alistair Cockburn). And in the meantime, we should always code as if the person who ends up maintaining our (test) code will be a violent psychopath who knows where we live ...

Klaus P. Berg has a master's equivalent (Diplom) in electrical engineering and applied informatics from the University of Karlsruhe in Germany. He was an architect and implementor for projects at Siemens focused on Java GUI development with Swing and Java Web Start, and he also acted on the server side, creating Java-based intranet applications. Now he works as a senior engineer in the area of software quality, focusing on functional and performance testing, mainly for JEE software.

Learn more about this topic

More from JavaWorld