Test-driven GUI development with FEST

Try a functional approach to testing Swing GUIs in test-driven environments

Alex Ruiz follows up his popular JavaOne 2007 presentation with this introduction to test-driven GUI development with FEST. Learn what differentiates FEST (formerly TestNG-Abbot) from other GUI testing frameworks and get a hands-on introduction to Swing GUI testing with this fast-growing developer testing library. Video demonstrations and working code examples are included.

Test-driven development (TDD) is an application-development technique wherein automated tests drive the software design and development process. Although TDD involves writing many tests, practitioners emphasize it is a design and development methodology. In TDD, tests are used to specify what the application code should do. When a test fails, developers implement the code necessary for the test to pass. The objective of TDD is clean production code that works as expected.

Developers using the iterative practice of TDD have reported the following benefits:

  • Shorter code implementation time
  • Reduction in code defects
  • Fewer instances of overcomplicated and unnecessary code
  • Reduced likelihood of introducing bugs when fixing bugs, refactoring or introducing new features

In spite of these benefits, developer adoption of TDD for GUI development has been slow, mainly because of the challenge of writing tests for GUIs. In addition, conventional unit testing, such as testing a class in isolation, often is not appropriate for GUI testing: A GUI "unit" can be made up of more than one component, each of them enclosing more than one class. In many cases, functional testing is a more effective way to test GUIs.

In this article I use code examples and video demonstrations to introduce test-driven GUI development with FEST, an open source library I created with Yvonne Wang Price to facilitate functional GUI testing in TDD environments. I start by using the java.awt.Robot class to create functional tests for a sample GUI. I walk through the steps of setting up a test and GUI, then explain the shortcomings of using Robot for functional GUI testing. I then introduce FEST and run the same tests using the FEST library. I conclude by discussing the differences between GUI testing with FEST vs. the Robot class.

Note that this article assumes you are familiar with GUI design and development and functional testing methodology. You need not be familiar with test-driven development to follow the examples.

Functional GUI testing with java.awt.Robot

The following factors are essential to creating a robust functional GUI test:

  1. Being able to simulate user events
  2. Having a reliable mechanism for finding GUI components
  3. Being able to tolerate changes in a component's position and/or layout.

java.awt.Robot is a class introduced in JDK 1.3 that can be used to test GUIs built using Swing and AWT. Robot meets the first of the three above requirements because it can be used to simulate user interaction by controlling the application environment's mouse and keyboard.

To demonstrate functional GUI testing with the Robot class, I first will create a simple Swing GUI that contains a JLabel and a JButton. The GUI's expected behavior is that when the user presses the JButton, the text of the JLabel will change from "Hi" to "Bye!" As good TDD practitioners do, I will start my development effort with a failing test.

 

Listing 1. A functional GUI test using java.awt.Robot

// Omitted imports and package declaration.

/**
 * Tests for {@link Naive}.
 *
 * @author Alex Ruiz
 */
public class NaiveTest {
  private Naive naive;

  @BeforeMethod public void setUp() {
    naive = new Naive();
    naive.setVisible(true);
  }

  @Test public void shouldChangeTextInTextFieldWhenClickingButton() throws AWTException {
    JButton button = naive.getButton();
    click(button);
    JTextField textField = naive.getTextField();
    assertEquals(textField.getText(), "Bye!");
  }

  private void click(JButton button) throws AWTException {
    Point point = button.getLocationOnScreen();
    Robot robot = new Robot();
    robot.setAutoWaitForIdle(true);
    robot.mouseMove(point.x + 10, point.y + 10);
    robot.mousePress(MouseEvent.BUTTON1_MASK);
    robot.mouseRelease(MouseEvent.BUTTON1_MASK);
  }
}

My next step is to create the GUI and eliminate the compilation errors in Listing 1. Figure 1 is a screenshot of my development setup for testing and building the GUI. Click the screenshot for a video demonstration of the steps taken to eliminate compilation errors from my code in preparing for running the test.

Creating the GUI to test.
Figure 1. Creating the GUI to test (click to see the demo)

 

Listing 2 shows the source code for my simple Swing GUI.

Listing 2. A simple Swing GUI

// Omitted imports and package declaration.

/**
 * Understands a simple GUI.
 *
 * @author Alex Ruiz
 */
public class Naive extends javax.swing.JFrame {

  // Omitted code generated by GUI builder.

  private javax.swing.JButton jButton1;
  private javax.swing.JLabel jLabel1;

  public JButton getButton() {
    return jButton1;
  }

  public JLabel getLabel() {
    return jLabel1;
  }

}

When I run the following test, I expect it to fail.

Running a failing test.
Figure 2. Running a failing test (click to see the demo)

As you can see from the video demonstration, the test launches the GUI and uses the Robot class to simulate a user moving a mouse pointer over the JButton and pressing the left mouse button. The JLabel does not change to the expected text, "Bye!", so the test fails. The test has given me a good idea of what's wrong with my code, so I just return to my IDE and write the necessary code to make the test pass, as shown in Figure 3.

Writing code to make the test pass.
Figure 3. Writing code to make the test pass (click to see the demo)

Listing 3 shows the revised code. Note that all I had to do was change the text of the JLabel to show the expected text, "Bye!", when the user clicks the JButton.

 

Listing 3. Revised code to make the test pass.

jButton1.addMouseListener(new java.awt.event.MouseAdapter() {
    public void mouseClicked(java.awt.event.MouseEvent evt) {
      jButton1MouseClicked(evt);
    }
  });

  private void jButton1MouseClicked(java.awt.event.MouseEvent evt) {
    jLabel1.setText("Bye!");
  }

Shortcomings of functional GUI testing with java.awt.Robot

Although I was able to simulate user events with the Robot class, I could not meet the other two requirements of functional GUI testing. Looking back to Listing 1, you may notice the following shortcomings of the test:

  1. I had to look up GUI components myself because Robot does not provide a way to find them. To get a reference to a GUI component using Robot,I would have to use accessor methods ("getters") or loop through the component hierarchy. Using getters for testing adds clutter to my GUI code. Likewise, looping through the component hierarchy can be tedious and may introduce code duplication.
  2. Tests written using the Robot class are fragile: Any change in layout will break them.

The second issue poses a serious problem. When testing my example Swing GUI, I assumed the button was bigger than 10 pixels. If for some reason the button had been any smaller than the expected size, the test would have failed, as shown in Figure 4.

Changes in layout break the Robot test.
Figure 4. Changes in layout break the Robot test (click to see the demo)

Tests designed to test the behavior of a GUI should not be affected by changes in the GUI layout. java.awt.robot also is a fairly low-level class and requires too much code to simulate user events. In the next section I'll introduce an alternative to functional GUI testing with the Robot class.

1 2 3 Page
Join the discussion
Be the first to comment on this article. Our Commenting Policies