Test-first development with FitNesse

Learn how FitNesse can solve your quality problems

During the last few years, I have worked in all roles of the testing process, using server-side JavaScript, Perl, PHP, Struts, Swing, and model-driven architectures. All projects differed, but they all had some commonalities: the deadlines ran late, and the projects had difficulties accomplishing what the customer really needed.

Every project had some kind of requirement, some were very detailed, some, only a few pages in length. Those requirements usually underwent three phases:

  • They were written (either by the customer or by the contractor) and received some kind of official acceptance
  • The testers tried to work with the requirements and found them more or less inadequate
  • The project entered a phase of acceptance testing, and the customer suddenly remembered all kinds of things the software needed to do additionally/differently

The last phase led to changes, which led to missed deadlines, which put stress on the developers, which in turn led to more mistakes. The bug count started to rise fast, and the overall quality of the system declined. Sound familiar?

Let's consider what went wrong in the projects described above: The customer, developer, and tester did not work together; they passed the requirements on, but each role had different needs. In addition, the developers usually developed some kind of automated tests, and the testers tried to automate some tests as well. Usually, they couldn't coordinate the testing sufficiently, and many items were tested twice, while others (usually the hard parts), not at all. And the customer saw no tests. This article describes a way to solve these problems by combining requirements with automated tests.

Enter FitNesse

FitNesse is a wiki with some additional functions for triggering JUnit tests. If these tests are combined with requirements, they serve as concrete examples, thereby making the requirements even more clear. Furthermore, the test data is logically organized. The most important thing about using FitNesse, however, is the idea behind it, meaning that the requirements turn out to be written (partly) as tests, making them testable and, therefore, their fulfillment verifiable.

Using FitNesse, the development process could look like this: The requirements engineer writes the requirements in FitNesse (instead of Word). He tries to get the customer involved as much as possible, but that usually cannot be achievable on a daily basis. The tester peeks at the document repeatedly and asks difficult questions from day one. Because the tester thinks differently, he does not think, "What will the software do?" but "What might go wrong? How can I break it?" The developer thinks more like the requirements engineer; he wants to know, "What does the software have to do?"

The tester starts writing his tests early, when the requirements are not even done yet. And he writes them into the requirements. The tests become part of not only the requirements, but also the review/acceptance process for the requirements, which has some important advantages:

  • The customer gets to think about the tests too. Usually, she even gets involved in creating them (you might be surprised how much fun she can have with this).
  • The specification becomes much more detailed and precise, as the tests are usually more precise than mere text.
  • Thinking early about real scenarios, providing test data and calculating examples results in a much clearer view of the software—like a prototype, only with more functions.

Finally, the requirements are handed to the developer. He has an easier job now, as the specification is less likely to change and because of all the included examples. Let's look at how this process makes a developer's job easier.

Implementing test-first

Usually, the hardest part of starting test-first development is that nobody wants to spend so much time writing tests, only then to find a way to make them work. Using the process described above, the developer receives the functional tests as part of his contract. His tasks change from "Build the thing I want and you are done, until I examine your work and make changes" to "Make these tests work and you are done." Now everybody has a better idea of what to do, when the work is to be completed, and where the project stands.

Not all those tests will be automated and not all will be unit tests. We usually divide tests into the following categories (details will follow):

  • Data-driven tests that need to be implemented as unit tests. Calculations are the typical example.
  • Keyword-driven tests that automate application usage. These are system tests and require the application to be running. Buttons are clicked, data is entered, and resulting pages/screens are checked to contain certain values. The test team usually implements these tests, but some developers benefit from them as well.
  • Manual tests. These tests are either too expensive to automate and the possible errors not severe enough, or they are so fundamental (start page not displayed) that their breakage would be discovered at once.

When I first read about FitNesse in 2004, I laughed and said that it would never work. The idea of writing my tests into a wiki that turned them automatically into tests seemed too absurd. Turned out, I was wrong. FitNesse really is as simple as it appears to be.

This simplicity starts with the installation. Just download the full distribution of FitNesse and unzip it. In the following discussion, I assume you have unzipped the distribution to C:\fitnesse.

Start FitNesse by running the run.bat (run.sh on Linux) script in C:\fitnesse. By default, FitNesse runs a Web server on port 80, but you can specify a different port, say 81, by adding -p 81 to the first line in the batch file. That's all there is to it; you can now access FitNesse at http://localhost:81.

In this article, I use the Java version of FitNesse on Windows. However, the examples can be used for other versions (Python, .Net) and platforms as well.

Some tests

FitNesse's online documentation provides some simple examples (comparable to the infamous money example from JUnit) to get you started. They are fine for learning how to use FitNesse, but they are not sufficiently complicated to persuade some skeptics. Therefore, I will use a real-world example from one of my recent projects. I've significantly simplified the problem, and the code, not taken directly from the project, was written for illustrative purposes. Still, this example should be sufficiently complicated to demonstrate the power of FitNesse's simplicity.

Let's assume we're working on a project that implements a complex enterprise Java-based software for a large insurance company. The product will handle the company's whole business, including customer and contract management and payments. For our example, we will look at a tiny part of this application.

In Switzerland, parents are entitled to one child allowance per child. They only receive this allowance if certain circumstances are met, and the amount varies. The following is a simplified version of this requirement. We will start with "traditional" requirements and move them to FitNesse afterwards.

Several phases of child allowance exist. The claim starts on the first day of the month the child is born and ends on the last day of the month the child reaches the age border, finishes his or her apprenticeship, or dies.

On reaching the age of 12, the claim is raised to 190 CHF (Switzerland's official currency symbol) starting on the first day of the month of birth.

Full-time and part-time employment of parents lead to different claims, as detailed in Figure 1.

Figure 1. Entitlements for child allowance. Click on thumbnail to view full-sized image.

The current rate of employment is calculated on the active work contracts. The contract needs to be valid, and if an end date is set, it needs to be situated in the "activate period." Figure 2 shows how much money a parent is entitled to, depending on the age of the child.

Figure 2. Claims based on age. Click on thumbnail to view full-sized image.

The regulations governing these payments are adapted every two years.

On first reading, the specification might sound exact, and a developer should be able to easily implement it. But are we really sure about the boundary conditions? How would we test these requirements?

Boundary conditions
Boundary conditions are the situations directly on, above, and beneath the edges of input and output equivalence classes. Experiences show that test cases exploring boundary conditions have a higher payoff than test cases that do not. A typical example is the infamous "one-off" on loops and arrays.

Scenarios can be a great help in finding exceptions and boundary conditions, as they provide a good way to get domain experts to talk about business.

Scenarios

For most projects, the requirements engineer hands the specification to the developer, who studies the requirements, asks some questions, and starts to design/code/test. Afterwards, the developer hands the software to the test team and, after some rework and fixes, passes it on to the customer (who will likely think of some exceptions requiring changes). Moving the text to FitNesse won't change this process; however, adding examples, scenarios, and tests will.

Scenarios are especially helpful for getting the ball rolling during testing. Some examples follow. Answering the question of how much child allowance is to be paid to each will clarify a lot:

  • Maria is a single parent. She has two sons (Bob, 2, and Peter, 15) and works part-time (20 hours per week) as a secretary.
  • Maria loses her job. Later, she works 10 hours per week as a shop assistant and another 5 hours as a babysitter.
  • Paul and Lara have a daughter (Lisa, 17) who is physically challenged and a son (Frank, 18) who is still in university.

Just talking through these scenarios should help the testing process. Executing them manually on the software will almost certainly find some loose ends. Think we can't do that, since we don't have a prototype yet? Why not?

Keyword-driven testing

Keyword-driven testing can be used to simulate a prototype. FitNesse allows us to define keyword-driven tests (see "Totally Data-Driven Automated Testing" for details). Even with no software to (automatically) execute the tests against, keyword-driven tests will help a lot.

Figure 3. ActionFixture test. Click on thumbnail to view full-sized image.

Figure 3 shows what a keyword-driven test might look like. The first column represents keywords from FitNesse. The second column represents methods in a Java class (we write those, and they need to follow the restrictions put on method names in Java). The third column represents data entered into the method from the second column. The last row demonstrates what a failed test might look like (passed tests are green). As you can see, it is quite easy to find out what went wrong.

Such tests are easy and even fun to create. Testers without programming skills can create them, and the customer can read them (after a short introduction).

Defining tests this way, right next to the requirement, has some important advantages over the traditional definition of test cases, even without automation:

  • The context is at hand. The test case itself can be written with the least possible amount of work and is still precise.
  • If the requirement changes, there is a strong chance that the test will change as well (not very likely when several tools are used).
  • The test can be executed at once to show what needs to be fixed to make this new/changed requirement work.

To automate the test, a thin layer of software is created, which is delegated to the real test code. These tests are especially useful for automating manual GUI tests. I developed a test framework based on HTTPUnit for automating the testing of Webpages.

Here is the code automatically executed by FitNesse:

 

package stephanwiesner.javaworld;

import fit.ColumnFixture;

public class ChildAllowanceFixture extends ColumnFixture { public void personButton() { System.out.println("pressing person button"); } public void securityNumber(int number) { System.out.println("entering securityNumber " + number); } public int childAllowance() { System.out.println("calculating child allowance"); return 190; } [...] }

The output of the tests can be examined in FitNesse as well (see Figure 4), which greatly helps with debugging. In contrast to JUnit, where one is discouraged from writing debug messages, I find them absolutely necessary when working with automated Web tests.

Figure 4. Standard output. Click on thumbnail to view full-sized image.

When testing a Web-based application, error pages are included in the FitNesse page and displayed, making debugging much easier than working with log files.

Data-driven testing

While keyword-driven testing is fine for GUI automation, data-driven testing is the first choice for testing code that does any kind of calculation. If you have written unit tests before, what is the single most annoying thing about those tests? Chances are, you think of data. Your tests will be full of data, which often changes, making maintenance a nightmare. Testing different combinations requires different data, probably making your tests complicated, ugly beasts.

With data-driven testing, the data is separated from the code. It is usually created in some kind of table, stored as a CSV file, and read in by the test code. With FitNesse, we can conveniently store, change, and access this test data.

1 2 Page 1