Test Web applications with HttpUnit

The HttpUnit framework allows the implementation of automated test scripts

In a typical enterprise application, many areas require testing. Starting from the simplest components, classes, the developers or specialized test developers need to program unit tests to ensure that the application's smallest units behave correctly. Each component can potentially pass the unit tests alone; however developers need to ensure they work together as expected—as part of a subsystem, and as part of the whole application—hence, integration tests must be performed. In some projects, performance requirements must be fulfilled, so the quality assurance engineers perform load tests to verify and document how the application performs under various conditions. During application development, quality assurance engineers perform automated and manual functional tests to test the application's behavior from the user's viewpoint. When a development project nearly completes a specific milestone, acceptance tests can be performed to verify that the application fulfilled the requirements.

HttpUnit is a framework based on JUnit, which allows the implementation of automated test scripts for Web applications. It is best suited for the implementation of automated functional tests, or acceptance tests. As the name suggests, it can be used for unit testing; however, typical Web layer components like JSP (JavaServer Pages) pages, servlets, and other template components do not lend themselves to unit testing. As for various MVC (Model-View Controller) framework-based components, these are better suited for testing with other testing frameworks. Struts actions can be unit tested with StrutsUnit, and WebWork 2 actions can be unit tested without a Web container, for example.

Test targets

Before we jump into the architecture and implementation details, it's important to clarify exactly what the test scripts will need to prove about the Web application. It is possible to just simulate the behavior of a casual Website visitor by just clicking on interesting links and reading pages in a random order, but the result of these random scripts would not describe the application's completeness and quality.

A typical enterprise Web application (or a complex Website) has several documents describing the requirements of the various users or application maintainers. These may include use-case specifications, nonfunctional requirements specifications, test-case specifications derived from the other artifacts, user interface design documents, mockups, actor profiles, and various additional artifacts. For a simple application, the whole specification could possibly consist of a simple text file with a list of requirements.

From these documents, we must create an organized list of test cases. Each test case describes a scenario that can be accomplished by a Web visitor through a Web browser. A good practice is to aim for similar-sized scenarios—larger scenarios can be broken down to smaller chunks. Many excellent books and articles discuss the creation of test-case specifications. For this article, let's assume you have a set of items you want to test for your Web application, organized into sets of test-case scenarios.

Time to download stuff!

Okay, now we know the boring stuff, let's download some cool toys! First of all, we need an installed Java 2 SDK to compile and execute our tests. Then we need to download the HttpUnit framework—currently at version 1.5.5. The binary package contains all the required third-party libraries. We will also need the Ant build tool to run the tests and generate reports automatically. Any fairly recent version of these tools would probably work; I just prefer to use the latest-and-greatest version of everything.

To write and execute tests, I recommend using an IDE that has an embedded JUnit test runner. I use Eclipse 3.0M7 to develop my test scripts, but IntelliJ has JUnit support as well, as do most recently released IDEs.

HttpUnit: The HTTP client simulator

As we want to test Web applications, ideally, the test tool should behave exactly as the users' Web browsers. Our application (the test target) should not be aware of any difference when serving up pages to a Web browser or the test tool. That's exactly what HttpUnit provides: it simulates a normal browser's GET and POST requests, and provides a nice object model with which to code our tests against.

Figure 1. The HttpUnit classes I frequently use. Click on thumbnail to view full-sized image.

Check out the detailed API guide for the rest of the classes and methods; Figure 1 just gives a brief overview of the classes I use most frequently. A user session (a sequence of interactions with the Web application) is encapsulated with a WebConversation. We construct WebRequests, typically configuring the URL and the parameters, and then we send it down through the WebConversation. The framework then returns a WebResponse, containing the returned page and attributes from the server.

Here's a sample HttpUnit test case from the HttpUnit docs:

     * Verifies that submitting the login form with the name "master" results
     * in a page containing the text "Top Secret"
    public void testGoodLogin() throws Exception {
        WebConversation     conversation = new WebConversation();
        WebRequest  request = new GetMethodWebRequest( 
            "http://www.meterware.com/servlet/TopSecret" );
        WebResponse response = conversation.getResponse( request );
        WebForm loginForm = response.getForms()[0];
        request = loginForm.getRequest();
        request.setParameter( "name", "master" );
        response = conversation.getResponse( request );
        assertTrue( "Login not accepted", 
                   response.getText().indexOf( "You made it!" ) != -1 );
        assertEquals( "Page title", "Top Secret", response.getTitle() );

Architectural considerations

Notice how the Java sample above contains the domain name of the server running the application. During development of a new system, the application lives on multiple servers, and the servers may run multiple versions. Obviously it's a bad idea to keep the server name in the Java implementation—for each new server, we need to recompile our sources. Other items should not live in the source files, such as usernames and passwords, which should be configurable for the specific deployment. On the other hand, we should not over-architect a simple test-case implementation. Normally the test-case specification already contains most of the system state and specific parameter descriptions for our scenario, so there's no point making everything parameterizable in the implementation.

During coding, you'll realize that many code sections appear in more than one test-case implementation (potentially in all of the test cases). If you're an experienced object-oriented developer, you'll be tempted to create class hierarchies and common classes. In some cases, that makes a lot of sense—for example, the login procedure should be a common method available for all test cases. However, you need to step back a bit and realize that you're not building a new production system on top of the target-of-test application—these Java classes are no more than test scripts to validate the Website output. Exercise common sense and aim for simple, sequential, and self-contained test scripts.

The test cases are typically fragile. If a developer changes a URL, reorganizes the layout's <table> structure, or changes a form element's ID, the visitor will probably not see any difference, but your test scripts will be blown. Expect a lot of rework and change for each test-case implementation. Object-oriented design could reduce the effort of reworking common parts in the test cases, but from the perspective of a quality assurance engineer or tester, I'm sure that a simple, sequential script that interacts with a Website is easier to maintain and fix.

Traceability is crucial for our test cases. If something goes KA-BOOM, or, for example, a calculation result is wrong, it's important to point the developer to the corresponding test-case specification and the use-case specification for a quick bug resolution. Therefore, annotate your implementation with references to the original specification documents. Including the version number of those documents is also useful. That could be just a simple code comment or a complex mechanism where the test reports themselves link to the documents; the important thing is to have the reference in the code and to keep the traceability.

When do I get to write code?

Now that you're aware of the requirements (use-case docs and corresponding test-case specifications), understand the framework's basics, and have a set of architectural guidelines, let's get to work.

For the development of the test-case implementations, I prefer to work in Eclipse. First of all, it has a nice JUnit test runner. You can select a Java class, and from the Run menu, you can run it as a JUnit unit test. The runner displays the list of recognized test methods and the execution result. When everything goes okay during the test run, it gives a nice green line. If an exception or assertion failure occurred, it displays a distressing red line. I think the visual feedback is really important—it offers a sense of accomplishment, especially when writing unit tests for your own code. I also like to use Eclipse for its refactoring capabilities. If I realize that within a test-case class I need to copy and paste code sections, I can just use the Refactoring menu to create a method from the code section instead. If I realize that numerous test cases will use the same method, I can use the menu to pull up my method into my base class.

Based on the architectural requirements above, for each project, I typically create a base test-case class, which extends the JUnit TestCase class. I call it ConfigurableTestCase. Each test-case implementation extends this class, see Figure 2.

Figure 2. Configurable test-case implementations

ConfigurableTestCase typically contains the common methods and initialization code for the test case. I use a property file to store the server name, the application context, various login names for each role, and some additional settings.

The specific test-case implementations contain one test method per test-case scenario (from the test-case specification document). Each method typically logs in with a specific role and then executes the interaction with the Web application. Most test cases do not need a specific user to accomplish the activities; they typically require a user in a specific role, like Administrator, or Visitor, or Registered User. I always create a LoginMode enum, which contains the available roles. I use the Jakarta Commons ValuedEnum package to create enums for the roles. When a specific test method in a test-case implementation logs in, it must specify which login role is required for that particular test scenario. Of course, the ability to log in with a specific user should also be possible, for example, to verify the Registered User use case.

After each request and response cycle, we typically need to verify if the returned page contains an error, and we need to verify our assertions about what content the response should contain. We must be careful here as well; we should only verify items that are not variable and not too fragile in the application. For example, if we assert specific page titles, our tests will probably not run if the language is selectable in the application and we want to verify a different language deployment. Similarly, there's little point in checking an item on the page based on its position within a table layout; table-based designs change frequently, so we should strive to identify elements based on their IDs. In case some important elements on the page don't have IDs or names, we should just ask the developers to add them, rather than trying to work around them.

JUnit assertions offer a poor approach for checking if the look and feel, layout, and page design comply with the requirements. It is possible, given an infinite amount of time for the test development, but a good human tester can assess these things more efficiently. So concentrate on verifying the Web application's functionality, rather than checking everything possible on the page.

Here's an updated test scenario based on our test-case architecture. The class extends ConfigurableTestCase, and the login details are handled in the base class:

     * Verifies that submitting the login form with the name "master" results
     * in a page containing the text "Top Secret"
    public void testGoodLogin() throws Exception {
        WebConversation     conversation = new WebConversation();
        WebResponse response = login(conversation, LoginMode.ADMIN_MODE);
           assertTrue( "Login not accepted", 
             response.getText().indexOf( "You made it!" ) != -1 );
        assertEquals( "Page title", "Top Secret", response.getTitle() );

Tips and tricks

Most scenarios can be handled quite easily by setting WebForm parameters and then looking for specific elements with results in the WebResponse pages, but there are always some challenging test cases.

  • The RhinoScript JavaScript engine works quite well, but some complex JavaScripts could trigger exceptions in the engine. If that happens, your test fails. For those specific test-case methods, ignore the exceptions by setting HttpUnitOptions.setExceptionsThrownOnScriptError() to false.
  • Select a form to submit based on its ID: even if only one form appears on the page currently, the application can change later. Developers could add a search box on the top, for example, and your tests will potentially fail even if the specific form you're using doesn't change.
  • Don't assume a specific execution order for your test classes or the test methods within the classes. Each method should be successfully runnable individually.
  • I usually create constants for page URLs and URL patterns in each test case. You could argue that externalizing the URLs to property files, for example, could potentially reduce the effort to maintain the test cases, but it would prove harder for a tester to review and update a test script. Having mostly everything in the test case itself allows the tester just to check the sequence of request/response interactions and assertions, and potentially paste the URLs into an actual browser to quickly pinpoint the problem. It's important not to treat the test-case code as significant systems code—it must be easy to maintain and should not require a J2EE architect to understand and work with it.
  • I use log4j statements to keep debug code in the test cases. I typically have a log.debug(response.getText()) after each significant step in the sequence. For the automated test runs, the debug info will not be logged, but when I need to fix a specific problem, I can turn it on and see the actual HTML response received from the server in the log file.

Automated, scheduled test runs

Now we know how to write and execute tests in the IDE. However, these tests should also execute automatically from time to time. Based on the development method, these tests could run after a daily build or during a testing period. But the best option executes the full test suite or at least a subset many times a day, hourly for example, or after each commit in the version control system.

Most Web applications require a database, and the test-case scenarios typically have preconditions, or initial system state. So any automated test run should be executed on a cleanly built system, using a fresh database, which may already contain some test data specified in the test cases. Actually other alternatives are available; for example, each test case can prepare the test data for itself during the setup method, but I prefer that the database scripts insert the test data. That way, the quality assurance engineers manually testing the application can see the same initial state and configured preconditions for their tests.

Some tests may put the system in an invalid state. Typically Web applications should not allow input that can cause a system-wide failure. But perhaps the reason for the automated test case is to highlight just such an issue in the application. Running this test case with the rest of the test cases could heavily influence many other tests. These "viral" tests can be excluded from the batch test run and can run separately, restoring the initial database state after running them.

The Ant build system has everything needed to run these tests: available tasks can load the test data in the database, a <junit> task can execute our test cases, and there is a task to format an HTML report from our test cases' result. So all we need to do is to create a build.xml file, which compiles the test-case classes, executes the database init scripts, executes the tests, and then outputs the report for us. It's useful to have up-to-date HTML reports available on a project-specific intranet site. That way, the developers and the management can see the application's status.

Here's a sample section from an Ant build.xml file:

<!-- runs all unit test -->
<target name="junit" depends="jar_test">
      <mkdir dir="${log.dir}" />
             <junit printsummary="yes" haltonfailure="no">
                   <classpath refid="compile.class.path"/>
                   <fileset dir="${dist.dir}" includes="*.jar" />
                   <pathelement location="$(conf.dir}" />
            <batchtest fork="yes" todir="${log.dir}">
          <formatter type="xml" />
          <fileset dir="${src.test.dir}">
            <include name="**/*Test.java" />
    <junitreport todir="${log.dir}">
         <fileset dir="${log.dir}">
           <include name="TEST-*.xml"/>
         <report format="frames" todir="${log.dir}"/>

Functional testing for programmers

While writing unit tests is a popular and fashionable activity, most software developers I know either hate or simply don't like to do functional testing on other people's applications. While using HttpUnit will not replace manual testing, it is nevertheless a great way to create automated tests programmatically, with much more freedom than the typical visual test scripting environments. While tools recording browser activity tend to record every insignificant detail, with big and hard-to-modify recorded scripts, using HttpUnit will allow a developer to create readable, concise, and cleanly structured code, which can focus on validating the important details. So if you get a task to do functional tests for a few weeks, I assure you that you'll encounter enough challenges to keep you interested and to enjoy your work.

Balazs Fejes is the chief Java architect at EPAM Systems, focusing on WebLogic solutions. Check out his blog at http://fb2.hu/blogs/x10.php.

Learn more about this topic

Join the discussion
Be the first to comment on this article. Our Commenting Policies