Integrating Arquillian and JBehave

Three Arquillian extensions for behavior-driven development in the container

From an agile perspective, integrating Arquillian and JBehave makes good sense, which is why a number of developers have already written open source extensions that bridge the two frameworks. Get introduced to three such extensions (one of them written by the author) and see how they can incrementally improve the experience of behavior-driven development with Arquillian. Who knows, one of them could be the missing piece you've been looking for -- or the foundation for your next open source project.

Arquillian is a Java EE integration testing framework that executes tests in the container, testing not just your core application but its interaction with business components. While Arquillian makes writing functional tests much easier, I found it lacking in support for behavior-driven development (BDD). Specifically, I wanted to create a bridge between Arquillian and JBehave, the popular BDD testing framework.

I wrote the Arquillian-JBehave extension to create a strong integration between Arquillian and JBehave. The extension is fast -- it cuts some corners to get around Arquillian's standard behavior of deploying once per test class -- and it provides detailed, BDD-style feedback about what happens at every step of the testing scenario and execution.

In this article I briefly introduce both Arquillian and JBehave and discuss the advantages of extending Arquillian to support behavior-driven development. I then introduce two of the existing integration solutions -- Arquillian JBehave TestRunner and JBehave JUnit Runner -- and explain how my extension builds on them. Finally, I offer a demonstration of my Arquillian-JBehave integration project. All three projects discussed in this article are open source and hosted on GitHub.

Integrating Arquillian & JBehave

Arquillian is an integration, functional, and acceptance testing framework that was founded as a JBoss community project. Based on the JBoss Test Harness, Arquillian breaks the golden rule of unit testing (no dependencies) by throwing out mock objects and testing directly in the container. Injecting CDI beans, EJBs, and other container resources directly into test classes and driving tests in the browser enables a richer, more informative, and more user-friendly testing experience. Arquillian is also container agnostic, integrates well with most testing frameworks and IDEs, and was designed to be an extensible platform. Demonstrations in this article were developed using Arquillian as a test runner inside of JUnit 4.

Listing 1 shows a very basic test in Arquillian

Listing 1. A test class in Arquillian


@RunWith(Arquillian.class)
public class GreeterTest {

    @Deployment
    public static WebArchive createDeployment() {
        return ShrinkWrap.create(JavaArchive.class)
            .addClass(MyWebService.class)
    }

    @Test
    public void myWebServiceTest() {
        Assert.fail("Not yet implemented");
    }
}

The two key elements in the above code are the @RunWith and @Deployment annotations. @RunWith tells JUnit to run the test class using Arquillian's own runner, while @Deployment marks which methods should be considered as deployment producers. A deployment is represented by a single Java archive or a multi-module enterprise archive and is the subject of the test.

JBehave is a popular framework for doing behavior-driven development in Java. Similar to test-driven development, BDD is iterative. But rather than running your code through the wringer, BDD focuses on application scenarios and behavior. In BDD, application testing means writing application requirements in executable form. These requirements are comprised of stories, each with one or more scenarios. To describe BDD scenarios, JBehave supports Gherkin, a powerful, simple (in the sense that it uses a small number of keywords) and platform-independent business readable DSL.

JBehave leverages unit testing frameworks like JUnit and TestNG to connect its scenarios to the test code. Listings 2 and Listing 3 show a very basic JBehave test class and its associated story.

Listing 2. A JBehave-JUnit test class


public class CalculatorStory extends JUnitStory {

    private int x, y;

    @Override
    public InjectableStepsFactory stepsFactory() {
        return new InstanceStepsFactory(configuration(), this);
    }

    @Given("X is $X and Y is $Y")
    public void givenMethod(@Named("X") int x, @Named("Y") int y) {
    	this.x = x;
    	this.y = y;
    }

    @When("I add X to Y")
    public void add() {
    }

    @Then("I get $R as result")
    public void testAddition(@Named("R") int r) {
        Assert.assertEquals(r, x + y);
    }
}

Listing 3. The associated story file


Scenario: Addition
Given X is 5 and Y is 3
When I add X to Y
Then I get 8 as result

Listing 2 shows the most basic JBehave test possible. It is compact because JBehave leverages the convention over configuration principle. The key elements in this example are the @Given, @When, and @Then annotations, along with the stepsFactory method. Note that the annotations' values match the text in each of the corresponding steps from the scenario in Listing 3, thus linking the test code and the scenario. The stepsFactory method tells JBehave in which classes to look for @Given, @When, and @Then annotated methods. (I supplied them in the same class for brevity but it would be more usual for them to exist in separate classes, also called steps candidate classes.)

JBehave's implementation of BDD concepts can add much to Arquillian's functional and integration testing framework, extending it to be a more business-driven testing environment. Bridging the two frameworks would enable developers to do BDD inside the container, without the mocking and test isolation specific to unit testing.

Arquillian-JBehave integration solutions and drawbacks

Arquillian has generated considerable interest since it was launched, and many extensions have been proposed or created. Several extend Arquillian to support BDD with JBehave, but none went as far as I needed it to in terms of scalability and granularity.

I'll go through my list of features for a successful Arquillian-JBehave integration, then point out which ones are currently supported and which are not. I'lll also briefly introduce the two extensions that I used as a starting point for my own project.

Feature 1: Executing and reporting scenarios in the container

The most basic feature of an Arquillian-JBehave integration would be the ability to execute JBehave scenarios inside the container. Arquillian JBehave TestRunner supports this functionality. The downside of TestRunner is that it requires that you run the whole JBehave story (or set of stories) inside a single @Test annotated method. The resulting JUnit report contains information only about the top-level test method: you can't drill down to a more granular level to view how individual scenarios and steps performed.

One approach to resolving this issue would be to enhance JBehave's JUnit runner with a better reporter, perhaps one that was linked with JUnit's RunNotifier. The JBehave JUnit Runner project does just that, generating a JUnit run report with a step-level granularity.

Combined, these two open source integration projects execute stories inside the container while reporting on the test's progress; but they still leave a few things to be desired.

For a start, let's consider a test scenario that would leverage Arquillian's persistence extension, which uses dbUnit behind the scenes. In order to test the search functionality of some library software, we would need to pre-populate the database with books, but not without cleaning it up first to ensure a clean test bed. We would need to do all this before executing the @Given step, which is always the first step in a BDD test scenario.

Listing 4 shows how such a test could look if Arquillian's persistence extension and JBehave were tightly integrated.

Listing 4. Arquillian's persistence extension in a JBehave step candidate


…
@Test
@UsingDataSet("datasets/books.xml")
@Given(“a set of books”)
@Cleanup(phase = TestExecutionPhase.BEFORE)
public void books() {
}
…

In this example the database cleanup is achieved via the @Cleanup(phase = TestExecutionPhase.BEFORE) while the population with test data is handled by the @UsingDataSet("datasets/books.xml"). But achieving this level of granularity (i.e., working at step level) is not possible with the existing framework and extensions because:

  1. Arquillian supports database-related annotations (like @Cleanup and @UsingDataSet) only on actual @Test methods; and
  2. the current integration between JBehave and Arquillian means providing a class with a single @Test annotated method that runs all the scenarios at once.

Annotating the only available @Test method instructs Arquillian to prepare the database as requested and then give control to JBehave to run the scenarios. It won't be able to intervene until all of the scenarios have been executed. So no matter how many scenarios we execute, the books inserted before will exist in the database until all of the tests have been executed, and not just for a particular one, which is what we require to properly test the library software.

Without scenario-level data isolation we can't properly test our software. This limitation affects not just the persistence extension but every other feature in the Arquillian framework that leverages before and after test events.

Feature 2: Running multiple stories

As a project grows, it's likely that we'll want to be able to run multiple stories at once. We could run each story separately but that approach leads to time wasted on packaging and deploying each story. JBehave does allow us to run multiple stories at once, but in order to work it must be able to detect the location of the story files, which is a more involved process inside of a container than out of it. An Arquillian-JBehave integration can't rely solely on JBehave's running framework to execute multiple stories at once because JBehave alone is not capable of locating the story files inside the container's virtual filesystem.

In order to test multiple stories, we would typically write multiple steps candidate classes. (Unless you like 3,000-line classes, that is!) In order to get down to the step level, in terms of granularity, and leverage Arquillian's before-and-after test events, Arquillian must be able to consider each steps candidate a test method, which would be annotated with @Test. Arquillian's big limitation in this regard is that it can only execute methods inside of a single class for each deployment. As a result, we have to redeploy after each test class (or JBehave steps candidate class). That isn't a big deal for a small number of stories, but it clearly isn't scalable.

The Arquillian-JBehave extension

In order to work around these issues I have developed a new Arquillian-JBehave integration that builds on the two projects that I mentioned earlier: Arquillian JBehave TestRunner and JBehave JUnit Runner. While my methods aren't quite models of the Gang of Four design patterns (for instance, I used class instrumentation to get around Arquillian's limitation of a single Java test class per deployment) the extension does what I need it to do. Specifically, here's what the arquillian-jbehave extension offers:

  • Arquillian annotations directly on step methods
  • Annotations to bind a test class to a story file
  • Ability to execute multiple stories with a single deployment

The limitation of the project is that steps candidate classes must be supplied to JBehave as static instances. Code cops will agree that this isn't perfect programming technique, but it doesn't limit the actual testing functionality. (See "A note about deployment" at the end of this article for an explanation.)

New Arquillian-JBehave integration demo

First I'll demonstrate how to run a JBehave story using the arquillian-jbehave extension, then I'll explain a few things about how it works.

Listing 5. Arquillian-JBehave integration demo


@RunWith(ArquillianJbehaveRunner.class)
@StoryFile("example")
public class ExampleStory extends AbstractArquillianStory {

	public static ExampleTestSteps steps = new ExampleTestSteps();

	@Deployment
	public static WebArchive createDeployment() {
		//create the WebArchive to deploy – skipped for brevity
	}

	@Override
	public InjectableStepsFactory stepsFactory() {
		return new ArquillianInstanceStepsFactory(this.configuration(), steps);
	}
}

The first thing to note about the example in Listing 5 is the @RunWith annotation, which features a new runner called ArquillianJbehaveRunner. This updated annotation is the main actor of the Arquillian-JBehave integration. It enables us to deploy multiple stories at once, and also gets down to the step level of granularity by leveraging before-and-after test events. The next annotation, @StoryFile, is a simple extension that tells JBehave the name of the associated story file, which is implied but can be configured. In this case, the annotation tells JBehave to look for a file named example.story in the classpath under the folder stories (the folder name and .story extension are default values but can be configured).

1 2 Page
Join the discussion
Be the first to comment on this article. Our Commenting Policies
See more