Agility meets the Waterfall

Agile best practices in Waterfall-based Java enterprise development

Some have said that 2007 was the year that Agile arrived, with agile development best practices such as automated builds, test automation, and continuous integration finally beginning to reach critical mass among Java developers. While this is true in theory, the fact is that most Java enterprise projects are still based on more traditional development methods, such as the Waterfall model. In this article, ShriKant Vashishtha explores some of the "pain points" of traditional Java EE development, from both a developer and a build manager's perspective. He then shows us how certain agile best practices can easily and naturally resolve these problems -- without altering the flow of a Waterfall project.

Java EE best practices

This article builds on my discussion in the JavaWorld article "J2EE project execution: Some best practices." That discussion focused on improving the productivity of Waterfall-based projects using Java EE development best practices such as a reference implementation, an effective developer's handbook, and automated code reviews. In this article I focus on the benefits of incorporating agile best practices like automated buiids, continuous integration, and test automation into Waterfall-based Java EE projects.

The buzz around agile development methodology is undeniable and growing, but actually transitioning a traditional shop to an agile one is a huge undertaking. The scope, timeline, and deliverables associated with traditional software development do not map easily to the agile process and some must be thrown out entirely. Managers, developers, and even clients must commit to the less predictable rhythm and greater personal demands of agile development. Change has to be managed across the entire enterprise and results evaluated based on a less familiar set of criteria. Geographically distributed teams are especially challenged to stay on target while implementing agile processes, which favor face-to-face communication. Is it any wonder that agile development appears to be more popular in theory than in practice?

According to a Forrester Research study, only 26% of North American and European enterprises were using agile practices in 2007. Despite rumors of its demise, the Waterfall model (where each stage of the software creation lifecycle must be complete before the next one begins) is still status quo in Java enterprise development. For some Java developers this is a frustrating state of affairs: we want to improve the process of development and the quality of our products, and we believe that Agile is the way to do it; but the Waterfall model stands in the way.

Or does it? While some of us would like to simply trade in traditional models for agile ones, a gradual transition is more practical. Even if your company or development group is tied to the Waterfall model, it is possible to incorporate agile best practices to improve productivity and efficiency. In fact, many Waterfall-based projects already support agile best practices, and could support even more.

In this article I present a scenario that illustrates common issues in traditional development projects, from both a developer's and a build manager's perspective. I then show where agile best practices such as automated builds, test automation, and continuous integration can help overcome some of these problems, even in a Waterfall-based context. I also introduce a number of tools that can be used to test, measure, and improve code quality in test automation and CI environments.

The evolution of the Waterfall model

It is tempting to start a discussion of this sort by enumerating the weaknesses of the Waterfall model and then showing how they can be overcome through agile practices. In fact, the Waterfall model has evolved to incorporate practices very similar to ones used in agile development. Consider these:

  • Build frequency: Gone are the days when most teams could wait until the final stage of development to integrate all of the pieces of an application. In today's world, it is common to develop, unit test, and integrate software in a short timespan. I have seen many Waterfall-based projects where builds happen two to three times per day.
  • Iterative development (small releases) and frequent feedback: In traditional Waterfall-style projects, software is delivered for system testing or acceptance testing after it has been developed. Few large or mission-critical projects can afford to work that way today. Customers often expect to monitor the quality of constructed software through continuous code reviews, encouraging the use of multiple iterations in shorter development cycles. The delivered functionality is prioritized based on the customer's requirements and development feasibility, and the customer provides feedback on each delivered iteration. Feedback has a direct impact on the next iteration and there are fewer (if any) surprises for the customer when the final product is received.
  • Change control: It has been argued that the basic disadvantage of the Waterfall model is that the requirements phase is closed before development begins. This process allows no room for changes later in the development cycle, even though the need for changes almost inevitably becomes apparent in these later stages. Many Waterfall-based IT shops have adopted a change control process based on how critical the change is, as well as the effort required to implement the change, as illustrated in Figure 1.

    Diagram of a Java EE change control process.
    Figure 1. A typical change control process

    An effective change control process is defined and documented to ensure that changes are effectively managed and controlled. (A

    change

    , in this context, can mean a revision to the program scope, deliverables, milestones or levels of services, new clarity configurations, or enhancements that affect the cost, schedule, resources, quality or conformance of the services to the agreed specifications.)

All of these typically agile best practices complement Waterfall-based development and make it more accessible and responsive to the needs of the client, while also breaking down the rigid separation of cycles that characterizes the Waterfall model.

Traditional Java EE projects could benefit from incorporating a few more best practices from the agile development world, as the next section reveals.

1 2 3 4 Page 1