Newsletter sign-up
View all newsletters

Enterprise Java Newsletter
Stay up to date on the latest tutorials and Java community news posted on JavaWorld

Sponsored Links

Optimize with a SATA RAID Storage Solution
Range of capacities as low as $1250 per TB. Ideal if you currently rely on servers/disks/JBODs

Agility meets the Waterfall

Agile best practices in Waterfall-based Java enterprise development

  • Print
  • Feedback

Some have said that 2007 was the year that Agile arrived, with agile development best practices such as automated builds, test automation, and continuous integration finally beginning to reach critical mass among Java developers. While this is true in theory, the fact is that most Java enterprise projects are still based on more traditional development methods, such as the Waterfall model. In this article, ShriKant Vashishtha explores some of the "pain points" of traditional Java EE development, from both a developer and a build manager's perspective. He then shows us how certain agile best practices can easily and naturally resolve these problems -- without altering the flow of a Waterfall project.

Java EE best practices

This article builds on my discussion in the JavaWorld article "J2EE project execution: Some best practices." That discussion focused on improving the productivity of Waterfall-based projects using Java EE development best practices such as a reference implementation, an effective developer's handbook, and automated code reviews. In this article I focus on the benefits of incorporating agile best practices like automated buiids, continuous integration, and test automation into Waterfall-based Java EE projects.

The buzz around agile development methodology is undeniable and growing, but actually transitioning a traditional shop to an agile one is a huge undertaking. The scope, timeline, and deliverables associated with traditional software development do not map easily to the agile process and some must be thrown out entirely. Managers, developers, and even clients must commit to the less predictable rhythm and greater personal demands of agile development. Change has to be managed across the entire enterprise and results evaluated based on a less familiar set of criteria. Geographically distributed teams are especially challenged to stay on target while implementing agile processes, which favor face-to-face communication. Is it any wonder that agile development appears to be more popular in theory than in practice?

According to a Forrester Research study, only 26% of North American and European enterprises were using agile practices in 2007. Despite rumors of its demise, the Waterfall model (where each stage of the software creation lifecycle must be complete before the next one begins) is still status quo in Java enterprise development. For some Java developers this is a frustrating state of affairs: we want to improve the process of development and the quality of our products, and we believe that Agile is the way to do it; but the Waterfall model stands in the way.

Or does it? While some of us would like to simply trade in traditional models for agile ones, a gradual transition is more practical. Even if your company or development group is tied to the Waterfall model, it is possible to incorporate agile best practices to improve productivity and efficiency. In fact, many Waterfall-based projects already support agile best practices, and could support even more.

In this article I present a scenario that illustrates common issues in traditional development projects, from both a developer's and a build manager's perspective. I then show where agile best practices such as automated builds, test automation, and continuous integration can help overcome some of these problems, even in a Waterfall-based context. I also introduce a number of tools that can be used to test, measure, and improve code quality in test automation and CI environments.

The evolution of the Waterfall model

It is tempting to start a discussion of this sort by enumerating the weaknesses of the Waterfall model and then showing how they can be overcome through agile practices. In fact, the Waterfall model has evolved to incorporate practices very similar to ones used in agile development. Consider these:

  • Build frequency: Gone are the days when most teams could wait until the final stage of development to integrate all of the pieces of an application. In today's world, it is common to develop, unit test, and integrate software in a short timespan. I have seen many Waterfall-based projects where builds happen two to three times per day.
  • Iterative development (small releases) and frequent feedback: In traditional Waterfall-style projects, software is delivered for system testing or acceptance testing after it has been developed. Few large or mission-critical projects can afford to work that way today. Customers often expect to monitor the quality of constructed software through continuous code reviews, encouraging the use of multiple iterations in shorter development cycles. The delivered functionality is prioritized based on the customer's requirements and development feasibility, and the customer provides feedback on each delivered iteration. Feedback has a direct impact on the next iteration and there are fewer (if any) surprises for the customer when the final product is received.
  • Change control: It has been argued that the basic disadvantage of the Waterfall model is that the requirements phase is closed before development begins. This process allows no room for changes later in the development cycle, even though the need for changes almost inevitably becomes apparent in these later stages. Many Waterfall-based IT shops have adopted a change control process based on how critical the change is, as well as the effort required to implement the change, as illustrated in Figure 1.
    Diagram of a Java EE change control process.

    Figure 1. A typical change control process


    An effective change control process is defined and documented to ensure that changes are effectively managed and controlled. (A change, in this context, can mean a revision to the program scope, deliverables, milestones or levels of services, new clarity configurations, or enhancements that affect the cost, schedule, resources, quality or conformance of the services to the agreed specifications.)

All of these typically agile best practices complement Waterfall-based development and make it more accessible and responsive to the needs of the client, while also breaking down the rigid separation of cycles that characterizes the Waterfall model.

Traditional Java EE projects could benefit from incorporating a few more best practices from the agile development world, as the next section reveals.

  • Print
  • Feedback

Resources
  • "J2EE project execution: Some best practices" (ShriKant Vashishtha, JavaWorld, November 2005) explains how template code, a developer's handbook, and automated code reviews can improve Java enterprise project execution.
  • "Introducing continuous integration" (Paul Duvall, Steve Matyas, Andrew Glover; JavaWorld, June 2007) is an overview of CI process and tools, excerpted from Continuous Integration: Improving Software Quality and Reducing Risk.
  • Java Power Tools (John Ferguson Smart, O'Reilly, April 2008) is a guide to open source tools for the entire software development lifecycle, including many that were mentioned in this article. JavaWorld recently interviewed the author about some of his favorite tools for test automation and CI.
  • "Groovy-power automated builds with Gant" (Klaus P. Berg, JavaWorld, February 2008) introduces the newest automated-build tool on the block: Groovy+Ant.
  • "Best practices for test-driven development" (Michael Grove and Brooks Bollich, JavaWorld, May 2004) introduces TDD -- an agile technique not so suitable for Waterfall-based projects -- and offers background on the use of mock objects in test automation.
  • "The demise of the Waterfall model Is imminent and other urban myths" (Phillip A. Laplante and Colin J. Neill, ACM Queue, February 2004) seeks the truth behind some popular folklore of computer programming.
  • Forrester Research has been tracking agile adoption for several years, with the most recent survey results released in February 2008.
  • "Agile people still don't get it" (Cedric Beust, Otaku, June 2006) expresses the occasional disconnect between agile evangelism and the challenge of organizing "huge code bases growing thousands of lines of code every day under the combined push of hundreds of a developers, all with their personal background, education and bias."
  • "Software testing with Spring framework" (Srini & Kavitha Penchikala, InfoQ, November 2007) is an overview of specifically agile software testing techniques like TDD, and of integration testing with the Spring framework.
  • "Testing persistent objects without Spring" (James Richardson, Time4Tea.com, November 2007) shows one way to test persistent classes without Spring.
  • Martin Fowler is always a good read. Here he explains the process of continuous integration.
  • " Philosophy of test automation" is an excerpt from xUnit Test Patterns (Gerard Meszaros, Addison-Wesley, May 2007).
  • See the JavaWorld Development Tools Research Center for hands-on introductions to the tools discussed in this article.
  • Also see Network World's IT Buyer's Guides: Side-by-side comparison of hundreds of products in over 70 categories.

Tools discussed in this article