Newsletter sign-up
View all newsletters

Enterprise Java Newsletter
Stay up to date on the latest tutorials and Java community news posted on JavaWorld

Sponsored Links

Optimize with a SATA RAID Storage Solution
Range of capacities as low as $1250 per TB. Ideal if you currently rely on servers/disks/JBODs

Agility meets the Waterfall

Agile best practices in Waterfall-based Java enterprise development

  • Print
  • Feedback

Page 4 of 4

Continuous integration

Uncompilable code was one of the problems mentioned by both the developer and manager in our Java EE project scenario. The developer often has to deal with broken code that has been uploaded to the SVN repository, and the manager also spends valuable time resolving compilation issues.

As a first observation, this development team needs to adopt a basic discipline that nobody will commit the code in SVN if it’s not compilable. Beyond that, the basic problem is that human is practically synonymous with error.

Who ensures that the build is ready to compile at all times? In the scenario described, it is a manual process where the build team checks out the code from source repository, makes the build, and often realizes that the build doesn’t compile. Something is wrong with this process, don’t you think? Build time is too late to report an error to a developer and then let him correct it!

These common problems are compounded by leaving integration as the very last step in a software development cycle. Instead, why not integrate the project multiple times per day? In a continuous integration environment each integration is verified by an automated build, which includes executing test cases to detect errors as soon as possible. Continuous integration, or CI, involves communication, discipline, and automation of trivial tasks. As such, it offers some good solutions to the issues raised by the developer and build manager.

CI servers

In a large project, manual checks are likely not sufficient to curb all of the problems raised by the developer and build manager. Getting them fixed can itself be a big task. Wouldn't it be better to automate this process? Here is where a continuous integration server like CruiseControl comes in handy. CruiseControl performs many trivial tasks automatically. Consider these:

  • Whenever someone commits changes to the source repository, it will check out those source files from the repository and build the project. It always has some kind of waiting period, so that if a person is committing multiple sources, it doesn’t start compilation in between. This time-lag is configurable. Whenever a build breaks, CruiseControl will check the user ID of the person who last committed changes to the source repository and will send a snapshot of the compilation issue through an email. You can configure the email IDs of stakeholders for each component. The team should take immediate action to fix the build issue based on the information provided.
  • CruiseControl provides a Web-based user interface where stakeholders can view the state of the build at all times. If you click on the link of a failed build, it will provide all problems from the last build.
  • If a build fails, CruiseControl doesn’t execute the build again until someone commits changes to the repository.
  • Various artifacts like EARs, WARs, JUnit reports, PMD reports, and more can be downloaded/seen from the Web-based user interface.
  • In small projects where there is no interdependence between components, it’s fine to build whenever someone commits changes to the repository. For projects with tight interdependence between components this could be a problem, however. In that case, CruiseControl can be set to compile individual components at the time of last commit, but build the entire project only periodically. (Building the entire project too frequently can lead to a situation where people are constantly buried in email; after a while it becomes tempting to avoid email from the build team.)

Ready to go?

Even with all of these solutions and tools at their disposal, the developer and build manager in our initial project scenario may still have some basic organizational problems to work out. Most importantly, both developer and build manager mentioned the need for code checked into the SVN repository to be compilable and ready to build at all times.

This project is typical of enterprise development projects in that it actually consists of many inter-related Java projects, or functional components. Each of these is likely dependent on one or more other projects, and each project likely provides a client JAR file (contains only interfaces and necessary classes of functional component to compile the client class) or a full JAR (contains entire distribution of the functional component) file depending on the type of functionality provided. All of the JAR files are available in the lib folder of each Java project, but keeping them updated across all the Java projects in such a large project is a big undertaking, as the manager mentioned.

It’s difficult to track which application component is running on which version of the internally generated JAR files, and this is also true for third-party JARs. It is possible to resolve such a problem by creating a single, centralized Java project that only contains JARs (third-party as well as internally generated).

Now, if the project team creates new internal JARs on a daily basis, and the source code available in SVN is in work-in-progress state, it may break the functionality of other components. You may start receiving runtime issues. The answer lies in released version of internal JARs. First of all, developers should commit a code in SVN which has passed a set of test cases. The source repository should not have unstable code. Whenever there are updates in the source code and it is functionally stable, the application component team may release an update version of the JAR file in the centralized libs. Each application component team should have a person assigned to this role. One needs to run local package’s tests and ensure they pass before committing changes in the source repository.

In conclusion

It has not been my goal in this article to advocate for agile development, per se. Even while suggesting some best practices from the agile world, I have tried to keep the spirit of the Waterfall model intact. You should be able to incorporate automated builds or test automation into a Waterfall project without disrupting its essential flow. When you start thinking about pair programming, just enough documentation or no documentation, business people as part of the development team, no upfront design, no roles (architect, build manager, etc.) -- well, then you are meddling with the spirit of the Waterfall model.

These factors will directly impact stakeholders and the way the IT team works as a whole. The best practices discussed in this article, by contrast, gel nicely with a Waterfall-based project -- automated builds provide ready-to-build code at all times; test automation brings predictability, precision, and reliability to the process of testing code; and continuous integration is helpful for maintaining ready-to-build code. Taken as a whole these techniques improve the efficiency of the project and optimize the way individual team members work, and they can do so without affecting the steady, incremental flow we've come to associate with Waterfall-based projects.

See the Resources section to learn more about the techniques and tools discussed in this article. You can also share your own experiences in the discussion forum associated with this article.

About the author

ShriKant Vashishtha currently works as a Principal Consultant for Xebia IT Architects India Private Limited. He has more than nine years of experience in the IT industry and is involved in designing technical architectures for various large-scale Java EE-based projects for the banking and retail domains. ShriKant holds a bachelor's degree in engineering from the Motilal Nehru National Institute of Technology in AllahaBad, India.
  • Print
  • Feedback

Resources
  • "J2EE project execution: Some best practices" (ShriKant Vashishtha, JavaWorld, November 2005) explains how template code, a developer's handbook, and automated code reviews can improve Java enterprise project execution.
  • "Introducing continuous integration" (Paul Duvall, Steve Matyas, Andrew Glover; JavaWorld, June 2007) is an overview of CI process and tools, excerpted from Continuous Integration: Improving Software Quality and Reducing Risk.
  • Java Power Tools (John Ferguson Smart, O'Reilly, April 2008) is a guide to open source tools for the entire software development lifecycle, including many that were mentioned in this article. JavaWorld recently interviewed the author about some of his favorite tools for test automation and CI.
  • "Groovy-power automated builds with Gant" (Klaus P. Berg, JavaWorld, February 2008) introduces the newest automated-build tool on the block: Groovy+Ant.
  • "Best practices for test-driven development" (Michael Grove and Brooks Bollich, JavaWorld, May 2004) introduces TDD -- an agile technique not so suitable for Waterfall-based projects -- and offers background on the use of mock objects in test automation.
  • "The demise of the Waterfall model Is imminent and other urban myths" (Phillip A. Laplante and Colin J. Neill, ACM Queue, February 2004) seeks the truth behind some popular folklore of computer programming.
  • Forrester Research has been tracking agile adoption for several years, with the most recent survey results released in February 2008.
  • "Agile people still don't get it" (Cedric Beust, Otaku, June 2006) expresses the occasional disconnect between agile evangelism and the challenge of organizing "huge code bases growing thousands of lines of code every day under the combined push of hundreds of a developers, all with their personal background, education and bias."
  • "Software testing with Spring framework" (Srini & Kavitha Penchikala, InfoQ, November 2007) is an overview of specifically agile software testing techniques like TDD, and of integration testing with the Spring framework.
  • "Testing persistent objects without Spring" (James Richardson, Time4Tea.com, November 2007) shows one way to test persistent classes without Spring.
  • Martin Fowler is always a good read. Here he explains the process of continuous integration.
  • " Philosophy of test automation" is an excerpt from xUnit Test Patterns (Gerard Meszaros, Addison-Wesley, May 2007).
  • See the JavaWorld Development Tools Research Center for hands-on introductions to the tools discussed in this article.
  • Also see Network World's IT Buyer's Guides: Side-by-side comparison of hundreds of products in over 70 categories.

Tools discussed in this article