Introducing continuous integration

How continuous is your integration and what could your team be doing to improve it? A JavaWorld excerpt from Continuous Integration: Improving Software Quality and Reducing Risk (Addison-Wesley Professional, June 2007)

1 2 3 Page 2
Page 2 of 3

How do I get to 'continuous' integration?

It's often surprising to learn the level of automation of most development organizations. Developers spend most of their time automating processes for their users, yet don't always see ways to automate their own development processes. Sometimes teams believe their automation is sufficient because they've written a few scripts to eliminate some steps in the development process. The following is a typical scenario.

Joan (developer): ...I already automated that. I wrote some batch scripts that drop and recreate the database tables.

Sue (technical lead): That's great. Did you apply it to the CVS repository?

Joan: No.

Sue: Did you make it a part of the build script?

Joan: No.

Sue: So, if it's not a part of the CI system, then it's not really automated yet, right?

CI is not just the process of gathering a few scripts together and running them all the time. In the preceding scenario, it's great that Joan wrote those automation scripts, but in order for them to actually add value to the end product, they must be added to the version-control repository and made a working part of the build process. Figure 2-2 illustrates the steps to making a process continual

These steps can be applied one by one to virtually every activity you conduct on a project.

  • Identify: Identify a process that requires automation. The process may be in the areas of compilation, test, inspection, deployment, database integration and so on.

  • Build: Creating a build script makes the automation repeatable and consistent. Build scripts can be constructed in NAnt for the .NET platform, Ant for the Java platform, and Rake for Ruby, to name a few.

  • Share: By using a version-control system such as Subversion, you make it possible for others to use these scripts/programs. Now the value is being spread consistently across the project.

  • Make it continuous: Make sure that the automated process is run with every change applied, using a CI server. If your team has the discipline, you can also choose to manually run the build with every change applied to the version-control system.

Here is an acrostic to help you remember and communicate this: "I Build So Consistently" -- for Identify, Build, Share and Continuous.

Aim for incremental growth in your CI system. This is simple to implement, the team gets more motivated as each new item is added and you can better plan what you need next based on what's working so far. Often, attempting to throw everything into a CI system immediately can be a bad move, just like refactoring a lot of code at once isn't the best approach when writing software. Get it to work first, get developers using it and then add other automated processes as needed based on the project risks.

figure 2.2
Figure 2-2

Getting to CI -- "I Build So Consistently"


Is it continuous compilation or continuous integration? I've worked with a number of organizations on implementing CI, and on several occasions I've heard the reply, "Yes, we do CI." Of course, I think, "Great!" and then ask a few questions. How much code coverage do you have with your tests? How long does it take to run your builds? What is your average code complexity? How much code duplication do you have? Are you labeling your builds in your version-control repository? Where do you store your deployed software?

I discover that what they've been doing all along is more like a "continuous compilation," in which they've set up a tool like CruiseControl to poll their version-control repository (e.g., CVS) for changes. When it detects changes, it retrieves the source code from CVS, compiles the code and sends an e-mail if anything goes wrong. Automatically compiling the software system on a separate machine is better than nothing at all, but doing that isn't going to provide all of the benefits of a full-featured CI system.


When and how should a project implement CI?

It is best to implement CI early in the project. Although possible, it is more difficult to implement CI late in a project, as people will be under pressure and more likely to resist change. If you do implement CI later in a project, it is especially important to start small and add more as time permits.

There are different approaches to setting up the CI system. Though you eventually want a build to run on every change to the system, you can start by running a build on a daily basis to get the practice going in your organization. Remember: CI is not just a technical implementation; it is also an organizational and cultural implementation. People often resist change, and the best approach for an organization may be to add these automated mechanisms to the process piece by piece.

At first the build can just compile the source code and package the binaries without executing the automated regression tests. This can be effective, initially, if the developers are unfamiliar with an automated testing tool. Once this is in place and developers have learned the testing tool, you can move closer to the benefits of CI: running these tests (and inspections) with every change.

The evolution of integration

Is CI the newest, latest, "whiz-bang" approach to software development? Hardly. CI is simply an advance in the evolution of integrating software. When software programs consisted of a few small files, integrating them into a system was not much of a problem. The practice of performing nightly builds has been described as a best practice for years. Similar practices have been discussed in other books and articles. In Microsoft Secrets, Michael A. Cusumano and Richard W. Selby discuss the practice of daily builds at Microsoft. Steve McConnell, in Software Project Survival Guide, discusses the practice of the "Daily Build and Smoke Test" as part of a software-development project.

In Object Solutions: Managing the Object-Oriented Project, Grady Booch writes, "The macro process of object-oriented development is one of 'continuous integration'... At regular intervals, the process of 'continuous integration' yields executable releases that grow in functionality at every release... It is through these milestones that management can measure progress and quality, and hence anticipate, identify, and then actively attack risks on an ongoing basis." With the advent of XP and other Agile methodologies, and with the recommended practice of CI, people began to take notice of the concept of not just daily, but "continuous," builds.

The practice of CI continues to evolve. You'll find the practice in almost every XP book. Often, when people discuss the practice of CI, they refer to Martin Fowler's seminal "Continuous Integration" article.4

As hardware and software resources continue to increase, you'll find that more processes will become a part of what is considered to be CI.

1 2 3 Page 2
Page 2 of 3