Create a nightly build system from scratch

Learn how to put together a sophisticated nightly build system in two days or less

Let's suppose you have arrived at a new job site or landed a new contract. You may be a bit nervous, but not too much so; you have a reasonable level of confidence in your Java development skills. The first thing you do is try to get up and running. You read some of the documentation. You look at the hardware available for use. You check out the code from the source control system and run the Ant scripts to see how the current product looks as opposed to the project described in the (probably out-dated) schedule. Good.

Then you actually get into the thick of things. Somebody on the team wastes half your day by making a bad check-in, which you end up debugging since you must get your code compiling (you do want to get some work done, after all). Another morning comes and goes when somebody makes a late-night check-in to meet a deadline, and nobody notices that the build is broken until half the team checks out the broken build. Any seasoned developer has dealt with these types of issues in a variety of forms (and any seasoned developer has broken the build as well). How do you protect yourself and the team from forgetting to check in files, from producing code that only compiles on your development machine, or from instigating other frustrating and time-wasting behavior?

One tool in your arsenal is a continual build system. Continual build systems may go as far as to compile and run the unit tests every time you check in modifications. Regardless of how often you build, you always want at least a nightly build system. The benefits of such systems are well understood in the industry now—I know of no software development methodology that does not advocate a nightly build. As with other tools, it is no magic bullet, but it is part of the toolkit necessary to produce quality software.

At some point, you will ask management how builds and releases are handled. You may be surprised to discover that the builds are handled by hand on a release-by-release basis, which is common. There may even be a person (or entire department) devoted to producing software builds. In such a shop, each developer is usually on her own vis à vis building and testing daily. You can imagine (and probably have experienced) the problems that such systems can produce. So why don't you have a nightly build?

Invariably, everyone agrees that a nightly build system would be nice. But no time has been budgeted for such a snazzy tool—tools are a notorious waste of time for engineers. Weeks can be whittled away playing with the tool and slipping on the feature set. Management will not budget time for a build tool that requires weeks to create.

In this article, I show you how to put together a basic build system in a couple of days. It is just a matter of making some installs and plugging in your Ant scripts. We use the open source Web application Anthill; you will find its configuration to be only a small part of setting up such a system. So let's get to it.

Prerequisites

To start, I presume the following about your situation and your abilities: You are a reasonably savvy Java developer. You have worked on at least a couple of Java projects, and you not only know the language, but you also know a bit about Web applications and servlet containers. I am also going to presume that you know a bit about Ant and how to use Ant scripts. Finally, I'm going to presume that you have some understanding of source code control and have already put your code into such a system.

What is my justification for such presumptions? First, brevity: my goal is explain how to quickly create a nightly build system, not how to use CVS (Concurrent Versions System). Second, practicality: while it is true that most shops have no functional build systems, it has always been true, in my experience, that shops have source code control. Third, my estimation of the time it will take you to set up the system is predicated on the expectation that such items like a Tomcat server don't intimidate you. If you already know about Web applications, then running one is no big deal. But if you have no idea about such things and something goes wrong, trouble-shooting how to run the Web application will dramatically increase your time. My assertion is that a professional Java developer can set up a nightly build system in two days or less. I say two days, because I recommend you spend some time reading the manuals for each system component. Reading manuals and downloading software simply takes time, so while you might do this in an afternoon, you should probably allocate a couple of days.

The machine and its OS

We will create the example system on a Windows XP machine because it is common and, most likely, the kind of machine that you have easily available for use. The OS is really rather irrelevant. For Mac OS X, Linux, or Solaris users: you will set up a Java server and run a Java Web application. All the tools used are cross-platform or have a Unix equivalent. You do have to know how to do one thing: if I ask you to set an environment variable in the Windows environment, you will need to know how to set the environment variable in your flavor of Unix. Yes, that is it. You must know how to set environment variables and how to install off-the-shelf software. You will spend most of your time performing installations.

One item involving the path environment variable is of some importance: if you rely on running an executable from Ant, then the Anthill Web application requires it to be in your path. Sometimes when running an Ant script from Anthill, you will encounter Java I/O exceptions—Anthill's rather nonintuitive way of letting you know that it could not find an executable. So if your javac or devenv is not in your path, you will see these exceptions when you try to build those targets.

At this point, you've probably either grabbed a machine leftover from the last round of outsourcing or the box that you have not used since you last upgraded your equipment. Maybe it is a bit old, but I cannot think of a better use for an older, slower machine than a task that runs in the early hours of the morning! I presume the box is tabula rasa (that is, it has no tools on it), so I now will walk you through all the installations.

Source code control

In my example, we use CVS. The Anthill Web application supports more than just CVS. If you have Perforce, fine, please use it! But if you operate on the cheap, you can use CVS and get your code from a CVS password server (pserver) machine.

We start by downloading and installing the CVS client, WinCvs. We are not so interested in using WinCvs as we are in getting CVSNT onto the box. This generally will not be an issue on a Unix box—CVS is standard fare on a box these days.

The installation itself (like most installations I mention) should be relatively simple and won't take much time. You won't have to use WinCvs (although you may want to test your CVS connection using the WinCvs client to ensure your machine can talk correctly to the CVS server). If you do choose to test your CVS connection, you will have to supply the server information either at startup or via the Admin/Preferences menu item (see Figure 1).

Figure 1. Advanced System Properties panel. Click on thumbnail to view full-size image.

An additional reason for using a client application such as WinCvs is because it provides a convenient way to log into the server. The build system requires that you log onto your CVS server—one of the simplest ways to do that is to simply start up WinCvs and log in using the Admin/Login menu item. Remember, you must be logged in for your build system to work.

The most important thing for you to do is to add CVS (in this case CVSNT) to your path. On an XP system, this will require going to the Control Panel and opening up the System dialog. Open the Advanced panel and select the Environment Variables button (see Figure 2). This will bring up a new dialog. Select the Path variable and edit it. Add the location of your CVSNT installation to your Path variable. In my case, I installed CVS into a standard location so that my new path looks like this:

%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;C:\Program Files\GNU\WinCvs 1.3\CVSNT

Figure 2. An example of CVS settings. Click on thumbnail to view full-size image.

Ant

You need Ant on the system to run your Ant scripts. The download is available from the Apache Website. You will want a new version of Ant since it includes support for newer targets as well as support for such things as bean shell scripting. On Windows machines, I typically install Ant on my C: drive—say, C:\apache-ant-1.6.1.

We need Ant to be in the path, so add c:\apache-ant-1.6.1\bin to your Path Environment variable. We also need to add ANT_HOME to our environment variables; you'll probably need to add this as a new value (ANT_HOME and C:\apache-ant-1.6.1).

More on Ant and targets

Ant has evolved into a great system over the years and I recommend using it not just for Java projects, but also for .Net projects. Chances are, your current Ant targets end up creating the final JARs, shared libraries, DLLs (dynamic link libraries), executables, and support files inside the standard dist subdirectory. Regardless, once we finish installing Anthill, we will need to add a new target, Publish, to move the final product over to the Anthill directories (where it will be visible on the Web). So keep in mind that we will need to add a new target before all is said and done.

Java

Of course! Make sure that your box has the Java SDK installed. A small point but, still, it must be there. Make sure that JAVA_HOME is set and that the sdk bin directory is your path for javac (in my case, that is C:\j2sdk1.4.2\bin).

At this point, you have a basic Java build environment on your system. You can test whether your Ant build scripts will run: you need to check the CVS module containing your code, open a command window, go to the check-out directory, and run the ant command. I presume your Ant script follows the conventional name: build.xml.

Tomcat

I freely admit that I am a bit of a Tomcat junkie. We need Tomcat to run our system, so go ahead and download it. I use Tomcat 5.0, which is installed on XP as a service, but I start and stop from the command line. You can run Tomcat as a service, but if you run a nightly build on a dedicated machine, you should have an open window showing the output from the build system. It will make troubleshooting that much easier. But if you want to run as a service, feel free to do so.

In my case, running Tomcat means opening a command prompt and going to the Tomcat directory C:\Program Files\Apache Software Foundation\Tomcat 5.0\bin> and then running the program catalina.bat start.

If you want to run Tomcat in the debugger, you run catalina.bat debug start and type run at the command prompt.

Regardless of how you run Tomcat, you should now be able to see your running Tomcat by clicking on your localhost at http://mymachine:8080. You should see the standard Tomcat page (I won't show it). Go ahead and shut down Tomcat by using the command catalina.bat stop.

Anthill

We are now ready for the final component in our build system: the open source Web application named Anthill. A professional version of Anthill is available, but, for our immediate purposes, the vanilla open source Web application will do.

The download will be a gzipped (GNU zipped) tar file. So first, you must be able to unzip and untar the file. If you have a cross-mounted Unix box, you can do that there. If not, you may need to download the Cygwin bash shell, which contains both the gunzip and tar commands. Regardless, put all of the untarred files into a base directory; I use C:\anthill as my base directory. Within this directory you will now have a variety of subdirectories with names such as conf and publishDir.

One final step and we can start up the system. Tomcat needs the anthill.war Web application, which is in the dist directory (as befits an Ant-related Web application). So go to the dist directory and grab a copy of the anthill.war file. Paste this copy into your Tomcat webapps subdirectory. This will be:

C:\Program Files\Apache Software Foundation\Tomcat 5.0\webapps

Now restart Tomcat.

Bring up a browser and hit the Anthill page: http://yourmachine:8080/anthill/. You'll be requested to enter in the base directory (C:\anthill). Figure 3 illustrates the Anthill page.

Figure 3. Enter the base Ant directory. Click on thumbnail to view full-size image.

Finally, you have the base Anthill page up and running (Figure 4). I suggest cleaning up the page by removing all of the samples. Do not remove stoppedSchedule though. This is useful to have around.

Figure 4. The main build page. Click on thumbnail to view full-size image.

Set the general Anthill properties

On the main build page, click on Anthill Properties. This link will bring you to the default page shown in Figure 5.

Figure 5. Default Anthill Properties page. Click on thumbnail to view full-size image.

We need to change this page like so:

  1. We want to point to our newer version of Ant so set anthill.ant.home to ..\apache-ant-1.6.1\lib
  2. The anthill.mail.host and anthill.mail.from fields should reflect your mail server configuration
  3. Rename the title of your system using anthill.title.text—doing so makes your system a bit more professional
1 2 Page 1