Continuous integration with Hudson

Open source CI server offers easy setup and configuration

Continuous integration has become common practice for teams focused on ensuring code quality throughout the software development lifecycle. In this article, Nicholas Whitehead introduces Hudson, a popular open source CI server. Learn how to set up a Hudson server in your application development environment (examples are given for Windows XP with Tomcat 6 or Ubuntu Linux with JBoss AS), get an overview of the many configuration options Hudson provides, then implement an automated build, test, and reporting process for an example project. Level: Beginner

Continuous integration (CI) is a set of practices intended to ease and stabilize the process of creating software builds. CI assists development teams with the following challenges:

  • Software build automation: With CI, you can launch the build process of a software artifact at the push of a button, on a predefined schedule, or in response to a specified event. If you want to build a software artifact from source, your build process is not bound to a specific IDE, computer, or person.
  • Continuous automated build verification: A CI system can be configured to constantly execute builds as new or modified source code is checked in. This means that while a team of software developers periodically checks in new or modified code, the CI system continuously verifies that the build is not being broken by the new code. This reduces the need for developers to check with each other on changes to interdependent components.
  • Continuous automated build testing: An extension of build verification, this process ensures that new or modified code does not cause a suite of predefined tests on the built artifacts to fail. In both build verification and testing, failures can trigger notifications to interested parties, indicating that a build or some tests have failed.
  • Post-build procedure automation: The build lifecycle of a software artifact may also require additional tasks that can be automated once build verification and testing are complete, such as generating documentation, packaging the software, and deploying the artifacts to a running environment or to a software repository. In this way artifacts can quickly be made available to users.

To implement a CI server you need, at minimum, an accessible source code repository (and the source code in it), a set of build scripts and procedures, and a suite of tests to execute against the built artifacts. Figure 1 outlines the basic structure of a CI system.

Basic structure of a CI system
Figure 1. Basic structure of a CI system

The system components come into play in the following sequence:

  1. Developers check new and modified code into the source code repository.
  2. The CI server creates a dedicated workspace for each project. When a new build is requested or scheduled, the source is retrieved from the repository into this workspace, where the build is then executed.
  3. The CI server executes the build process on the newly created or refreshed workspace.
  4. Once the build completes, the CI server can optionally invoke the defined test suite on the new artifacts. If the build fails, registered individuals can be notified by email, instant messaging, or some other method.
  5. If the build is successful, the artifacts are packaged and transmitted to a deployment target (such as an application server) and/or stored as a new versioned artifact in a software repository. This repository can be part of the CI server, or could be an external repository, such as a file server or a software distribution site like Java.net or SourceForge. The source code repository and the artifact repository can be separate, and it is actually possible to use some CI servers without any formal source control system at all.
  6. CI servers usually have some sort of console where projects can be configured and debugged, and where requests can be issued for operations such as ad hoc immediate builds, report generation, or retrieval of built artifacts.

Hudson: A continuous integration server

Continuous integration has grown in popularity over the past several years and today you have quite a few CI servers to choose from, both commercial and free. I personally had used four CI servers before a colleague recommended that I look at Hudson. I was immediately impressed by it. While I initially assumed that Hudson was not well known, a survey at the Java Power Tools site shows it as the most widely used CI server among respondents, garnering (at time of this writing) 37.8 percent of all votes.

Supported SCMs

Hudson has integrated support for Subversion right out of the box, and only a small amount of configuration is required to integrate with CVS, assuming the CVS client is installed on the Hudson host. Several other source code management (SCM) solutions are supported in the form of Hudson plugins. At time of this writing, the following SCMs are supported:

  • Accurev
  • BitKeeper
  • ClearCase
  • Git
  • Mercurial
  • Perforce
  • StartTeam
  • Team Foundation Server
  • Visual SourceSafe
  • URL SCM (a special SCM plugin that allows the use of URLs for SCM)

In this article, I will be using Subversion and the source repository at Java.net, so you won't need to install any of these plugins. (As an aside, I know someone who is working on a MKS SourceIntegrity Hudson plugin. If you are interested in that, send me an email.)

Hudson is a free and open source product hosted at Java.net It was originally written by Kohsuke Kawaguchi, a staff engineer at Sun Microsystems, who announced its release on his blog in February of 2005. Hudson has since had approximately 154 releases.

Here are some of the reasons why I like Hudson, and why I would recommend it to you, barring any unusual requirements:

  • Of all the CI products I've used, it is by far the easiest to install and configure.
  • Its Web-based user interfaces are very friendly, intuitive, and responsive, in many cases providing immediate Ajax-enabled feedback on individual configuration fields.
  • Hudson is Java-based (which is useful if you're a Java developer) but is not limited to building Java-based software.
  • Hudson is cleanly componentized and offers a well-defined and documented extensibility API in the form of Hudson plugins. This has in turn led to a large library of Hudson plugins that extend the functionality of the server; these are freely available and installable from the Hudson console.

Installing Hudson: Windows XP or Ubuntu Linux

To use Hudson, you'll need an accessible and supported source control system (see the "Supported SCMs" sidebar for a listing), source that can be built into an artifact, and a working build script. Beyond that, all you really need to install and configure a working Hudson server is an installation of Java, version 1.5 or above, and the Hudson install file, which comes in the form of a Java EE Web archive (WAR). You can start the server very simply using the following command line:

C:\hudson> java -jar hudson.war

It is probably more common, however, to deploy Hudson onto a Java servlet container that is based on the Servlet 2.4 and JSP 2.0 specs, such as GlassFish, Tomcat, JBoss, or Jetty. In the next sections, I will walk you through two Hudson installation scenarios: one using Tomcat 6 on Windows XP, and another using JBoss 4.2.3 on Ubuntu Linux. (JBoss AS 5.0 was released after this article's submission date.)

Installing Hudson: Tomcat 6 and Windows XP

I will assume that you already have version 1.5 or higher of Java installed on your Windows XP machine. Following the steps below will install Tomcat 6.0.18 using the Windows Service Installer, so that Hudson starts immediately after Windows XP boots up and will run in the background even when no user is logged in. The download file for Tomcat is apache-tomcat-6.0.18.exe, which you should execute to begin the Tomcat install.

The Tomcat installation will prompt you to select install options. Be sure to select Custom options and then Service, as shown in Figure 2, so that Tomcat will run as a service.

Tomcat installation options
Figure 2. Tomcat installation options

Next, select a directory where you want to install Tomcat, as shown in Figure 3. I highly recommend that you pick a directory with no spaces. You can thank me later.

Selecting an installation directory
Figure 3. Selecting an installation directory

Now the installer will ask you which port you want to listen on. The default is port 8080, which is probably fine; just make sure that you do not have another application using that port. If you do, Tomcat will not start properly. You will also be asked to provide a Tomcat administrator username and password. All this is shown in Figure 4.

Selecting a listening port and administrative user name and password
Figure 4. Selecting a listening port and administrative user name and password

The installer will then ask you to provide the location of the Java JRE you have installed. As you can see in Figure 5, I used Sun Java 1.6.0_07.

Selecting a Java JRE to run Tomcat
Figure 5. Selecting a Java JRE to run Tomcat

Once you click Install, the installation should run to completion and the service will start running. You can make sure that Tomcat is operating correctly by pointing your Web browser to http://localhost:8080 (substituting the appropriate name or IP address for localhost if you aren't using a Web browser running on the computer where Tomcat is installed). The Web page displayed should look something like the screenshot in Figure 6.

Verifying Tomcat installation and operation
Figure 6. Verifying Tomcat installation and operation

Now, to install Hudson, copy the hudson.war file to the webapps subdirectory of your Tomcat installation directory. If you used the same install directory shown in Figure 3, this would be C:\Tomcat6\webapps. Tomcat will hot-deploy WAR files, but the easiest thing to do now is restart Tomcat. There are two ways to do this. The first is to open a DOS shell and enter the following commands:

 C:\Tomcat6>net stop Tomcat6
    C:\Tomcat6>net start Tomcat6

The second option is to open the Services applet. This applet can be found in the Administrative Tools group in the Control Panel, which can be located by clicking the Start button on the Windows toolbar, then selecting Settings and then Control Panel. In the Services applet, locate the service named Apache Tomcat and then click the Restart button. This is illustrated in Figure 7.

The Services applet
Figure 7. The Services applet

Hudson should now be installed. You can verify this by pointing your Web browser to http://localhost:8080/hudson. The main Hudson screen is shown in Figure 8.

The Hudson start page
Figure 8. The Hudson start page

That's all there is to it! If you're comfortable with an application development environment based on Windows XP and Tomcat, you're all set. If you prefer a system running JBoss and Ubuntu Linux, read on.

Installing Hudson: JBoss 4.2.3 on Ubuntu Linux 8.04 (Hardy Heron)

To install Sun Java 1.6 on Ubuntu, open a shell and execute the following command:

 sudo apt-get install sun-java6-jdk

When issuing a sudo command, you will be prompted to enter your password.

Note that there are several ways to install JBoss; in the technique outlined here, you'll create a dedicated jboss user. This is considered a best practice, and is preferable to installing JBoss in your own home directory. The procedure outlined here has been condensed from a useful description at the Ubuntu forums.

First, you need to download the JBoss 4.2.3.GA package. Look for the file named jboss-4.2.3.GA.zip.

Next, you will need to create a user, a home directory, and a group, all named jboss. The group is a convenience not explored in this article; it will allow you to extend JBoss privileges to other users on your Ubuntu server.

Listing 1 shows the commented commands to create the jboss home directory, user, and group, and then install the JBoss server. Some commands are prefixed with sudo because they are root-privileged commands.

Listing 1. Creating the jboss account and installing the server

echo Create the jboss group
sudo groupadd jboss
echo Create the jboss user, define bash as the user's default shell and /home/jboss as the home directory
echo and make the user jboss part of the group jboss
sudo useradd -s /bin/bash -d /home/jboss -m -g jboss jboss
echo Copy the jboss-4.2.3.GA file to /home/jboss or download directly into that directory
sudo mv jboss-4.2.3.GA /home/jboss
echo Change the owner of the file to jboss
sudo chown jboss:jboss /home/jboss/jboss-4.2.3.GA
echo Log into the jboss account
sudo su jboss
echo Go to the jboss home directory
cd ~
echo Unzip the file jboss-4.2.3.GA
unzip jboss-4.2.3.GA
echo Create a symbolic link "jboss" for "jboss-4.2.3.GA".
echo This allows you to change JBoss versions with minimal changes
ln -s jboss-4.2.3.GA jboss

If the unzip command is not already installed, enter the following command (while logged in as a sudo-enabled user) to install it:

Sudo apt-get install unzip

The JBoss server is now basically installed. You could start the server using the following command:

/home/jboss/jboss/bin/run.sh

In this example, however, you will instead install an auto startup script so that the service starts up automatically when the host starts. The JBoss download comes with three different int.d scripts, but each needs to be tweaked; you can download the jboss-init.sh script, which will enable the automatic start and stop of the server. Then run the commands shown in Listing 2.

Listing 2. Installing the autostart script for JBoss

echo Move the jboss-init.sh file to /etc/init.d/ and rename it to jboss
sudo mv jboss-init.sh /etc/init.d/jboss
echo Change the owner of the /etc/init.d/jboss file to root
sudo chown root:root /etc/init.d/jboss
echo Make the /etc/init.d/jboss file executable
sudo chmod ug+x /etc/init.d/jboss
echo Activate the /etc/init.d/jboss file in the rc.d lifecycle process.
sudo update-rc.d jboss defaults

Now you can start the JBoss server as a background process (which will not terminate when you log out) with the following command:

sudo /etc/init.d/jboss start

To verify that JBoss is running (it may take up to a few minutes to start), point your browser to http://localhost:8080/jmx-console. Figure 9 shows the JBoss JMX console that should appear.

The JBoss JMX console
Figure 9. The JBoss JMX console

Deploying Hudson is simple and is done by copying the hudson.war file to /home/jboss/jboss/server/default/deploy. This is best done using the jboss user to ensure that the JBoss server has the privileges to read the file. Hudson should deploy in a few seconds, and then you can validate the deployment by pointing your browser to http://localhost:8080/hudson. The page that displays should be the same as the one shown in Figure 8.

Setting up a build in Hudson

You've now stepped through the procedures for installing Hudson in Tomcat running on Windows XP or JBoss running on Ubuntu Linux. Most of these steps involved installing the application servers and performing administrative tasks in the operating system; I think you will agree that the Hudson installation itself was trivial. Next, you will see how to configure some required basics in Hudson so that you can set up a build job.

Prerequisites

Before you set up a build job in Hudson, the following conditions must be met:

  • You must have an accessible source code repository.
  • The repository must contain the source code you want to build.
  • The repository must contain build scripts that will build the source. These are usually Ant or Maven scripts, although Hudson also supports simple shell scripts, NAnt, and MSBuild.

In this article, you will be using Ant version 1.7 to build the example source.

Configuring Hudson

You do need to attend to some minor configuration in Hudson before you can start building software. To begin, you must tell Hudson where your Java JDK and Ant installations are. On the main Hudson page at http://localhost:8080/Hudson, click the Manage Hudson link. On the page that comes up next, click Configure System.

On the system configuration page, note the first item in the list, which is Home directory. This is where Hudson performs all its work and stores all its configuration. I will come back to this directory later in this article.

In order to configure both a JDK and an Ant instance, click the Add button under each section. A set of fields will expand out under the configuration section. These two sections are illustrated in Figure 10.

Configuring Ant and JDK
Figure 10. Configuring Ant and JDK

In each case, you assign an arbitrary name identifying the JDK and Ant installation. You need to assign these because multiple instances can be configured, depending on the needs of your project; for instance, you might have Java JDK instances for 1.4, 1.6, and 1.7, as well as Ant 1.6 and 1.7. When you configure your project build, you will select the instance you want to use. Note that Hudson immediately notifies you that the blank fields representing JAVA_HOME and ANT_HOME are not valid directories. When a valid directory is entered, these warning will go away. This is a simple example of Hudson's user-friendliness: it gives you immediate feedback rather than waiting for an error to occur when the directory cannot be found.

Because I already installed a Java 1.6 JDK, I can configure the JDK installation as follows:

  • name: JDK 1.6.0_07
  • JAVA_HOME:
    • Windows: C:\jdk1.6.0_07
    • Linux: /usr/lib/jvm/java-6-sun

If you're using Windows, you can simply download an Ant installation and unzip it into the target directory. On Ubuntu, you will use the following commands to install Ant:

sudo apt-get install ant
sudo apt-get install ant-optional
ant -version
Apache Ant version 1.7.0 compiled on August 29 2007

The Ubuntu installation will install Ant in /usr/share/ant.

Figure 11 displays the Ubuntu configuration for the JDK and Ant. When I tabbed out of the fields specifying the locations of the installs, the warnings disappeared. I also added a second instance of Ant, which is badly configured. In the figure, you can see that, if you enter a valid directory path that isn't actually the correct directory for the Ant instance, Hudson warns you that your entry does not appear to be correct. It will do this for both the JDK and Ant. I subsequently deleted this erroneous Ant entry, leaving only the valid one.

JDK and Ant configured under Ubuntu
Figure 11. JDK and Ant configured under Ubuntu

Further down in the same form, you will see a warning that Hudson cannot locate the CVS executable. If you are not using CVS, you can safely ignore this.

The last important entry on this page is the SMTP configuration, which allows Hudson to email you notifications of important events like failed builds. If your SMTP server requires authentication, you will need to use the advanced options. I use my Google Apps hosted domain, but if you have a valid Gmail account, you can use the Google SMTP server and your Gmail address. Figure 12 shows my Hudson server configuration for mail.

Configuring Hudson's SMTP forwarder using Google's mail server
Figure 12. Configuring Hudson's SMTP forwarder using Google's mail server

The basic system-level configuration is now complete. You're ready to move on to configuring a specific build job.

Configuring a build job in Hudson

On the main Hudson page at http://localhost:8080/Hudson, click the New Job link. Figure 13 illustrates the screen that will come up next; here, you assign a name to your new build job. There are several options for new job types, but for the scope of this article, you will focus on the type titled Build a free-style software project. The one other type I frequently use is Copy existing job; this comes in handy when I want to create a new job copied from an existing job.

New job name
Figure 13. New job name

In this case, I am going to set up a build job for my HeliosJMX project, stored in Subversion at java.net. I am going to build the source, which is hot and fresh right out of the trunk, so I will call the build job HeliosJMXTrunk. Once I enter that name, I click OK.

The next form that comes up is the bulk of the new job configuration, and it is quite long. The details of this configuration are listed below. On the right of each option on this form, there is a little question-mark (?) icon. Do not hesitate to click on it for a more detailed explanation of any configuration option. This will not drag you off to another page, losing your work in process; rather, it simply inserts the help text directly into the page, so using the Help is negligibly intrusive. Virtually all of this configuration is identical under both Windows and Linux.

  • Project name: I already named the project HeliosJMXTrunk, but you can change it here.
  • Description: A free-form field where you can describe the build job.
  • Discard old builds: Hudson will retain a history of prior builds unless you check this box.
  • This build is parameterized: If you select this option, Hudson will allow you to provide a set of arbitrary parameters using name-value pairs that will be passed to the build process. Configured parameters will be set as environmental variables in the environment of the running build.
  • Enable project-based security: Hudson supports a full security scheme that can force users to authenticate when accessing Hudson Web pages; it can also control which users can fiddle with which jobs. I have not configured security in this instance of Hudson.
  • Disable build: If this is checked, builds for this job will not be executed until the option is disabled.
  • Advanced options: I am not using either of these options, but when this box is selected, the following options are exposed in the interface:
    • Quiet period: Here you can configure a quiet period that occurs when the build is scheduled to run, but before it is actually executed.
    • Use custom workspace: By default, Hudson will create the job's workspace in ${jboss-home}/.hudson/jobs/[project name]. This option allows you to specify an alternate location.
  • Source code management: The three options available by default are:
    • Subversion
    • CVS
    • None
    The None option belies my earlier assertion that a source code repository is a prerequisite, but I maintain that, in most cases, a repository of some sort is necessary to do anything useful with Hudson. Later in this article, I will address how Hudson plugins are installed. If you install a SCM-related plugin and restart Hudson, that new option will also appear in this list. For this article, I am using Subversion. When you select Subversion, a configuration section expands into the form. I will address this configuration separately in the next section (see "Subversion job configuration").
  • Build after other projects are built: This option supports an assembly line -- job dependencies where one job depends on the output of another -- or possible scenarios where you simply want to group some related project builds together. When you select this, you will be provided with a field to enter the comma-separated names of the other projects after which this build should run.
  • Poll SCM: This is the classic option for CI systems. When you select this option, you can specify a cron expression that defines how often Hudson should check your source repository for changes. If changes are found, a build will be executed. For example, the expression 0,15,30,45 * * * * will make Hudson check your repository for changes every 15 minutes. See the Quartz CronTrigger javadoc for additional details on this cron syntax.
  • Build periodically: This option (also defined using a cron expression) simply instructs Hudson to build the project on a specific frequency regardless of changes in SCM. This may be helpful if you want to run some test cases where the target test environment is somehow modified periodically but the SCM is static with respect to this job.

  • Add build step: Click this button to add a directive to run a build script. Your directive can be one of the following:

    • Execute shell
    • Execute Windows batch command
    • Invoke Ant (this is the option I will be using, and the details will be described below)
    • Invoke top-level Maven targets
  • Archive the artifacts: When you select this option and specify the file and directory mask (Ant-style masks that can specify include and exclude), the artifacts matching the mask will be added to the Hudson artifact repository when the build is complete, keyed by job and build sequence number. All prior artifacts of successful builds can optionally be discarded to save disk space on your Hudson server.
  • Record fingerprints of files to track usage: Using a similar Ant-style mask when you select this option, you can direct Hudson to maintain a fingerprint of generated artifacts that enables you to more easily track where else in the system these artifacts are being used.
  • Publish javadoc: If your build script generates javadoc content, this option will direct Hudson to publish the content and expose it directly on the job's home page. The javadoc content for every single successful build can be retained, but by default only the latest is kept.
  • Publish JUnit test result report: If your build script executes JUnit tests, this option directs Hudson to process the XML test result document and generate an ongoing report of each successive build, aggregating the results on a ongoing basis. The result is a report on the home page of the job that displays the historical trends of unit testing for the job over time.
  • Aggregate downstream test results: In some cases, the elapsed time of a job's unit test suites is dramatically longer than that of the actual build itself. In these cases, you may elect to separate the build and test into different jobs so that the build completes relatively quickly, but the one or more test jobs implemented will be executed as soon as the build completes successfully. When you chose this option, Hudson is directed to aggregate the test results of all the post-build jobs and retroactively report them as test results for the primary build job.
  • Build other projects: Related to the prior item, this option is used to implement one logical build and test process into two or more physical jobs that are executed serially. When this option is selected, you will be provided a field in which you can enter the comma-separated job names that you want to execute after the current job. This can optionally be done even if the current job execution determines the build to be unstable. (See the section on job states for more information about job stability.)
  • E-mail notification: When you select this option, you can enter one or more whitespace-separated email addresses that will be sent Hudson job execution completion notifications. Events that will trigger an email include failed builds, unstable builds. An additional option here is to send a "special" email to the SCM committer that checked in the code that Hudson determines broke the build.

That's a lot of information to absorb! To get a better grasp of how all this works, you'll next take a look at an example build job, so you can see what these configuration options mean in practice.

Configuring a build job in Hudson: An example

The following is an example of an actual job configured in my Hudson server. The job builds a utility library I am writing called HeliosJMX. All the images in this section are clipped from the New Job screen outlined in the previous section.

In Figure 14, you can see the project name, description, and a discard policy that directs Hudson to keep the last five builds but discard any older ones.

Example job setup, Part 1
Figure 14. Example job setup, Part 1

Figure 15 displays the job's Subversion setup. The URL for the Subversion repository on java.net is https://helios.dev.java.net/svn/helios/helios-jmx/trunk. The Local module directory property is an optional and additional subdirectory that will be created in the job's workspace.

The Use update checkbox is quite important. The fastest way for Hudson to prepare the workspace for a build execution is to simply refresh the directory from the Subversion repository. This works well in most cases and is quite fast. However, in some instances, source artifacts that have been deleted from the repository may linger when the workspace is updated. The alternative is to disable this option, in which case Hudson will purge the workspace and repopulate from the repository.

The last option assigns a source repository browser, such as FishEye or VisualSVN. Select the applicable browser if you have one of these products available and pointed at your source repository.

Example job setup, Part 2
Figure 15. Example job setup, Part 2

In Figure 16, you can see the build trigger I selected. I want to poll Subversion every five minutes and build if there are changes.

Example job setup, Part 3
Figure 16. Example job setup, Part 3

Figure 17 displays the setup of the Ant task I have defined to execute the build process. The configuration options are as follows:

  • Ant version: The predefined Ant instance to be used when executing the build.
  • Targets: A list of targets in the specified Ant script to be invoked. This can be left blank, in which case the script's default task will be executed.
  • Build file: The location of the Ant script to be executed, relative to the effective workspace.
  • Properties: Additional system property definitions that will be passed to the Ant script. I have used these properties to pass my Subversion credentials to the script, because my process includes a step that commits some changes back into the repository. Additionally, I have defined some properties that are configuration parameters for my unit tests.
  • Java options: Java command-line options can be passed in here. In this case, I am passing the Ant -debug, as I was working on debugging a problem in the script and the option causes Ant to generate extra diagnostics in the log. Other common options are directives to specify the invoked JVM's initial and maximum heap sizes (-Xms and -Xmx). This is a reminder that a whole new JVM instance will be launched by Hudson to run your build script.
Example job setup, Part 4
Figure 17. Example job setup, Part 4

Figure 18 displays the post-build actions I have defined:

  • Archive artifacts: Directs Hudson to archive the built artifacts when the build is complete. This is an Ant-style file mask that specifies the directory relative to the effective workspace, file names, and extensions to be archived. The archived items will be accessible through the job's build instance home page. The advanced options also allow you to specify an exclude mask, and you have the option to delete all archived artifacts except those generated in the last successful build.
  • Publish javadoc: Similar to the previous option, but applicable to any javadoc content that was generated by the build process.
  • Publish JUnit test result report: Directs Hudson to acquire the JUnit XML results document in the defined location and aggregate it into the historical trend report.
Example job setup, Part 5
Figure 18. Example job setup, Part 5

Figure 19 displays more post-build actions I have defined:

  • Publish FindBugs analysis results: My build script executes FindBugs static code analysis on the source code associated with the job and generates a report of the findings. This option is made available when the Hudson FindBugs plugin is installed. It directs Hudson to retrieve the defined FindBugs XML results report and aggregate into the job's historical FindBugs trends exposed on the home page of the job. The advanced options for the FindBugs plugin allow you to determine the categories of FindBugs assertions that are reported and how they should influence the final determination that Hudson makes about the state of the job. (See the section entitled "Job states" for more on this.)

  • E-mail notification: Defines a whitespace-delimited list of email addresses to which build failure notifications will be sent. When a job is perpetually unstable or breaking, the Send email for every unstable build option can be unchecked to prevent constant notifications concerning a known condition.

  • Publish Cobertura coverage report: My build script uses Cobertura to instrument the generated classes with code coverage directives. When the JUnit tests run, Cobertura tracks the code coverage and generates a coverage report when the testing is complete. This option is made available when the Hudson Cobertura plugin is installed. It directs Hudson to retrieve the defined Cobertura XML coverage report and aggregate into the job's historical Cobertura trends exposed on the home page of the job. The section titled Coverage Metric Targets allows you to specify how determined levels of code coverage map influence the final determination that Hudson makes about the state of the job. (See the section on job states for more about this.)

Example job setup, Part 6
Figure 19. Example job setup, Part 6

At this point, if you have defined a new job, it is time to run it and see how it works. Regardless of how you configured your build triggers, you can always request an ad hoc (on demand) build, unless you have Disable Build checked. When you are setting up a new build job, or debugging one like I was in the screenshots above, it does not make sense to wait for a build trigger. In the next section, I will show you how to request an ad hoc run of a build, and how the running job can be watched.

Running and watching a job

To run the first execution of your newly created job, navigate to the Hudson dashboard at http://localhost:8080/hudson. Figure 20 illustrates the brand new instance of Hudson I set up on the Ubuntu server.

The Hudson dashboard
Figure 20. The Hudson dashboard

In the center of the screen, you can see the HeliosJMXTrunk job I just defined. A few other notables items on this screen:

  • The grey globe icon represents the state of the job. In this case, it is grey because the job has never been built.
  • The table on the left labelled Build Queue will display jobs currently running or queued to run. The table labelled Build Executor Status reports the state of the allocated build threads. By default, Hudson allocates two build threads, which means that Hudson can execute up to two concurrent builds. To modify this number, you can click Manage Hudson and then Configure Executors. Be cautious with this number, as software builds tend to be fairly resource intensive; too many concurrent builds will put an unreasonable load on the Hudson server, which in turn may slow down all your jobs.
  • Note the RSS feed icons. Hudson provides RSS feeds as another form of event notification, and there are feeds for the overall system and for each individual job that will report:
    • Successful builds
    • Unstable builds
    • Broken builds
    • SCM changes

To request a build, you can click on the build icon on the far right of the job listing table. Alternatively, you can click on the job name, which will navigate you to the job's home page, and click on the icon link titled Build Now.

Once a job is running, you will see the running job listed in the queue in the dashboard and on the job home page. Both of these views are illustrated in Figure 21.

Two different views of the running job
Figure 21. Two different views of the running job

Typically, at some point you will want to watch the progress of the job by viewing the output as the job is running. To do this, navigate to the job home page and click Console Output. If the job is complete, this will display the static output that was generated by the build script; if the job is still running, Hudson constantly refreshes the content of the page so that you can see output as it occurs. This very useful feature is illustrated in Figure 22.

The live console display of the running job output
Figure 22. The live console display of the running job output

Once the build is complete, there are three places where the completed job will be displayed.

  • You can see it on the Hudson dashboard, as in Figure 23.
  • You can see it on the job home page, as in Figure 24.
  • By clicking on the specific build link in the build history, you can navigate to the Build Home Page that Hudson created specifically for this build instance. This is illustrated in Figure 25.
The Hudson dashboard, showing the completed job
Figure 23. The Hudson dashboard, showing the completed job

The significance of the symbols on the dashboard is outlined in the section titled "Job state," below; briefly, the yellow globe means that the build succeeded but is considered unstable. The little sun icon in the W column represents the "weather," which is sunny because the build succeeded and none of the post-processing plugins instructed Hudson to report anything worse. Also notable here is that the last build duration was 15 minutes. This is because there are unit tests in my test suite for some scheduling components that are deliberately long running. This job might be a good candidate for splitting the build and tests.

The job home page, displaying the completed job
Figure 24. The job home page, displaying the completed job

The job home page contains some interesting items. Among the links on the left side are commands to configure the job (in order to modify what you defined in New Job), delete the job, and build the job. On the right side are links to the latest project reports and artifacts.

 

Figure 25. The build home page for the completed build

The build home page is specific to this one instance of the build. Note that Hudson allocates an internal build number, which can be useful for tracking distributed builds. This page also lists the revisions that were found in Subversion (none in this build) as well as three JUnit test cases that failed. These three failures are what caused Hudson to mark the build as unstable, meaning that the software built without errors but had unit testing failures. Figure 26 shows sections of the dashboard and project page after the next build, which encountered no unit test errors, resulting in a stable build.

The dashboard and job home page after build #2
Figure 26. The dashboard and job home page after build #2

I have now covered the steps necessary to configure a job, trigger a job, and request an ad hoc build. What's left is to briefly discuss the scheme that Hudson uses to display the state of a job.

Job states

In the screenshots displayed above, you will observe the two icons representing the current state of a job. Hudson uses two notions to present the overall condition of a job:

  • Job state: Figure 27 outlines the symbols for the four possible states for the most recently executed build of a job:
    • Successful: The build completed and was considered stable.
    • Unstable: The build completed and was considered unstable.
    • Failed: The build failed.
    • Disabled: The job is disabled.
  • Job stability: While a job may build to completion and generate the target artifacts without issue, Hudson will assign a stability score to the build (from 0-100) based on the post-processor tasks, implemented as plugins, that you have set up to implicitly evaluate stability. These can include unit tests (JUnit), coverage (Cobertura), and static code analysis (FindBugs). The higher the score, the more stable the build. Figure 28 outlines the symbols for the stability score ranges. Anything less than an 80 out of 100 will flag the build's job state as unstable.
Job states
Figure 27. Job states
Job stability
Figure 28. Job stability

Build tracks

Hudson allows you to create multiple build tracks; depending on your software development process, you may want to create more than one build track per software project. A build track is a build job for a specific project or product that has a unique configuration. The factors that distinguish between build tracks for the same project may lie in the SCM branch from which the build job acquires source code; alternately, different tracks could be different sets of tasks that are executed for the same source. Among the influences that may direct how you create different build tracks for the same logical software project are:

  • Source control branches: Many development teams maintain a series of branches in their source control system in order to cleanly support the concurrent development of multiple versions of their source code. In cases like these, it makes sense to create a separate build track in Hudson for each branch under development, as well as for some older inactive branches so that older versions of software are catalogued and can easily be recreated. You may have separate branches for the following processes:
    • Trunk builds: The most recent, active, and frequently changing source base.
    • Release builds: Separate branches dedicated to ongoing releases, where the source code has mostly stabilized but trunk development is continuing on changes for future releases.
    • Bug fix builds: Separate branches created so that a bug can be fixed in isolation from the mainline activity.
    • Experimental builds: Separate branches created in order to test experimental ideas on the source base that may not end up in the mainline of the code.
  • Build job task categories: Builds for different source bases may require a different set or sequence of build tasks, depending on the current state of or activities on specific branches. For example, you may want to set up separate tracks for the following:
    • Simple build verification jobs where you are not interested in stability but simply want to regularly verify that the code still builds and to make the build artifacts available.
    • Build and stability verification builds that may run slightly less frequently, and that build the artifacts and execute a full testing suite on the source code and built artifacts.
    • Release candidate jobs that execute the full build and stability test suites and then complete a series of tasks to package the deliverables for distribution.
    • Full release jobs executed when a release candidate is ready for distribution and the packaged artifacts are automatically uploaded to a public repository, along with any other tasks required to complete a full release.
  • Deliverable considerations: The exact same branch of code and build tasks may need to be built using slightly different tools. For example, you might want to release separate versions of your classes for Java 1.5 and Java 1.4.

Figure 29 represents a possible logical structure of source control branches and the related Hudson build tracks.

Source control branches and Hudson build tracks
Figure 29. Source control branches and Hudson build tracks

Hudson makes it very easy to create a new job by copying an existing job. To do this, navigate to the Hudson dashboard and click the New Job link. Enter the name of the new job and then select Copy existing job. Note that as you start typing, Hudson will populate a list box of existing jobs that you can copy that match what you have typed. Then click OK, and your new job will be created. This process is illustrated in Figure 30.

Creating a new job as a copy of an existing job
Figure 30. Creating a new job as a copy of an existing job

Once you have created a new job in this way, it is effectively identical to the job you copied except for the job name, so you will want to modify the new copy by changing items. For instance, you may want to:

  • Modify the source control target branch from which the workspace is populated.
  • Modify the build script or build script targets that are executed by Hudson.
  • Modify the system properties that are passed to your Ant script by Hudson.

Using views

Once you have created a few build tracks, you may find that your dashboard has become a bit disorganized with a long list of jobs. A convenient way to organize the dashboard is to create views. A dashboard view is a group of related jobs that you can define that is displayed in a separate tab on the dashboard. When creating views grouping related jobs, you will see that implementing a consistent job naming convention is beneficial.

To create a view, click on the small tab labeled + on the dashboard. On the new view page, enter the name of the new group and an optional description. Hudson will present you with a labeled checkbox for every job currently configured so you can select the specific jobs that you want to include in the view.

However, a better way to group jobs, in my opinion, is to click the checkbox labeled Use a regular expression to include jobs in the view and then provide a regular expression that will match the names of the jobs that you want to include. This is where a consistent naming convention comes in handy. You can configure views that group jobs together by general software project, or possibly by the type of build. I've taken the latter approach in Figure 31, where I have created a view named Release Builds that will include all jobs with the word release in their name.

Creating a new dashboard view
Figure 31. Creating a new dashboard view

The added advantage of using the regular-expression-inclusion method is that, as new jobs are created, they will automatically be added to any views that the job name matches, whereas views that group specifically selected jobs will need to be updated manually.

Figure 32 displays a newly organized dashboard based on build type.

Dashboard views by build type
Figure 32. Dashboard views by build type

Hudson plugins

The term Hudson plugins can denote both libraries of existing functional extensions to Hudson and the method for providing extensibility for developers who want to add new functionality to Hudson. Some plugins may simply be useful additions to your build process, while others, such as SCM plugins that implement support for source control systems other than CVS and Subversion, may be necessary for you to use Hudson with your setup.

Many plugins are currently available, so I will not list them all here, but the general categories of plugins are:

  • SCM: Plugins that implement Hudson support for source control systems other than CVS and Subversion.
  • Triggers: Plugins that listen on events and trigger a build. For example, the URL Change Trigger will monitor a URL; when the content at that address changes, the trigger will execute a job.

  • Build tools: Plugins that implement additional build tools, such as MSBuild and Rake. These are particularly useful if you would like to build non-Java software in Hudson.

  • Build wrappers: Plugins that typically involve executing controlled events before and after the build process itself. For example, the VMware plugin will start a guest VM before the build and shut it down afterwards. This is useful for situations where you might need that guest VM to execute unit tests.

  • Build notifiers: These plugins supply alternate ways of issuing notifications about job events -- via Twitter, IRC, Google Calendar events, and the like.

  • Slave launchers and controllers: One very powerful feature of Hudson not addressed in this article is the ability to have slave Hudson instances that perform work on behalf of a master Hudson instance. There is currently only one plugin in this category at this time: the SSH Slaves plugin, which allows slaves to be managed over an SSH link.

  • Build reports: A series of plugins that create useful reports based on some form of analysis of your source code or generated artifacts. For example, the Cobertura plugin aggregates ongoing coverage reports generated by your build scripts.

  • External site integrations: Plugins that assist in integrating Hudson with other applications, such as Jira or Bugzilla.

  • Artifact uploaders: Plugins that assist you in distributing built artifacts to some networked endpoint, such as the java.net file repository or an FTP server.

  • Page decorators: Plugins that add some cosmetic or useful decoration to Hudson Web pages, like the Google Analytics plugin that adds Google tracking to all pages served by Hudson.

The Hudson plugin manager allows you to install new plugins and update existing ones on your Hudson server. The manager will connect to the online repository to retrieve a list of available plugins and plugin updates. If your Hudson server cannot connect to outside resources, you can download your desired plugins from the Hudson Website. Click on the plugins folder on the site and you will see a list of plugins available. The individual plugin files have the .hpi extension. Once you have downloaded the plugin, copy it to the plugins subdirectory in the Hudson home directory. (The Hudson home directory is called .hudson and will be in the home directory of the user running the Hudson server.) Once the file is copied, you will need to restart Hudson for the plugin to take effect.

To use the Hudson plugin manager, click the Manage Hudson link on the dashboard, and then Manage Plugins. The plugin manager is displayed in Figure 33, and it contains four tabs:

  • Updates: A list of plugins for which Hudson has detected an available update. Each listed plugin can be selected to apply updates.
  • Available: A list of plugins that are available for installation (and not currently installed). Each listed plugin can be selected to install.
  • Install: A list of plugins already installed.
  • Advanced: Allows you to configure an HTTP proxy through which Hudson can communicate with the online plugin repository. Additionally, there is an upload facility that can be used to install plugins that you have downloaded outside of Hudson.
The Hudson plugin manager
Figure 33. The Hudson plugin manager

Once your desired plugins have been installed or updated, Hudson needs to be restarted for them to take effect. If you are using JBoss, note that you can effect this by touching hudson.war in the JBoss deploy directory into which you originally deployed. (touch is a utility that updates the timestamp of a file.) This causes JBoss to redeploy the Hudson server, which has the same effect as restarting it.

One important concept to understand regarding some types of build report plugins is that they do not necessarily generate the core report for you. Rather, they take care of the additional task of aggregating ongoing reports as they are generated; in some cases, they will reformat your generated reports into native Hudson integrated reports. For example, the Cobertura plugin requires that your build script implement the process of instrumenting your target test classes, execute unit tests, and generate the build-specific coverage report. Then the plugin updates the running historical trend of coverage reports so that you can visualize how your coverage percentages have varied over time. An example of these trend reports is illustrated in Figure 34.

Historical trends for unit tests, static code analysis, and code coverage
Figure 34. Historical trends for unit tests, static code analysis, and code coverage

In addition, JUnit test results and FindBugs are examples of plugins that will create Hudson-native reports displayed on the job or build instance home pages. (The JUnit plugin is actually built in and need not be installed.) Figure 35 is an example of the built-in report generated by the FindBugs plugin, which is displayed on the build instance page.

FindBugs summary report for a build
Figure 35. FindBugs summary report for a build

You can implement your own plugin to provide virtually any type of Hudson extension you can think of. If you are interested in doing so, you can find references, documentation, and tutorials on the Hudson wiki (see the Resources section below).

In conclusion

This concludes my introduction to the Hudson continuous integration server. I think you will find it an excellent piece of software; thanks to its ease of installation and configuration, you can try it out and be up and running very quickly. Based on the volume of activity on the java.net site that hosts the Hudson development project, Hudson clearly has a lot of momentum. My cursory browse of the mailing lists indicates that people consistently and quickly receive responses to their inquiries, although I must add that in the time I have been using it, I have not encountered any issues that would prompt me to seek support. I hope you have enjoyed exploring Hudson -- check out the Resources section below for more articles, downloads, and related links.

Nicholas Whitehead is a senior technology architect at the Small Business Services division of ADP in Florham Park, N.J.

Learn more about this topic

Downloads

Learn more

More from JavaWorld

Join the discussion
Be the first to comment on this article. Our Commenting Policies
See more