Featured White Papers

• Measuring Risk to Improve Java Software Quality
• Do You Really Get Classloaders?
• Pragmatic Continuous Delivery with Jenkins, Nexus, and LiveRebel
• Your Next Java Web App: Less XML, No Long Restarts, Fewer Hassles
• Coding with JRebel, Java Forever Changed
• 2012 Developer Productivity Report: Java Tools, Tech, Devs, and Data

Sponsored Links

• Graph databases are 1000x faster than relational.

  Learn More!

• Career opportunities at Harman- Careers That'll Rock Your Socks Off.

  Learn More!

• Developer-friendly Java PDF libraries by Qoppa Software

  Live Demos

• Stop blaming your app! Diagnose Java performance issues.

  Free tool >

• UrbanCode: Go From Continuous Integration to
  Continuous Delivery

  Learn More!

Sponsored Links

Optimize with a SATA RAID Storage Solution
Range of capacities as low as $1250 per TB. Ideal if you currently rely on servers/disks/JBODs

Continuous integration with Hudson

Open source CI server offers easy setup and configuration

By Nicholas Whitehead, JavaWorld.com, 12/18/08

Page 7 of 7

• Project name: I already named the project HeliosJMXTrunk, but you can change it here.
• Description: A free-form field where you can describe the build job.
• Discard old builds: Hudson will retain a history of prior builds unless you check this box.
• This build is parameterized: If you select this option, Hudson will allow you to provide a set of arbitrary parameters using name-value pairs that will be passed to the build process. Configured parameters will be set as environmental variables in the environment of the running build.
• Enable project-based security: Hudson supports a full security scheme that can force users to authenticate when accessing Hudson Web pages; it can also control which users can fiddle with which jobs. I have not configured security in this instance of Hudson.
• Disable build: If this is checked, builds for this job will not be executed until the option is disabled.
• Advanced options: I am not using either of these options, but when this box is selected, the following options are exposed in the interface:
  • Quiet period: Here you can configure a quiet period that occurs when the build is scheduled to run, but before it is actually executed.
  • Use custom workspace: By default, Hudson will create the job's workspace in ${jboss-home}/.hudson/jobs/[project name]. This option allows you to specify an alternate location.
• Source code management: The three options available by default are:
  • Subversion
  • CVS
  • None
  The None option belies my earlier assertion that a source code repository is a prerequisite, but I maintain that, in most cases, a repository of some sort is necessary to do anything useful with Hudson. Later in this article, I will address how Hudson plugins are installed. If you install a SCM-related plugin and restart Hudson, that new option will also appear in this list. For this article, I am using Subversion. When you select Subversion, a configuration section expands into the form. I will address this configuration separately in the next section (see "Subversion job configuration").
• Build after other projects are built: This option supports an assembly line -- job dependencies where one job depends on the output of another -- or possible scenarios where you simply want to group some related project builds together. When you select this, you will be provided with a field to enter the comma-separated names of the other projects after which this build should run.
• Poll SCM: This is the classic option for CI systems. When you select this option, you can specify a cron expression that defines how often Hudson should check your source repository for changes. If changes are found, a build will be executed. For example, the expression 0,15,30,45 * * * * will make Hudson check your repository for changes every 15 minutes. See the Quartz CronTrigger javadoc for additional details on this cron syntax.

• Build periodically: This option (also defined using a cron expression) simply instructs Hudson to build the project on a specific frequency regardless of changes in SCM. This may be helpful if you want to run some test cases where the target test environment is somehow modified periodically but the SCM is static with respect to this job.

• Add build step: Click this button to add a directive to run a build script. Your directive can be one of the following:

  • Execute shell
  • Execute Windows batch command
  • Invoke Ant (this is the option I will be using, and the details will be described below)
  • Invoke top-level Maven targets
• Archive the artifacts: When you select this option and specify the file and directory mask (Ant-style masks that can specify include and exclude), the artifacts matching the mask will be added to the Hudson artifact repository when the build is complete, keyed by job and build sequence number. All prior artifacts of successful builds can optionally be discarded to save disk space on your Hudson server.
• Record fingerprints of files to track usage: Using a similar Ant-style mask when you select this option, you can direct Hudson to maintain a fingerprint of generated artifacts that enables you to more easily track where else in the system these artifacts are being used.
• Publish javadoc: If your build script generates javadoc content, this option will direct Hudson to publish the content and expose it directly on the job's home page. The javadoc content for every single successful build can be retained, but by default only the latest is kept.
• Publish JUnit test result report: If your build script executes JUnit tests, this option directs Hudson to process the XML test result document and generate an ongoing report of each successive build, aggregating the results on a ongoing basis. The result is a report on the home page of the job that displays the historical trends of unit testing for the job over time.
• Aggregate downstream test results: In some cases, the elapsed time of a job's unit test suites is dramatically longer than that of the actual build itself. In these cases, you may elect to separate the build and test into different jobs so that the build completes relatively quickly, but the one or more test jobs implemented will be executed as soon as the build completes successfully. When you chose this option, Hudson is directed to aggregate the test results of all the post-build jobs and retroactively report them as test results for the primary build job.
• Build other projects: Related to the prior item, this option is used to implement one logical build and test process into two or more physical jobs that are executed serially. When this option is selected, you will be provided a field in which you can enter the comma-separated job names that you want to execute after the current job. This can optionally be done even if the current job execution determines the build to be unstable. (See the section on job states for more information about job stability.)
• E-mail notification: When you select this option, you can enter one or more whitespace-separated email addresses that will be sent Hudson job execution completion notifications. Events that will trigger an email include failed builds, unstable builds. An additional option here is to send a "special" email to the SCM committer that checked in the code that Hudson determines broke the build.

That's a lot of information to absorb! To get a better grasp of how all this works, you'll next take a look at an example build job, so you can see what these configuration options mean in practice.

Configuring a build job in Hudson: An example

The following is an example of an actual job configured in my Hudson server. The job builds a utility library I am writing called HeliosJMX. All the images in this section are clipped from the New Job screen outlined in the previous section.

In Figure 14, you can see the project name, description, and a discard policy that directs Hudson to keep the last five builds but discard any older ones.

Figure 14. Example job setup, Part 1

Figure 15 displays the job's Subversion setup. The URL for the Subversion repository on java.net is https://helios.dev.java.net/svn/helios/helios-jmx/trunk. The Local module directory property is an optional and additional subdirectory that will be created in the job's workspace.

The Use update checkbox is quite important. The fastest way for Hudson to prepare the workspace for a build execution is to simply refresh the directory from the Subversion repository. This works well in most cases and is quite fast. However, in some instances, source artifacts that have been deleted from the repository may linger when the workspace is updated. The alternative is to disable this option, in which case Hudson will purge the workspace and repopulate from the repository.

The last option assigns a source repository browser, such as FishEye or VisualSVN. Select the applicable browser if you have one of these products available and pointed at your source repository.

Figure 15. Example job setup, Part 2

In Figure 16, you can see the build trigger I selected. I want to poll Subversion every five minutes and build if there are changes.

Figure 16. Example job setup, Part 3

Figure 17 displays the setup of the Ant task I have defined to execute the build process. The configuration options are as follows:

• Ant version: The predefined Ant instance to be used when executing the build.
• Targets: A list of targets in the specified Ant script to be invoked. This can be left blank, in which case the script's default task will be executed.
• Build file: The location of the Ant script to be executed, relative to the effective workspace.
• Properties: Additional system property definitions that will be passed to the Ant script. I have used these properties to pass my Subversion credentials to the script, because my process includes a step that commits some changes back into the repository. Additionally, I have defined some properties that are configuration parameters for my unit tests.
• Java options: Java command-line options can be passed in here. In this case, I am passing the Ant -debug, as I was working on debugging a problem in the script and the option causes Ant to generate extra diagnostics in the log. Other common options are directives to specify the invoked JVM's initial and maximum heap sizes (-Xms and -Xmx). This is a reminder that a whole new JVM instance will be launched by Hudson to run your build script.

Figure 17. Example job setup, Part 4

Figure 18 displays the post-build actions I have defined:

• Archive artifacts: Directs Hudson to archive the built artifacts when the build is complete. This is an Ant-style file mask that specifies the directory relative to the effective workspace, file names, and extensions to be archived. The archived items will be accessible through the job's build instance home page. The advanced options also allow you to specify an exclude mask, and you have the option to delete all archived artifacts except those generated in the last successful build.
• Publish javadoc: Similar to the previous option, but applicable to any javadoc content that was generated by the build process.
• Publish JUnit test result report: Directs Hudson to acquire the JUnit XML results document in the defined location and aggregate it into the historical trend report.

Figure 18. Example job setup, Part 5

Figure 19 displays more post-build actions I have defined:

• Publish FindBugs analysis results: My build script executes FindBugs static code analysis on the source code associated with the job and generates a report of the findings. This option is made available when the Hudson FindBugs plugin is installed. It directs Hudson to retrieve the defined FindBugs XML results report and aggregate into the job's historical FindBugs trends exposed on the home page of the job. The advanced options for the FindBugs plugin allow you to determine the categories of FindBugs assertions that are reported and how they should influence the final determination that Hudson makes about the state of the job. (See the section entitled "Job states" for more on this.)

• E-mail notification: Defines a whitespace-delimited list of email addresses to which build failure notifications will be sent. When a job is perpetually unstable or breaking, the Send email for every unstable build option can be unchecked to prevent constant notifications concerning a known condition.

• Publish Cobertura coverage report: My build script uses Cobertura to instrument the generated classes with code coverage directives. When the JUnit tests run, Cobertura tracks the code coverage and generates a coverage report when the testing is complete. This option is made available when the Hudson Cobertura plugin is installed. It directs Hudson to retrieve the defined Cobertura XML coverage report and aggregate into the job's historical Cobertura trends exposed on the home page of the job. The section titled Coverage Metric Targets allows you to specify how determined levels of code coverage map influence the final determination that Hudson makes about the state of the job. (See the section on job states for more about this.)

Figure 19. Example job setup, Part 6

At this point, if you have defined a new job, it is time to run it and see how it works. Regardless of how you configured your build triggers, you can always request an ad hoc (on demand) build, unless you have Disable Build checked. When you are setting up a new build job, or debugging one like I was in the screenshots above, it does not make sense to wait for a build trigger. In the next section, I will show you how to request an ad hoc run of a build, and how the running job can be watched.

Running and watching a job

To run the first execution of your newly created job, navigate to the Hudson dashboard at http://localhost:8080/hudson. Figure 20 illustrates the brand new instance of Hudson I set up on the Ubuntu server.

Figure 20. The Hudson dashboard

In the center of the screen, you can see the HeliosJMXTrunk job I just defined. A few other notables items on this screen:

• The grey globe icon represents the state of the job. In this case, it is grey because the job has never been built.
• The table on the left labelled Build Queue will display jobs currently running or queued to run. The table labelled Build Executor Status reports the state of the allocated build threads. By default, Hudson allocates two build threads, which means that Hudson can execute up to two concurrent builds. To modify this number, you can click Manage Hudson and then Configure Executors. Be cautious with this number, as software builds tend to be fairly resource intensive; too many concurrent builds will put an unreasonable load on the Hudson server, which in turn may slow down all your jobs.
• Note the RSS feed icons. Hudson provides RSS feeds as another form of event notification, and there are feeds for the overall system and for each individual job that will report:
  • Successful builds
  • Unstable builds
  • Broken builds
  • SCM changes

To request a build, you can click on the build icon on the far right of the job listing table. Alternatively, you can click on the job name, which will navigate you to the job's home page, and click on the icon link titled Build Now.

Once a job is running, you will see the running job listed in the queue in the dashboard and on the job home page. Both of these views are illustrated in Figure 21.

Figure 21. Two different views of the running job

Typically, at some point you will want to watch the progress of the job by viewing the output as the job is running. To do this, navigate to the job home page and click Console Output. If the job is complete, this will display the static output that was generated by the build script; if the job is still running, Hudson constantly refreshes the content of the page so that you can see output as it occurs. This very useful feature is illustrated in Figure 22.

Figure 22. The live console display of the running job output

Once the build is complete, there are three places where the completed job will be displayed.

• You can see it on the Hudson dashboard, as in Figure 23.
• You can see it on the job home page, as in Figure 24.
• By clicking on the specific build link in the build history, you can navigate to the Build Home Page that Hudson created specifically for this build instance. This is illustrated in Figure 25.

Figure 23. The Hudson dashboard, showing the completed job

The significance of the symbols on the dashboard is outlined in the section titled "Job state," below; briefly, the yellow globe means that the build succeeded but is considered unstable. The little sun icon in the W column represents the "weather," which is sunny because the build succeeded and none of the post-processing plugins instructed Hudson to report anything worse. Also notable here is that the last build duration was 15 minutes. This is because there are unit tests in my test suite for some scheduling components that are deliberately long running. This job might be a good candidate for splitting the build and tests.

Figure 24. The job home page, displaying the completed job

The job home page contains some interesting items. Among the links on the left side are commands to configure the job (in order to modify what you defined in New Job), delete the job, and build the job. On the right side are links to the latest project reports and artifacts.

Figure 25. The build home page for the completed build

The build home page is specific to this one instance of the build. Note that Hudson allocates an internal build number, which can be useful for tracking distributed builds. This page also lists the revisions that were found in Subversion (none in this build) as well as three JUnit test cases that failed. These three failures are what caused Hudson to mark the build as unstable, meaning that the software built without errors but had unit testing failures. Figure 26 shows sections of the dashboard and project page after the next build, which encountered no unit test errors, resulting in a stable build.

Figure 26. The dashboard and job home page after build #2

I have now covered the steps necessary to configure a job, trigger a job, and request an ad hoc build. What's left is to briefly discuss the scheme that Hudson uses to display the state of a job.

Job states

In the screenshots displayed above, you will observe the two icons representing the current state of a job. Hudson uses two notions to present the overall condition of a job:

• Job state: Figure 27 outlines the symbols for the four possible states for the most recently executed build of a job:
  • Successful: The build completed and was considered stable.
  • Unstable: The build completed and was considered unstable.
  • Failed: The build failed.
  • Disabled: The job is disabled.

• Job stability: While a job may build to completion and generate the target artifacts without issue, Hudson will assign a stability score to the build (from 0-100) based on the post-processor tasks, implemented as plugins, that you have set up to implicitly evaluate stability. These can include unit tests (JUnit), coverage (Cobertura), and static code analysis (FindBugs). The higher the score, the more stable the build. Figure 28 outlines the symbols for the stability score ranges. Anything less than an 80 out of 100 will flag the build's job state as unstable.

Figure 27. Job states

Figure 28. Job stability

Build tracks

Hudson allows you to create multiple build tracks; depending on your software development process, you may want to create more than one build track per software project. A build track is a build job for a specific project or product that has a unique configuration. The factors that distinguish between build tracks for the same project may lie in the SCM branch from which the build job acquires source code; alternately, different tracks could be different sets of tasks that are executed for the same source. Among the influences that may direct how you create different build tracks for the same logical software project are:

• Source control branches: Many development teams maintain a series of branches in their source control system in order to cleanly support the concurrent development of multiple versions of their source code. In cases like these, it makes sense to create a separate build track in Hudson for each branch under development, as well as for some older inactive branches so that older versions of software are catalogued and can easily be recreated. You may have separate branches for the following processes:
  • Trunk builds: The most recent, active, and frequently changing source base.
  • Release builds: Separate branches dedicated to ongoing releases, where the source code has mostly stabilized but trunk development is continuing on changes for future releases.
  • Bug fix builds: Separate branches created so that a bug can be fixed in isolation from the mainline activity.
  • Experimental builds: Separate branches created in order to test experimental ideas on the source base that may not end up in the mainline of the code.
• Build job task categories: Builds for different source bases may require a different set or sequence of build tasks, depending on the current state of or activities on specific branches. For example, you may want to set up separate tracks for the following:
  • Simple build verification jobs where you are not interested in stability but simply want to regularly verify that the code still builds and to make the build artifacts available.
  • Build and stability verification builds that may run slightly less frequently, and that build the artifacts and execute a full testing suite on the source code and built artifacts.
  • Release candidate jobs that execute the full build and stability test suites and then complete a series of tasks to package the deliverables for distribution.
  • Full release jobs executed when a release candidate is ready for distribution and the packaged artifacts are automatically uploaded to a public repository, along with any other tasks required to complete a full release.
• Deliverable considerations: The exact same branch of code and build tasks may need to be built using slightly different tools. For example, you might want to release separate versions of your classes for Java 1.5 and Java 1.4.

Figure 29 represents a possible logical structure of source control branches and the related Hudson build tracks.

Figure 29. Source control branches and Hudson build tracks

Hudson makes it very easy to create a new job by copying an existing job. To do this, navigate to the Hudson dashboard and click the New Job link. Enter the name of the new job and then select Copy existing job. Note that as you start typing, Hudson will populate a list box of existing jobs that you can copy that match what you have typed. Then click OK, and your new job will be created. This process is illustrated in Figure 30.

Figure 30. Creating a new job as a copy of an existing job

Once you have created a new job in this way, it is effectively identical to the job you copied except for the job name, so you will want to modify the new copy by changing items. For instance, you may want to:

• Modify the source control target branch from which the workspace is populated.
• Modify the build script or build script targets that are executed by Hudson.
• Modify the system properties that are passed to your Ant script by Hudson.

Using views

Once you have created a few build tracks, you may find that your dashboard has become a bit disorganized with a long list of jobs. A convenient way to organize the dashboard is to create views. A dashboard view is a group of related jobs that you can define that is displayed in a separate tab on the dashboard. When creating views grouping related jobs, you will see that implementing a consistent job naming convention is beneficial.

To create a view, click on the small tab labeled + on the dashboard. On the new view page, enter the name of the new group and an optional description. Hudson will present you with a labeled checkbox for every job currently configured so you can select the specific jobs that you want to include in the view.

However, a better way to group jobs, in my opinion, is to click the checkbox labeled Use a regular expression to include jobs in the view and then provide a regular expression that will match the names of the jobs that you want to include. This is where a consistent naming convention comes in handy. You can configure views that group jobs together by general software project, or possibly by the type of build. I've taken the latter approach in Figure 31, where I have created a view named Release Builds that will include all jobs with the word release in their name.

Figure 31. Creating a new dashboard view

The added advantage of using the regular-expression-inclusion method is that, as new jobs are created, they will automatically be added to any views that the job name matches, whereas views that group specifically selected jobs will need to be updated manually.

Figure 32 displays a newly organized dashboard based on build type.

Figure 32. Dashboard views by build type

Hudson plugins

The term Hudson plugins can denote both libraries of existing functional extensions to Hudson and the method for providing extensibility for developers who want to add new functionality to Hudson. Some plugins may simply be useful additions to your build process, while others, such as SCM plugins that implement support for source control systems other than CVS and Subversion, may be necessary for you to use Hudson with your setup.

Many plugins are currently available, so I will not list them all here, but the general categories of plugins are:

• SCM: Plugins that implement Hudson support for source control systems other than CVS and Subversion.

• Triggers: Plugins that listen on events and trigger a build. For example, the URL Change Trigger will monitor a URL; when the content at that address changes, the trigger will execute a job.

• Build tools: Plugins that implement additional build tools, such as MSBuild and Rake. These are particularly useful if you would like to build non-Java software in Hudson.

• Build wrappers: Plugins that typically involve executing controlled events before and after the build process itself. For example, the VMware plugin will start a guest VM before the build and shut it down afterwards. This is useful for situations where you might need that guest VM to execute unit tests.

• Build notifiers: These plugins supply alternate ways of issuing notifications about job events -- via Twitter, IRC, Google Calendar events, and the like.

• Slave launchers and controllers: One very powerful feature of Hudson not addressed in this article is the ability to have slave Hudson instances that perform work on behalf of a master Hudson instance. There is currently only one plugin in this category at this time: the SSH Slaves plugin, which allows slaves to be managed over an SSH link.

• Build reports: A series of plugins that create useful reports based on some form of analysis of your source code or generated artifacts. For example, the Cobertura plugin aggregates ongoing coverage reports generated by your build scripts.

• External site integrations: Plugins that assist in integrating Hudson with other applications, such as Jira or Bugzilla.

• Artifact uploaders: Plugins that assist you in distributing built artifacts to some networked endpoint, such as the java.net file repository or an FTP server.

• Page decorators: Plugins that add some cosmetic or useful decoration to Hudson Web pages, like the Google Analytics plugin that adds Google tracking to all pages served by Hudson.

The Hudson plugin manager allows you to install new plugins and update existing ones on your Hudson server. The manager will connect to the online repository to retrieve a list of available plugins and plugin updates. If your Hudson server cannot connect to outside resources, you can download your desired plugins from the Hudson Website. Click on the plugins folder on the site and you will see a list of plugins available. The individual plugin files have the .hpi extension. Once you have downloaded the plugin, copy it to the plugins subdirectory in the Hudson home directory. (The Hudson home directory is called .hudson and will be in the home directory of the user running the Hudson server.) Once the file is copied, you will need to restart Hudson for the plugin to take effect.

To use the Hudson plugin manager, click the Manage Hudson link on the dashboard, and then Manage Plugins. The plugin manager is displayed in Figure 33, and it contains four tabs:

• Updates: A list of plugins for which Hudson has detected an available update. Each listed plugin can be selected to apply updates.
• Available: A list of plugins that are available for installation (and not currently installed). Each listed plugin can be selected to install.
• Install: A list of plugins already installed.
• Advanced: Allows you to configure an HTTP proxy through which Hudson can communicate with the online plugin repository. Additionally, there is an upload facility that can be used to install plugins that you have downloaded outside of Hudson.

Figure 33. The Hudson plugin manager

Once your desired plugins have been installed or updated, Hudson needs to be restarted for them to take effect. If you are using JBoss, note that you can effect this by touching hudson.war in the JBoss deploy directory into which you originally deployed. (touch is a utility that updates the timestamp of a file.) This causes JBoss to redeploy the Hudson server, which has the same effect as restarting it.

One important concept to understand regarding some types of build report plugins is that they do not necessarily generate the core report for you. Rather, they take care of the additional task of aggregating ongoing reports as they are generated; in some cases, they will reformat your generated reports into native Hudson integrated reports. For example, the Cobertura plugin requires that your build script implement the process of instrumenting your target test classes, execute unit tests, and generate the build-specific coverage report. Then the plugin updates the running historical trend of coverage reports so that you can visualize how your coverage percentages have varied over time. An example of these trend reports is illustrated in Figure 34.

Figure 34. Historical trends for unit tests, static code analysis, and code coverage

In addition, JUnit test results and FindBugs are examples of plugins that will create Hudson-native reports displayed on the job or build instance home pages. (The JUnit plugin is actually built in and need not be installed.) Figure 35 is an example of the built-in report generated by the FindBugs plugin, which is displayed on the build instance page.

Figure 35. FindBugs summary report for a build

You can implement your own plugin to provide virtually any type of Hudson extension you can think of. If you are interested in doing so, you can find references, documentation, and tutorials on the Hudson wiki (see the Resources section below).

In conclusion

This concludes my introduction to the Hudson continuous integration server. I think you will find it an excellent piece of software; thanks to its ease of installation and configuration, you can try it out and be up and running very quickly. Based on the volume of activity on the java.net site that hosts the Hudson development project, Hudson clearly has a lot of momentum. My cursory browse of the mailing lists indicates that people consistently and quickly receive responses to their inquiries, although I must add that in the time I have been using it, I have not encountered any issues that would prompt me to seek support. I hope you have enjoyed exploring Hudson -- check out the Resources section below for more articles, downloads, and related links.

About the author

Nicholas Whitehead is a senior technology architect at the Small Business Services division of ADP in Florham Park, N.J.

Read more about Tools & Methods in JavaWorld's Tools & Methods section.