easyb will support behavior tagging

Back in the Age of Aquarius, I wrote an article entitled “Use test categorization for agile builds” in which I attempted to delineate various types of tests and then went on to suggest how to categorize these various tests so as to get the most out of a build run (i.e make it fast and effective at providing meaningful feedback). Back then (and to an extent now) my concern was with test execution; that is, I like to categorize tests as fast and slow. Accordingly, I run the fast ones all the time, such as before I check-in (I prefer a fast build to be less than 3 minutes — longer and I get bored). What’s more, for the slow ones, I usually schedule a build (via Hudson, for instance) that runs them hourly, nightly, etc — it all depends on how slow they are, man!

As it turns out, unit-like tests are usually the fast ones and functional-like tests are the slow ones. This all makes sense as functional ones usually require a lot of test up, like seeding a database or firing up Selenium.

Categorizing tests by execution speed it helpful, but a bit broad in some cases. Thus, I often find myself grouping tests (or behaviors) by feature, behavior or goal. That is, there might be a macro-feature in development broken up by a series of stories (or tests) and I find it’s helpful to run those stories as a group in isolation for focus. Those behavior groups are then aggregated into iteration builds that correspondingly execute all targeted behaviors, for instance.

This exercise, combined with the influence of TestNG (which supported test groups long long ago) has lead to a new feature in easyb: tags. With the tags keyword, you can categorize a behavior and then exercise only those desired behaviors via the -tags command line option or via the tags attribute in easyb’s Ant task, for instance.

Thus, the following story is tagged as “23949314″ — this happens to be a Pivotal Tracker story ID and in the current iteration, there are 4 stories 23949314, 23949315, 23949316, 23949317.

tags "23949314"

scenario "AcmeFoo should support expired account deletions"
scenario "AcmeFoo should support child account deletions"

The beauty here is that when this particular story is run, its status is pending, thus providing some status as to a particular feature.

Going forward in this iteration, subsequent easyb stories are tagged with appropriate IDs (23949315, etc) and during this iteration, builds are run that exercise these particular tags in isolation. Of course, there are other builds that exercise all other existing stories (via a build pipeline) so as to perform a continuous regression.

The tags keyword is flexible too — you can add multiple tags by including the tag names in a List like so:

tags ["23949314", "functional"]

scenario "AcmeFoo should support expired account deletions"
scenario "AcmeFoo should support child account deletions"

In this case, this story is tagged as both 23949314 and functional — accordingly, a build can be configured to execute only functional behaviors and, of course, this one will be included. Conversely, if a build is run with a tag such as prototype, then this particular story won’t be executed.

This is an evolving feature in easyb. Right now, only story level tags are supported. Scenario tagging is planned. What’s more, tags are currently supported in the tip of easyb’s SVN repository — if you’re eager to start using them now, you’ll have to build your own version of easyb. Otherwise, a release is forthcoming, which will contain this feature. Can you dig it?

Looking to spin up Continuous Integration quickly? Check out www.ciinabox.com.