I was happy to accept Packt Publishing [1]'s invitation to review the recently released (July 2013) book TestNG Beginner's Guide [2] because I welcomed the opportunity to learn more about TestNG [3]. I have read about and played a little bit with TestNG before, but the vast majority of my Java unit testing experience has been with JUnit [4].
Author Varun Menon [5] opens TestNG Beginner's Guide with the following statement in the Preface, "Currently, TestNG is the most widely used testing framework in the software industry." This is definitely not what I've found in my experience where JUnit has been used far more than TestNG in the different Java projects that I've worked on. The Preface lists four things needed for getting most use out of TestNG Beginner's Guide: Java JDK [6], Eclipse [7], Linux [8] or Windows [9], "basic knowledge of Java and testing."
Chapter 1: Getting Started
The first chapter [10] of TestNG Beginner's Guide opens with a brief introduction to testing and automated testing. The chapter then introduces TestNG [3] and provides a brief history and overview of the features of that open source unit testing framework. The chapter describes how to acquire TestNG via download [11] (testng-6.8.jar in the book) and explains five ways it can be run: command line, Eclipse plugin [12], IntelliJ IDEA plugin [13], Ant [14], and Maven [15]. The chapter spends multiple pages and several screen snapshots demonstrating configuration of TestNG within Eclipse [16] via the plugin update site http://beust.com/eclipse [17]. It demonstrates creating an Eclipse project, associating the TestNG library with the Eclipse project, writing a TestNG-based test class, and executing the TestNG test class in Eclipse. There are numerous screen snapshots in this Equinox-heavy portion of the chapter. Similar instructions on using TestNG with Eclipse can be found in the TestNG Eclipse documentation [12]. The TestNG documentation [18] includes a section on running TestNG from the command line [19].
Chapter 2: Understanding testng.xml
The second chapter [20] of TestNG Beginner's Guide focuses on the testng.xml file [21], which it describes as "a configuration file for TestNG" that is "used to define test suites and tests in TestNG" and is "used to pass parameters to test methods." The chapter focuses on using testng.xml to specify test suites. It covers use of TestNG within Eclipse and from the command line. This chapter contains some of the concepts central to using TestNG. The examples shown are Eclipse-oriented, but XML snippets are shown as well, which is useful for those not using Eclipse. As testng.xml is a core part of TestNG, it is not surprising that Chapter 2 is a core part of TestNG Beginner's Guide.
Chapter 3: Annotations
The third chapter [22] of TestNG Beginner's Guide, like the second chapter, focuses on a core part of TestNG. In this case, it's on TestNG's annotations [23]. The chapter introduces Java annotations [24] and standard Java annotations (@Override [25], @Deprecated [26], and @SuppressWarnings [27]) before covering TestNG's specific annotations in a table with annotation name and brief description.
Chapter 3 of TestNG Beginner's Guide examines TestNG's annotations in more detail after the initial listing of them in the table with annotations names and descriptions. This more detailed examination includes code uses of these annotations. Annotations covered in this chapter include the @Test [28] annotation (including specification of timeouts [29] and expected exceptions [30]), several of the wide variety of "before" and "after" TestNG annotations (such as @BeforeClass [31] and @AfterMethod [32]), @Optional [33], @Parameters [34], and @DataProvider [35]. This, along with Chapter 2, is another "core" chapter of the book because annotations are core to TestNG. The third chapter also includes additional references to testng.xml.
Chapter 4: Groups
The fourth chapter [36] of TestNG Beginner's Guide covers TestNG Groups [37]. Coverage of test groups covers grouping of tests, running those groups of tests, specifying tests as parts of more than one group, using regular expressions to specify which groups to run, and grouping test groups within groups [38] (metagroups). As with many of the other chapters, this chapter includes numerous screen snapshots showing Eclipse used with the different concepts, but code snippets are also included.
Chapter 5: Dependencies
TestNG Beginner's Guide's fifth chapter [39] is on dependencies [40], a TestNG mechanism that allows tests to be executed in a particular order via expression of dependencies between tests and other tests or groups of tests. Screen snapshots of Eclipse and code listings are used to illustrate creating a dependency between one test on another, between one test on multiple other tests, between one test and a test in another class in the same inheritance hierarchy, and between a test and a group. The chapter also covers use of regular expressions with test dependencies and using XML [41] to specify test dependencies.
Chapter 6: The Factory Annotation
The sixth chapter [42] of TestNG Beginner's Guide introduces TestNG factories [43]. The chapter explains the use of factories to "define and create tests dynamically at runtime." Through more screen snapshots of Eclipse and mode code listings, the chapter introduces the @Factory [44] annotation and shows how it is used. This chapter also introduces the @DataProvider [35] annotation and demonstrates using these two annotations together. The chapter also includes explanation of how @Factory and @DataProvider differ [45] and when each is most appropriate.
Chapter 7: Parallelism
The seventh chapter [46] of TestNG Beginner's Guide covers parallelism [47], or running tests with multiple threads. The chapter includes a brief section describing two reasons why being able to run tests in parallel can be advantageous.
Chapter 8: Using Build Tools
The eighth chapter [48] of TestNG Beginner's Guide begins by outlining the "advantages of automating a build" and highlights Ant [49], Maven [50], and Gradle [51] as the "major build tools that are being used by the software industry when it comes to Java or Java related languages." The rest of the chapter focuses on incorporating TestNG with Ant and with Maven using TestNG's custom Ant task [14] and Maven plug-ins [15].
Chapter 9: Logging and Reports
Chapter 9 [52] of TestNG Beginner's Guide introduces TestNG support [53] for reporting and logging with an introductory paragraph on how reporting and logging are generally beneficial. The chapter introduces the interfaces ITestListener [54] and IReporter [55] and very briefly differentiates between these and where each is more appropriate. Like several other chapters, the ninth chapter mixes screen snapshots of Eclipse with code listings to demonstrate how to write a custom logger implementing the ITestListener [56] interface, how to write a custom reporter implementing the IReporter [57] interface, using TestNG-provided XML and HTML reports [58], and generating a JUnit HTML report [59] with Ant. This chapter also introduces ReportNG [60], which is described on its website [61] as "a simple HTML reporting plug-in for the TestNG unit-testing framework" that "is intended as a replacement for the default TestNG HTML report." This chapter also introduces Reporty-ng [62], formerly known [63] as TestNG-XSLT.
Chapter 10: Creating a Test Suite through Code
The tenth chapter of TestNG Beginner's Guide covers "how to configure and run [a] test suite through code" to help "in configuring and running tests at runtime." In short, this chapter covers how to run TestNG tests programmatically [64] rather than via specification in a testng.xml file. The chapter introduces and describes "the API classes" XmlSuite [65], XmlTest [66], XmlClass [67], XmlPackage [68], XmlDependencies [69], and a few more. The chapter adapts examples that were used earlier in the book to demonstrate using testng.xml for configuration but instead specify configuration in the code itself.
Chapter 11: Migrating from JUnit
The eleventh chapter [70] of TestNG Beginner's Guide was one that I most looked forward to as I scanned the table of contents [2] of the book. I find documentation on migration of one product (or version) to another product (or version) useful for two reasons. First, if I know the product or version being migrated from, I can better understand how the product or version being migrated to implements the same functionality. It helps me to learn the new product or version. Second, if I decide to adopt the new product or version, the migration guide is helpful in transitioning to that new product or version.
The section of the TestNG documentation [18] called Migrating from JUnit [71] states, "TestNG can automatically recognize and run JUnit tests, so you can use TestNG as a runner for all your existing tests and write new tests using TestNG." TestNG Beginner's Guide begins Chapter 11 with coverage of executing JUnit tests in TestNG. The chapter continues with a demonstration of running JUnit and TestNG tests together via testng.xml specification and then demonstrates "running JUnit tests along with TestNG [tests] through Ant."
Chapter 11 also covers true migration of JUnit-based unit tests to TestNG-based unit tests (as opposed to simply running JUnit-based tests in TestNG). This section includes an extremely handy table that compares the feature of TestNG to use compared to the feature of JUnit 3 and JUnit 4 to fulfill the same use case. This table is one of my favorite features of TestNG Beginner's Guide. The table validated my own internal mapping that I had been creating in my mind of TestNG concepts to JUnit concepts. The chapter also discusses how assert statements [72] are handled differently in JUnit versus TestNG and this reminded me of why I prefer Hamcrest [73]-based assertions with their high readability over straight JUnit assertions in which I must remember which of the two is an expected value and which is the actual value.
Chapter 12: Unit and Functional Testing
The final chapter [74] of TestNG Beginner's Guide focuses on using TestNG for unit and functional testing. The chapter provides basic definitions of unit testing and functional testing before examining each approach in more detail via examples.
The unit testing section of Chapter 12 provides a table of assertions supported by TestNG and demonstrates use of an assertion in a code sample. That section also focuses on mocking with TestNG. Although this chapter specifically mentions jMock [75], Mockito [76], EasyMock [77], PowerMock [78], and JMockit [79] as a "few of the mocking utilities," the chapter's focus is on jMock and Mockito.
After contrasting functional testing against unit testing, Chapter 12 includes a "list of functional testing automation tools that are available in the market" including Selenium [80], Sikuli [81], and SilkTest [82]. Selenium is the product in the functional test coverage that gets the same focus that jMock and Mockito got in the unit test coverage.
The final paragraph of the Summary of Chapter 12 is also the summary paragraph for the entire book.
Overall Positives
- For Beginners - As its title TestNG For Beginners correctly advertises, this book is aimed at beginners. I found my experience with JUnit to make it very easy to understand the concepts covered in this book, but I think most Java developers, even with less JUnit experience, would find this book highly approachable.
- Screen snapshots and Images - TestNG Beginner's Guide features numerous screen snapshots that can be helpful in learning how to use TestNG, particularly in conjunction with the Eclipse plugin. The one downside is that many of the images are Eclipse-specific. This is a positive for Eclipse users, but may be a bit of a negative for users of other IDEs.
- Code Listings - TestNG Beginner's Guide includes numerous code examples and listings. Because they are listed for nearly every covered topic, users of IDEs other than Eclipse should still be able to use this book.
Overall Negatives
- Editing and Polish - One of the weaknesses of several (not all) of the Packt titles I have reviewed has been occasional convoluted sentence structure and weird grammar and incorrect spelling. This particular book has some strange grammar in a couple places as well as several spelling errors. Examples of spelling errors include @SupressWarnings (only one "p") and "TestNg" rather than "TestNG" on the same page 52. The good news is that these are mostly just minor distractions or annoyances and rarely take away from understanding of the meaning.
- For Beginners - This is an advantage for new Java developers and developers new to unit testing, but is a bit of a disadvantage for more experienced developers (especially with JUnit). The book does a sound job of covering the most important features of TestNG, but there is very little coverage of why these features are useful. The simplicity and clarity of the examples comes at the price of the examples not providing very realistic ideas of how these TestNG features might be used in real code.
- Eclipse-centric - The book provides enough code listings to largely overcome this limitation and this is not a limitation, of course, for Eclipse users.
Other Observations
The title says it all: TestNG Beginner's Guide is for those new to TestNG.
As already mentioned this book is targeted most closely at Eclipse users. That stated, I think developers with at least some Java experience will find it easy to pick up the concepts and even implement the code listings in their favorite IDE even when that IDE is not Eclipse. Complete code examples are typically provided and most of the many Eclipse screen snapshots are showing reported results which will be shown on the command line or in other IDEs.
The ordering of the chapters is not the same order I'd have chosen, but this is not a big deal. For the most part, one could read the chapters in the order one prefers or even skip chapters on subjects that are not of interest or which the reader is already familiar. The chapters rely slightly on each other, but for the most part could be read fairly independently of one another. There are references to earlier parts of the book in several chapters, but these references are often not necessary to understand the newly covered topic.
Most of the material covered in TestNG Beginner's Guide is covered online in TestNG documentation [18] and in blog post. I have linked to specific sections or pages of these resources several times in my review of this book. Cédric Beust's blog [83] is another obvious source of information on TestNG. The TestNG site includes the More TestNG Reading [84] page with links to numerous additional online TestNG resources. Although there is little in TestNG Beginner's Guide that is not available online, this book provides organization for the introductory material and can provide context that is especially helpful when accessing seemingly unrelated blog postings.
TestNG Beginner's Guide is close to 250 pages of content plus more pages for table of contents, index, and so forth. Much of the space on these pages consists of code listings and screen snapshots, so the overall prose content is well below 250 pages. The examples are very simple, making it easy to focus on the TestNG feature being demonstrated, but also meaning that more subtle understanding of when certain features would be useful might be missed, especially by those not familiar with unit testing.
Conclusion
TestNG Beginner's Guide is squarely aimed at developers new to TestNG. Its twelve chapters cover the basics of TestNG and are approachable for anyone with some basic Java ability, but are especially easy to understand with some Java and JUnit experience. The book is probably too introductory for advanced TestNG users, but its title suggests that this is intentional. The only caveat I'd offer to a person new to TestNG who is trying to decide whether to obtain this book is to consider whether the book helps fill a need that is not covered adequately by the broad TestNG documentation [18] or blog posts such as those [85] by mkyong [86]. I personally find some advantages to learning from books in conjunction with online resources, but I understand that this is a matter of taste.