Face it -- three years ago, the writing on the wall should have been visible to anyone who cared to look.
Test for fun and profit: Read the whole series!
While many of the pundits and analysts saw Java as a fine language for building dinky little applets and little else, those who were actually struggling to build robust, distributed, object-oriented applications in heterogeneous operating environments in languages like C++ knew that it had the potential to be much more. In fact, many of us willingly struggled with the obscurities (and even the bugs) of the early versions of Java in a attempt to plumb its depths and prove its worth in the enterprise.
Over the last eighteen months or so, I've noticed a pattern emerge. The types of projects that I've come across have all shared several interesting characteristics.
First, projects based on Java technologies have moved from small, prove-to-me-it-can-be-done efforts on the fringe of the corporate environment to a more central position within the overall IS or IT space.
Second, projects based on Java technologies now find real support and funding easily. Gone (forever, I hope) are the days of all-you-get-to-build-it-is-5,000-bucks.
Third, customers expect the resulting applications to exhibit the same level of reliability and robustness as applications written in any other language on any other platform.
This last characteristic is of particular interest to me because it validates the first two. In my experience, there is a direct relationship between how core an application is to central business processes (not to mention how much funding management is willing to provide) and how reliable the final result must be.
Therefore, if Java is going to continue its advance into the heart of the business world, Java developers must continue their relentless march toward software of higher quality.
One way to quantify such an admittedly abstract concept as software quality is to measure related concrete phenomena. An easy method is to count the number of software defects, better known as bugs.
There are two basic approaches we can take to the task of building applications that minimize the number of intrinsic defects. The first approach tries to prevent defects from being introduced and errors from occurring. The second approach tries to detect application defects after the application components are built.
The two approaches complement each other. Each brings benefits to the table, and both deserve a place in the spotlight. The first approach would ba a suitable topic for discussion in a software engineering discussion forum; since this column is a vehicle for Java technology exploration, tips and techniques, and an occasional framework or two, I will focus on the second approach, which tends to be more hands-on.
With these thoughts in mind, this month I'd like to look at testing in general and Java platform-related testing in particular. Next month, I'll begin to build a framework for testing your Java code.
What is testing?
Let me begin by presenting the overall picture into which testing falls. Most software applications are complex and abstract. The requirements that guide their implementation are often unclear and even contradictory. Consequently, building applications that do what they are supposed to do -- and that continue to do so as changes are made -- is not as easy as it might seem from the outside. In addition, human architects, designers, and developers often make mistakes. In either case, the results are defects.
By testing an application, we hope to uncover the defects that arise from these sources and many others. These defects include typographical errors, logic errors, errors introduced due to insufficient understanding of the problem domain, and errors introduced due to poor programming practice. The process of uncovering these defects is made more difficult due to the nature of the software defects themselves: they don't occur where you expect to find them. Indeed, the likelihood of a defect existing in a particular piece of code is inversely proportional to the probability that a program path through that code will be executed.
Consequently, someone who is testing a program for defects must have a different mindset than someone putting it through everyday use. Remember this: when we test, we execute the functions of an application with the intent of exposing defects. A good test is one that is likely to uncover a flaw that was previously unknown.
Java's quality advantages over other languages and platforms
Before we go any further, it's worth pointing out the features inherent in the Java language and Java platform that improve our chances of building quality software. Most of these features remove entire classes of defects from consideration.
Since Java compiles applications before they run, checking for errors as it does so, the compiler catches most typographical errors before they can cause runtime errors -- a major improvement over interpreted languages. Similarly, static type checking catches invalid assignments and method parameters during compilation rather than during program execution.
Additionally, if you've migrated to the Java programming language from a language like C++, one of the first differences you probably noticed was all of the details you no longer had to think about while developing applications. You can thankfully forget about object deletion (which, if missed, would eventually lead to your application running out of memory) and a whole host of errors relating to pointer arithmetic and array bounds (both of which can lead to hard-to-find memory-corruption problems).
Java is not magic, however. Even though Java goes a long way toward bullet-proofing your applications, there is still room for defects to creep into the finished product. These defects tend to arise for one of the following reasons:
First, a programming language, by itself, cannot insulate you from defects arising from a poor understanding of the requirements, nor can it prevent many errors related to logic (typing
< rather than
> in a comparison, for example).
Second, Java can't prevent you from accidentally replacing a working version of a program module with a previous nonworking version. This class of defects actually includes a number of related source-code control problems (for example, Java can't prevent some klutz from editing your source code and breaking it).
Finally, Java's nonsupport of programming contracts (à la Eiffel) makes it possible for subclasses to ignore the published semantics of a superclass, thereby introducing defects when the subclass is incorporated into a framework.
For these reasons and more, Java programs -- just like their non-Java brethren -- require testing.
In the testing vernacular there are a number of terms that you'll frequently encounter.
- White-box testing
- White-box testing is a design strategy. It treats a software component like a transparent box into which test designers can peek while designing a test. Test designers can take advantage of their knowledge of the component's implementation to design tests that will cover all possible paths of execution through the component.
- Black-box testing
- Black-box testing is another design strategy. It treats a software component like an opaque box. Test designers seek to test how well the component conforms to the published requirements for the component, rather than concerning themselves with the implementation.
- Unit and component testing
- Unit and component tests seek to test the building blocks of an application, typically in isolation from the application's other units and components. Units are generally the atomic elements of an application (single classes, for instance), while components may be composed of one or more units.
- Integration testing
- Integration tests assess the integration and communication between the components and systems that compose an application.
- System testing
- System testing seeks to uncover defects that are properties of the entire system or application rather than of its individual components. This category includes defects due to resource utilization, performance, security, and recovery.
- Scenario testing
- Scenario testing aims to test an application against a realistic set of common uses, perhaps as embodied by the business scenarios and use cases used to drive the design process.
- Regression testing
- Regression testing refers to the practice of testing a modified version of a component or application in order to ensure that prior features are still intact. A regression test suite typically includes a set of tests that fully exercise the features of a component or application.
- Beta testing
- Beta testing consists of distributing a prerelease version of a product to a subset of its intended audience for the purpose of obtaining real-world feedback. When done properly, it's a formal process with well-defined feedback channels. Distributing prerelease software to every taker on the Internet is called marketing.
I hope this month's column has given you an overview of the important role testing plays in the software development process. All real applications, even those written in Java, need to be tested, both during development and before the final product ships. If you keep this fact in mind while you design and build your Java applications, and implement some of the strategies I'll cover in the next several months, your applications will benefit, your clients will be happier, and you will, I hope, reap great financial reward.
Next month we'll more closely examine the requirements for conducting tests and will begin to develop a simple testing language tailored for testing Java-based components.
Learn more about this topic
- "Runnability testing of Java Programs," Roger Hayes and Manoochehr Ghiassi (JavaWorld, August 1998) -- explains how to apply and adapt standard quality assurance techniques to efficiently assure the runnability of your programs
- Reliable Software Technologies hosts the comp.software.testing FAQ, a good source for testing-related information
- The comp.software.testing newsgroup features discussions on testing related topics and provides an excellent source of answers and ideasnews://comp.software.testing/