The myth of code-centricity

Break through productivity barriers with a new software model

Standing on the shores of the software development industry today, we see an ocean of endless code. The average developer spends most of his or her time coding or supporting code production. Developers earn respect based primarily on their coding abilities and obtain jobs based on the languages they know; a super developer is really a super coder.

The two major productivity advances of the 1990s were both code-centric: Java, a new language to produce even more code, and the Internet, which required an abundance of code to implement. The industry's problems are rooted in code management; they include code quality (bugs), code completion time (schedules), code capacity (how much you can produce), obsolete code (legacies), and, among many others, complexity in the ever-growing bodies of code. In short, the industry is so code-centric that it sees most problems and opportunities as best resolved with more handwritten code or more efficient code-production activities, such as requirements, design, and testing.

Before we delve into the problem, consider one insight: Brilliant tactics cannot save bad strategy.

The software industry has been searching for solutions to the code productivity problem for decades, as shown by attention to software engineering lore, code production process, coding tools, and coding languages. Just as Fred Brooks predicted in his 1986 essay "No Silver Bullet: Essence and Accident in Software Engineering" (see also the sidebar at the end of this article, "A Dissenting Opinion from Fred Brooks"), nothing has helped in the range of an order of magnitude:

But as we look to the horizon of a decade hence, we see no silver bullet. There is no single development, in either technology or management technique, which by itself promises even one order-of-magnitude improvement in productivity, in reliability, in simplicity.

Guess what? We're focusing on the wrong problem. The real problem is not productivity at the code/developer level, but at the system/user level.

Code-centricity is a dangerous myth because it stifles productivity and prevents fundamental progress in the science of computation. If we don't use drastically better alternatives, the software industry runs the risk of becoming a dinosaur relic, replaced by up-and-coming industries such as biotechnology and electromechanical nanotechnology, or by new ones not yet on anyone's radar. Congruence -- how closely two things match -- is at least one way out of the tar pit of code-centricity.

Another myth is that only radical innovation can achieve order-of-magnitude advances. Not so. Slow but steady improvement over a long period of time can have the same effect. In fact, most improvement is by evolution rather than revolution. This article presents nothing revolutionary or high risk. Instead, it proposes modest evolutionary progress by using bits and pieces of today's technology, focused tightly in the right direction.

Software generations

We'll explore the problem by learning from the past. Software development productivity trends are best understood if traced through successive generations, starting in the early 1950s. A new generation arose approximately once per decade. Table 1 classifies each generation by its proven productivity.

Table 1. Software generations from the 1950s to the present
Generation languageDescriptionExamplesKey limitation
First (1GL)Machine code1101 0111 0011 0010It's hard to think in binary.
Second (2GL)Assembly language

SUB..PROC

.........CMP....AX, 97

.........JL.........DONE

Rather tedious for general use. The sample shown is from the Intel 8086 family of processors (comments not shown).
Third (3GL)Procedural languagesJava, C, Python, Perl, COBOL, BASIC, SmalltalkLow productivity; historic average of 10 lines of code (LOC) per day per developer for the first three generations.
Fourth (4GL)GUI tools with direct WYSIWYGDrag-and-drop GUI builders, Dreamweaver, PhotoShopCannot handle most domains well because, in most cases, product does not equal image.

Each successive generation increased productivity by an order of magnitude. What can we learn from these generations? Why did they increase productivity so dramatically? Because each generation allows higher expressiveness through closer tool congruence.

Note that I have classified generations by quantum increases in productivity, not by conventions such as language constructs. Thus the four generations you see above don't match those defined in conventional wisdom. This classification prevents false generations from fooling us into thinking we have advanced.

The problem of tool congruence

As mentioned earlier, congruence is a measure of how closely two things match. Expressiveness is a measure of how well a tool lets a user accomplish a task. We each have a mental model of anything we build, including software systems. Tool congruence is how closely a tool's external appearance and use matches that mental model.

Thus, the more congruent the mental model and visual image, the more expressive a tool is. The more expressive a tool is, the more it becomes transparent to the user, and therefore the more productive it is. A well-designed tool disappears altogether if it completely merges your mental and physical activity.

For example, suppose you need to write some narrative text. Any reasonable 4GL text editor will let you do that in WYSIWYG style. This is 100 percent congruence because your mental model of paragraphs and sentences is the same as what the tool shows.

But suppose you must write a complex SQL statement with multiple tables and fancy joins. Your mental model is that of several tables with certain fields joined to other fields. This is not what a SQL line shows. Thus, using a text editor to write SQL results in a classic impedance mismatch and has low congruence. In this example, you can achieve high congruence with a GUI tool that allows modeling the SQL statement by dragging tables around and connecting fields to define joins. But not all domains can achieve congruence.

Currently, the most heavily used tools are either 3GL (such as code editors for C, Perl, Python, and Java) or 4GL (such as Dreamweaver, Visual Basic, and JBuilder). Visual Basic and JBuilder are a mixture of 3GL and 4GL. For more than 10 years, we've been stuck with 4GLs; the march of software generations has come to a roadblock.

4GLs are easy if product equals image. For example, you may need to write text, HTML, or a GUI, so the tool shows the result as it's created. When product doesn't equal mental image, 4GLs are difficult or impossible to use because, for example, they can lack the logic. Here the product is code but your mental image can include many things, such as a web of objects with associations, groups, and states. Since a code editor shows an image of text, the tool has very low congruence and expressiveness. The inevitable result is that coding requires a high skill level, goes slowly, and has frequent bugs.

Expressiveness is stuck at coding today; we've hit a wall because product doesn't equal image. Monumental attempts have been made to improve coding congruence, with modeling (like UML), unit assembly (such as IDE widgets and data icons), and even synchronized UML models and code (TogetherJ). However, we still produce and manage code, code, and more code.

For further proof of this stagnation, can you, in two seconds, name the most popular software metric? Too easy: it's lines of code (LOC). Even though function points are far more accurate, LOC predominates. In fact, the industry has made the interesting discovery that, regardless of language, the average output per developer day is about 10 LOC. How ironic to know so precisely where our field is stuck.

The chemistry field in the late 1700s encountered the same problem; it was "full of facts and techniques but no way to make sense of them" (Galileo's Commandment, edited by Bolles, 1997, page 379). Antoine-Laurent Lavosier introduced a new way to view chemistry: he reformed the language, redefined central concepts such as "elements," and gave chemicals new names. He forced clearer thought and understanding by merely introducing new viewpoints and clarifying central tenants. We can do the same to plow through the 3GL and 4GL roadblock using today's technology.

Our task is made easier, however, since we know where to apply leverage: increase congruence. The problem of congruence is: How can we organize modern software so that its physical structure and construction is the same as its visual viewpoint?

Solving the tool congruence problem

The latest high productivity initiatives from Sun and Microsoft (the Java 2 Platform, Enterprise Edition (J2EE) and .Net) confirm that we have entered the age of highly reusable components running in container frameworks. These systems are configured not by code but by declarative knowledge (DK), such as XML or HTML, which declares what to do. Procedural knowledge (PK), such as code, tells how to do things. J2EE and similar frameworks separate DK from PK; Table 2 describes their key practices.

Table 2. Key high-productivity practices of frameworks like J2EE
PracticeDescriptionWhy so productive?
ComponentsSoftware units designed for reuse in a variety of contexts.Allows bodies of code to be loosely coupled and highly cohesive. Causes higher reuse, easier maintenance, and third-party code production. Higher reuse leads to higher quality, lower cost, and faster time to market.
Container frameworkSoftware system that achieves its domain-specific behavior through the components it contains and manages.Components no longer need to manage their own or other component's lifecycles and interactions. The system designer is also relieved of that task.
Standard servicesA set of widely needed services available to components. Usually used directly (component managed) but sometimes transparently (container managed).Many components need the same behaviors, called services. Standard services relieve the system designer from figuring out again and again how best to provide certain behavior.
Separation of DK from PKBehavioral knowledge is broken into two types: data that declares behavioral policy (DK) and procedures that define behavior (PK).This is the most important practice. Each PK unit (a component) is designed so that DK defines what varies in a particular reuse case. This lets you use components in a larger number of contexts than would be possible if you couldn't customize a component's behavior at system assembly time. Think of PK as "what you reuse" and DK as "reuse case policy."

J2EE uses DK for deployment description but excludes detailed bean configuration with DK (though a bean can do this using a reference). I recommend DK for components as well as system configuration, and view systems as large components.

Congruence with visual tools is not yet emphasized in comparatively young frameworks like J2EE and .Net. Exposure to component assembly tools and study of UI design principles shows that to obtain congruence, we must follow one simple strategy: use a mental model with which users already build systems. Such simplicity!

That mental model is assembling big things from little things and connecting them to make them run. We've all seen this done before. A city is basically buildings connected by roads and utilities. A house is constructed of many standardized parts (lumber, plywood, concrete, rebar, appliances, and so on) that are assembled with standardized connections (nails, screws, glue, wiring, and so on). Even a corporation is constructed of people, organized into groups, configured with training and policy, and connected with lines of authority and communication. For lack of a better phrase, I call this ubiquitous ability system assembly from parts and connections.

System assembly is inherently a visual process because we already think in spatial mechanics as we assemble or examine things. Thus, system assembly is easily shown and used in visual tools.

Solving the simpler problem

The problem now becomes much simpler: How can we organize modern software so that its physical structure and construction is one of system assembly? To borrow heavily from biology and electronics, we add only a few more key practices -- connections, anonymous collaboration, and hierarchical composition -- to the previous four. The new practices are outlined in Table 3.

Related:
1 2 3 Page 1
Page 1 of 3