"Agile is treating the symptoms, not the disease"

The above quote was tossed off by Billy Hollis at the patterns&practices Summit
this week in Redmond. I passed the quote out to the Twitter masses, along with my
+1, and predictably, the comments started coming in shortly thereafter. Rather than
limit the thoughts to the 120 or so characters that Twitter limits us to, I thought
this subject deserved some greater expansion.

But before I do, let me try (badly) to paraphrase the lightning talk that Billy gave
here, which sets context for the discussion:

• Keeping track of all the stuff Microsoft is releasing is hard work: LINQ, EF, Silverlight,
  ASP.NET MVC, Enterprise Library, Azure, Prism, Sparkle, MEF, WCF, WF, WPF, InfoCard,
  CardSpace, the list goes on and on, and frankly, nobody (and I mean nobody) can track
  it all.
• Microsoft released all this stuff because they were chasing the "enterprise"
  part of the developer/business curve, as opposed to the "long tail" part
  of the curve that they used to chase down. They did this because they believed that
  this was good business practice—like banks, "enterprises are where the money
  is". (If you're not familiar with this curve, imagine a graph with a single curve
  asymptotically reaching for both axes, where Y is the number of developers on the
  project, and X is the number of projects. What you get is a curve of a few high-developer-population
  projects on the left, to a large number of projects with just 1 or 2 developers. This
  right-hand portion of the curve is known as "the long tail" of the software
  industry.)
• A lot of software written back in the 90's was written by 1 or 2 guys working for
  just a few months to slam something out and see if it was useful. What chances do
  those kinds of projects have today? What tools would you use to build them?
• The problem is the complexity of the tools we have available to us today preclude
  that kind of software development.
• Agile doesn't solve this problem—the agile movement suggests that we have to create
  story cards, we have to build unit tests, we have to have a continuous integration
  server, we have to have standup meetings every day, .... In short, particularly among
  the agile evangelists (by which we really mean zealots), if you aren't doing
  a full agile process, you are simply failing. (If this is true, how on earth did
  all those thousands of applications written in FoxPro or Access ever manage to succeed?
  –-Me) At one point, an agilist said point-blank, "If you don't do agile,
  what happens when your project reaches a thousand users?" As Billy put it, "Think
  about that for a second: This agile guy is threatening us with success."
• Agile is for managing complexity. What we need is to recognize that there is a place
  for outright simplicity instead.

By the way, let me say this out loud: if you have not heard Billy Hollis speak, you
should. Even if you're a Java or Ruby developer, you should listen to what he has
to say. He's been developing software for a long time, has seen a lot of these technology-industry
trends come and go, and even if you disagree with him, you need to listen to him.

Let me rephrase Billy's talk this way:

Where is this decade's Access?

It may seem like a snarky and trolling question, but think about it for a moment:
for a decade or so, I was brought into project after project that was designed to
essentially rebuild/rearchitect the Access database created by one of the department's
more tech-savvy employees into something that could scale beyond just the department.

(Actually, in about half of them, the goal wasn't even to scale it up, it was
just to put it on the web. It was only in the subsequent meetings and discussions
that the issues of scale came up, and if my memory is accurate, I was the one who
raised those issues, not the customer. I wonder now, looking back at it, if that was
pure gold-plating on my part.)

Others, including many people I care about (Rod Paddock, Markus Eggers, Ken Levy,
Cathi Gero, for starters) made a healthy living off of building "line of business"
applications in FoxPro, which Microsoft has now officially shut down. For those who
did Office applications, Visual Basic for Applications has now been officially deprecated
in favor of VSTO (Visual Studio Tools for Office), a set of libraries that are available
for use by any .NET application language, and of course classic Visual Basic itself
has been "brought into the fold" by making it a fully-fledged object-oriented
language complete with XML literals and LINQ query capabilities.

Which means, if somebody working for a small school district in western Pennsylvania
wants to build a simple application for tracking students' attendance (rather than
tracking it on paper anymore), what do they do?

Bruce Tate alluded to this in his Beyond Java, based on the realization that
the Java space was no better—to bring a college/university student up to speed on
all the necessary technologies required of a "productive" Java developer,
he calculated at least five or six weeks of training was required. And that's not
a bad estimate, and might even be a bit on the shortened side. You can maybe get away
with less if they're joining a team which collectively has these skills distributed
across the entire team, but if we're talking about a standalone developer who's going
to be building software by himself, it's a pretty impressive list. Here's my back-of-the-envelope
calculations:

• Week one: Java language. (Nobody ever comes out of college knowing all the Java language
  they need.)
• Week two: Java virtual machine: threading/concurrency, ClassLoaders, Serialization,
  RMI, XML parsing, reference types (weak, soft, phantom).
• Week three: Infrastructure: Ant, JUnit, continuous integration, Spring.
• Week four: Data access: JDBC, Hibernate. (Yes, I think you need a full week on Hibernate
  to be able to use it effectively.)
• Week five: Web: HTTP, HTML, servlets, filters, servlet context and listeners, JSP,
  model-view-controller, and probably some Ajax to boot.

I could go on (seriously! no JMS? no REST? no Web services?), but you get the point.
And lest the .NET community start feeling complacent, put together a similar list
for the standalone .NET developer, and you'll come out to something pretty equivalent.
(Just look at the Pluralsight
list of courses—name the one course you would give that college kid to
bring him up to speed. Stumped? Don't feel bad—I can't, either. And it's not them—pick
on any of the training companies.)

Now throw agile into that mix: how does an agile process reduce the complexity
load? And the answer, of course, is that it doesn't—it simply tries to muddle
through as best it can, by doing all of the things that developers need to be doing:
gathering as much feedback from every corner of their world as they can, through tests,
customer interaction, and frequent releases. All of which is good. I'm not here
to suggest that we should all give up agile and immediately go back to waterfall and
Big Design Up Front. Anybody who uses Billy's quote as a sound bite to suggest that
is a subversive and a terrorist and should have their arguments refuted with extreme
prejudice.

But agile is not going to reduce the technology complexity load, which is the root
cause of the problem.

Or, perhaps, let me ask it this way: your 16-year-old wants to build a system to track
the cards in his Magic deck. What language do you teach him?

We are in desperate need of simplicity in this industry. Whoever gets that,
and gets it right, defines the "Next Big Thing".

Enterprise consulting, mentoring or instruction. Java, C++, .NET or XML services.
1-day or multi-day workshops available. Contact
me for details.