Rebooting the Agile Manifesto – Part 2

“Working Software Over Comprehensive Documentation”

Human communication is imperfect, especially oral communication. If you’ve ever played the Game of Telephone you’ve experienced signal degradation first-hand. All kinds of factors interfere with fidelity: personalities, cultural differences, moods, domain knowledge, and the non-stop voices inside our heads that filter everything we hear and see through the cumulative effects of our personal experiences. But you don’t need to be scientific to acknowledge that when we speak we don’t formulate our statements with precision, and we don’t always listen with our full attention. Spoken language is convenient yet imperfect.

One thing researchers have learned is that the stuff coming out of our mouths is not always accurate. In the criminal justice system, for example, eye witness testimony was long considered authoritative until we learned that multiple witnesses can all tell different stories about first-hand observations. Similarly, former NBC News anchorman, Brian Williams learned the hard way that emotions can cause us to re-interpret reality when he reported that the helicopter in which he was riding in Iraq was hit by an RPG, when what he had actually seen was an RPG striking another helicopter flying ahead of his.

Oral traditions were entertaining, but unreliable and didn’t scale well. That may be a good thing. I’d bet at least one anthropologist has claimed that people in cultures constrained by the capacities of memory and oral transmission are happier and more tranquil than those of us suffering information overload. Nevertheless, once humans started recording their knowledge and stories, information began to stabilize.

Written language has a few clear advantages:

  • All recipients get the same message. It may still be subject to personal interpretation, but within less fuzzy boundaries.
  • When we write down a complicated message we have time to think about gaps and ambiguities, and close them before transmission.
  • The message persists with limited degradation over time.

Software design may not be affected by reality-distorting adrenaline surges, but it is subject to the strengths and weaknesses of the human brain, an imperfect instrument attempting to create unambiguous, intricate instructions for brainless machines.

As humans we’ve overcome some of our brains’ intrinsic imperfections by inventing clever augmentations, such as pictures, written words, and code. That last bit captures the bit about “Working software” in the Agile Manifesto’s second declaration. Some ideas should be tested and worked out in code. Expedient experiments often answer and raise important questions. More often, draft code is a way to work out how to solve a problem rather than what problem to solve. It’s easy to split hairs and claim that the solution to a business problem can be prototyped into existence. Here’s where we weigh the two sides of the declaration. In some cases, it truly will be more efficient to code it up and put in front of a stakeholder. In other cases, that could lead to inefficient trial and error. If we begin with User Stories to clarify requirements, it makes sense to also spell out an approach to each Story before implementing new or changed data models, services, and user interfaces.

network-diagramnetwork-spoke-pattern-shadowed-w-docBut why do we need more than a conversation and a whiteboard? The communications matrix grows exponentially. With two team members, there is one channel. For four members, six channels. Eight members, twenty-eight channels ((n^2-n)/2) That’s lots of opportunity for breakdowns.

Have you ever asked, or been asked, “Were you and I at the same meeting?” We can reduce the likelihood of misunderstanding by getting everyone in the same room, but we’re still subject to all kinds of communication glitches. A spec transforms the web into a hub and spokes, with far less room for interpretation.

Curbing My Own Dogma

And now I’ll appear to contradict myself. Bear with me. First, let me re-state my claim:

Written specifications are, by far, the most reliable and expeditious path to useful code.

That includes crisp requirements, as well as technical designs. Working out functionality in code as a trial and error exercise leads to misunderstandings and unexpected artifacts. However—brace yourself for the contradictions:

  1. It’s possible to include too much detail in a spec.
  2. Circumstances do exist in which detailed documentation is unnecessary.

Regarding my first contradiction above, I’m referring to degree. It’s hypothetically possible to write a spec so thorough that you could toss it over the wall to your developers and they could implement the application with perfect fidelity. However, the cost of such detail is extraordinarily high. Unless the owner of the spec is going on sabbatical just before implementation, it’s fine and efficient to allow for clarifications through conversations, both casual and in scheduled meetings. For example, the PO may not think of every possible error condition. Some of those may pop out as the development and test teams dig into the implementation. That’s no reason to kick the entire project back to the PO. The construction industry makes a great analogy for the realities of software design, and not in the ways you may expect. Sticklers—like me—love construction metaphors, such as “you wouldn’t build a bridge without a blueprint.” In construction, however, some projects are completed as “design builds,” in which the major structural elements are planned, but many details are worked out on site.

As for my second contradiction, certain applications, and the teams that have developed and maintained them, have reached such a high degree of maturity that incremental change often needs little explanation. A few bullet points, a quick flow diagram or UI wireframe may suffice. Don’t write documentation for it’s own sake. The goal is to get stuff done.

As for any technique, find your sweet spot. Start with more detail than you think you need, and then dial back with each successive initiative or iteration. I’ve found it takes only a couple of cycles to optimize for cost, time, and quality.

More to Come

In Part 3 of this series, as I promised, I’ll take a broader look at the intentions, implications, and practical application of the Agile Manifesto as a whole. And in a future post I’ll suggest what types of content to include in succinct functional and technical specs.