“There are more things in heaven and earth, Horatio, than are dreamt of in your philosophy.” — Hamlet, Act 1, scene 5.

Oh the eternal words of Shakespeare. His works still apply today, and his words span the world over. You’ve heard him quoted before. If only he knew how I’ve quoted him today.

I’ve been doing some thinking. Not just the everyday thinking — not about what to eat tonight, or what I’m going to wear tomorrow (see, I rarely even do that). But the kind of thinking you usually do when you’re stuck fifty feet in a well, and all you can do to entertain yourself is knit. Except you don’t like knitting. So you let your imagination run wild.

My imagination lately has been running around Agile culture. The good, the bad, the not so good and the not so bad. The main takeaway, in the whirlwind of possibilities and dot-matrix printer-like results, is a question: Is Agile a culture of confirmation? And is this okay?

Micheal Bolton expresses his views on confirmation in his blog post here. In general, his view is that confirmation is the simple act of checking the correctness of known assumptions about a piece of software, while testing is the act of garnering new and useful information. I’ll admit, when I first wrote this blog post I hadn’t read Micheal’s, and I had my own ideas, but he expresses those ideas much better than I could. This post, then, is a bit of spin.

I was calling confirmation verification. I was redefining the term widely used in testing, and describing it colloquially as making sure stuff works and does what’s expected. The inverse then, and the million dollar question and what I care most about, is, what — and who — is making sure that same stuff doesn’t do what it’s not expected?

Now, this is a bit of a simple description, but let’s get specific, and bring this to Agile. Agile has standups. It has TDD, it has retrospectives. It has championed “the iteration”, continuous integration, and the utility of scripting languages. It’s also reduced the need for extensive design documentation, which is my personal favorite, and it promotes, at least in its marketing, very close interaction with the customer.

And let’s be clear: All of this is good stuff, and it’s very valuable. But there’s something missing. A hole. A way of thinking that isn’t quite included in the hype, and in the massive jigsaw puzzle of software development philosophy, there’s a piece yet to be filled.

The piece, of course, is the unknown. Continuous integration, TDD, unit tests, large suites of functional tests, etc., can only verify that the software is working. It can tell us when things stop working — which is perhaps its greatest strength — but it can’t tell us anything about the problems we don’t write tests for; the problems we aren’t aware could be problems. Sometimes these are user interface issues. Sometimes these are product oversights. Sometimes these are problems we think our automated tests cover, but don’t actually test. There could be any number of reasons. But if these problems are left unfound, they make it to the customer’s screen. And this, I think, is where much of the trouble lies.

The Agile philosophy and marketing material I’ve seen says nothing about detailed, investigative testing. Instead, it promotes what I see as rigorous confirmation using primarily automated testing practices. These practices verify the happy paths and the not-so-happy paths that we can think of at the time of writing, but they miss the things we don’t know. As well, when it’s time for the customer — the product manager or the client, not the end user — to review the software for correctness, they generally focus on their positive assumptions of functionality rather than focusing on the unknowns. And I think this is important. Because it might be my Kaner and context-driven upbringing, but the only techniques I can think of to help us discover what we don’t know revolve around exploration and manual testing. And an internal sense of curiosity. I just can’t think of any other way to learn.

In that respect — and a tip of the hat to those who gave me much appreciated feedback — I’ve learned from a previous post about dating that I need to state my conclusions more clearly. Here, I intend to be clear. Resisting another quote and some iambic pentameter, or some vague call for idealism, the moral of this story is: Agile practices are useful, have value, and I appreciate them. But in many cases, they can be made better when bolstered by a team of interested, investigative testers. It’s a matter of managing our unknowns, and I think the culture should embrace that.

Now — to end this soliloquy. Because the dot-matrix printer just died.

Fin. Exuent left.