“There are more things in heaven and earth, Horatio, than are dreamt of in your philosophy.” — Hamlet, Act 1, scene 5.

Oh the eternal words of Shakespeare. His works still apply today, and his words span the world over. You’ve heard him quoted before. If only he knew how I’ve quoted him today.

I’ve been doing some thinking. Not just the everyday thinking — not about what to eat tonight, or what I’m going to wear tomorrow (see, I rarely even do that). But the kind of thinking you usually do when you’re stuck fifty feet in a well, and all you can do to entertain yourself is knit. Except you don’t like knitting. So you let your imagination run wild.

My imagination lately has been running around Agile culture. The good, the bad, the not so good and the not so bad. The main takeaway, in the whirlwind of possibilities and dot-matrix printer-like results, is a question: Is Agile a culture of confirmation? And is this okay?

Micheal Bolton expresses his views on confirmation in his blog post here. In general, his view is that confirmation is the simple act of checking the correctness of known assumptions about a piece of software, while testing is the act of garnering new and useful information. I’ll admit, when I first wrote this blog post I hadn’t read Micheal’s, and I had my own ideas, but he expresses those ideas much better than I could. This post, then, is a bit of spin.

I was calling confirmation verification. I was redefining the term widely used in testing, and describing it colloquially as making sure stuff works and does what’s expected. The inverse then, and the million dollar question and what I care most about, is, what — and who — is making sure that same stuff doesn’t do what it’s not expected?

Now, this is a bit of a simple description, but let’s get specific, and bring this to Agile. Agile has standups. It has TDD, it has retrospectives. It has championed “the iteration”, continuous integration, and the utility of scripting languages. It’s also reduced the need for extensive design documentation, which is my personal favorite, and it promotes, at least in its marketing, very close interaction with the customer.

And let’s be clear: All of this is good stuff, and it’s very valuable. But there’s something missing. A hole. A way of thinking that isn’t quite included in the hype, and in the massive jigsaw puzzle of software development philosophy, there’s a piece yet to be filled.

The piece, of course, is the unknown. Continuous integration, TDD, unit tests, large suites of functional tests, etc., can only verify that the software is working. It can tell us when things stop working — which is perhaps its greatest strength — but it can’t tell us anything about the problems we don’t write tests for; the problems we aren’t aware could be problems. Sometimes these are user interface issues. Sometimes these are product oversights. Sometimes these are problems we think our automated tests cover, but don’t actually test. There could be any number of reasons. But if these problems are left unfound, they make it to the customer’s screen. And this, I think, is where much of the trouble lies.

The Agile philosophy and marketing material I’ve seen says nothing about detailed, investigative testing. Instead, it promotes what I see as rigorous confirmation using primarily automated testing practices. These practices verify the happy paths and the not-so-happy paths that we can think of at the time of writing, but they miss the things we don’t know. As well, when it’s time for the customer — the product manager or the client, not the end user — to review the software for correctness, they generally focus on their positive assumptions of functionality rather than focusing on the unknowns. And I think this is important. Because it might be my Kaner and context-driven upbringing, but the only techniques I can think of to help us discover what we don’t know revolve around exploration and manual testing. And an internal sense of curiosity. I just can’t think of any other way to learn.

In that respect — and a tip of the hat to those who gave me much appreciated feedback — I’ve learned from a previous post about dating that I need to state my conclusions more clearly. Here, I intend to be clear. Resisting another quote and some iambic pentameter, or some vague call for idealism, the moral of this story is: Agile practices are useful, have value, and I appreciate them. But in many cases, they can be made better when bolstered by a team of interested, investigative testers. It’s a matter of managing our unknowns, and I think the culture should embrace that.

Now — to end this soliloquy. Because the dot-matrix printer just died.

Fin. Exuent left.

Kurt Schrader, otherwise known as “The Shrade”, “Shradester”, and my boss, recently wrote a post about how we roll at Intent Media. And by how we roll, I mean how we use Cucumber to power automated functional tests of a large scale ad platform. Oh ya, that.

When I first mentioned Cucumber to a close member of the context-driven testing community, the reaction was “Oh, so you’re just entering manual scripted tests into a computer.” She’s South African, so it was more like, “You’re ent-ring scripted tastes into the computah!” (Ya, she knows who she is.)  Though I felt her pings of discontent quickly, the answer was — “Well ya, I guess…”

In the tools world, Cucumber is the blood relative of both your normal manually-scripted tests and, say, FIT or FitNesse, with a bent toward integration tests (if that’s what you use it for). It’s manual scripted testing because you type your manual scripts into the computer; it’s automated testing because you also define pieces of code that match up with each line of each test. All Cucumber does is find the right match — and then happily goes chug, chug, chug.

It’s been a fun ride so far, and I don’t have many complaints. There is a question about whether to write imperative or declarative steps throughout your tests, as well as where to put your implicit state (we leave it in the browser). But as Kurt mentioned, we’ve been highly successful so far. I’d be interested to see if we have the same troubles that plague manual scripted tests over the long term, or if automating them keeps them in the public eye (i.e., continuous integration = continuous test fixing). We shall see.

In any case, it should be noted that Cucumber isn’t our only line of defense, and part of the reason Cucumber works is because of everything else going on around it. We have PMs doing acceptance testing, devs doing TDD and unit testing, I’m leading exploratory testing (which arguably the PMs are doing as well), and Cucumber is backing us up with integration tests that both the devs and I write. And that’s only functional: We have performance testing, user testing, smoke testing, and the whole nine-yards going on elsewhere throughout the company.

So ya, I’m proud to say that’s how we roll. If you’re interested, I’d encourage you to follow Kurt’s blog for more updates about the technical aspects of Intent Media, and to see how we’re building “the next great online advertising startup” (TM).

I’d like to express this articulately some day, but for now I’ll just stand on the shoulders of greatness — and then correct him:

Given enough eyeballs, all shallow bugs are shallow. — Linus Torvalds Tim Coulter

Maybe it’s an economy of scale thing. e.g., When is there ever enough competent, manageable, able-to-fix-your-problem eyeballs?

I saw Star Trek today and was given a little treat as I walked into the theater.

I used Fandango. I bought the tickets online, and then printed them using Fandango’s machine stationed in the movie theater lobby. Instead of getting the usual ticket and receipt, I was given this little extra:

It was a ticket for Watchmen! Not only is Watchmen out of theaters, but the date the movie was intended to be shown was March 9th. If you look closely, on the bottom of each stub, you’ll see a printing date of May 29th — way past the showing date.

So the question is: Did Fandango’s machine erroneously print me an extra ticket, leading to this little tidbit? Or did I pay for a ticket I never picked up?

The world may never know.

It’s beautiful. Take a look.

I got to it using Fandango’s search box in the top right hand corner twice in a row. The first time I searched for something that returned no results (try “Court St.”, as if you want to search for a theater on Court St.). The second time I used the box to search for something else, like “Star Trek”.

Not a huge error, but I’m always a fan of professional error messages.

Life is unexpected. Yep. If no other theme better permeates these Flash Forwards (of which there are only two), it would be that we should wholeheartedly embrace the uncertainty of the future. Unfortunately, that future at times means we cannot attend to our blog posts, and sometimes that’s a travesty. Here, however, I’m finally catching up.

The Dr. Coulter I meant to introduce you to — or at least one incantation of him, of possibly many — would look something like this: Aside from his stunning good looks, brilliant personality, admirable intelligence and, let’s rightly admit it right here and now: his deeply poetic romanticism (to which all the women swoon), Dr. Coulter would be a computer science or software engineering professor, likely more teacher than researcher, somewhere respected, holding a Ph.D. in microsociology.

“But why do they swoon? Charlie, tell me why they swoon!” — Knox Overstreet

Okay. I admit: I’ve embellished on some points. But the real kicker was supposed to be the microsociology.

I stumbled upon microsociology when I was thinking about an MBA, back when I wrote The Prestige. Many schools in New York City focus on finance, and beside their MBA programs offer classes in economics. A few clicks on Wikipedia gets you to sociology, or better, macrosociology, dealing with human interaction on a wide scale (such as with economics). A few more clicks gets you to microsociology, the inverse, dealing with human interaction on a more personal scale.

Microsociology, to the extent I understand it, extremely interests me. I like paying attention to personal communication, almost overly so, to the point where I reflect on it as it happens; in fact, I wouldn’t mind researching it for its own sake. However, I want to keep using the degree I received from Florida Tech, and I still want to focus on software, namely testing. Thankfully, microsociolgy fits right in; that is, if we swap some humans for computers.

Some might not call this microsociology and instead call it human-computer interaction — which it probably is — but I don’t intend to focus on haptics interfaces or better UI design, etc. At least, not directly. Instead, I hope to research the more personal side of computing (i.e., “What makes users angry,”) and try to use that research to improve how we develop software and what bugs we decide to fix first.

My hypothesis is that, based on human social expectations with other people, we may place similar expectations on computers. And if so, the bugs that “break” these expectations may be the most damaging, eliciting the most emotion from the user. Perhaps the reason we personify computers at all (I’m sure we’ve all proclaimed, at one point, “it’s thinking”) is because we have unconscious expectations that they (they???) are thinking entities. And that they’re stupid. And that they don’t do what we tell them to do. And maybe — and this is only a hypothesis: If we study this phenomenon, it may help inform our design decisions up front as well as help us decide which bugs to fix after the fact. And if James Bach is right, and a bug is truly “anything that bugs you,” then we may really be onto something.

But, of course, life is unexpected, and the beauty of the future is, to quote greatness, “[that] you never know what you’re going to get.” Still, this research would be exciting in it’s own right (assuming it hasn’t already been done), easily earning its candidacy as my next Flash Forward.

Note: Flash Forwards are only flashes. I’m hoping someone corrects me on the things I take for granted, as I have a tendency to unintentionally hurt books I read, and my microsociology books are currently unscathed.

Update: See comments thread for more discussion.

From here:

The Association for Software Testing is pleased to announce its fourth annual conference, CAST 2009, to be held July 13-16. The meeting will be held in sporty Colorado Springs, Colorado, at the Antlers Hilton Hotel.  The Antlers Hilton offers stunning views of the Rocky Mountains and Pikes Peak, which serve as a dramatic backdrop for this year’s theme: “Serving our Stakeholders“.

[...]

The CAST 2009 Program Committee is seeking proposals for papers and presentations that explain how testers can serve stakeholders. Both academic research papers and industrial experience reports are welcome.

In addition to presentations that demonstrate service to stakeholders, we’re looking for personal experience reports that clearly demonstrate skills and practices of seasoned software testing professionals. We are looking for rich, diverse experiences and ideas that illuminate the theme.

If you have hands-on experience, and a fascinating story to tell, contact us and we will assist you in evolving your tale so it will be ready to present at CAST.

Read the full CFP.

And if you check out the speakers list, you may just find someone you recognize.

On the heels of James Bach’s Buccaneer Scholar, a new idea, book and blog where he details his experience with exploratory thinking and learning, comes my own wrestling, not only with how I learn, but what I should learn. Due to recent events I’ll make clear in a few weeks, this question has begun to capture all of my spare CPU cycles — and it won’t quit.

The crux of the question, I think, is this: When I was thirteen, I set a goal to go to college, get a degree, and work for Microsoft (then the most popular software company in existence). While in college, I latched onto open-source software and the free software movement — happily ditching Windows for Gentoo, Mandrake, then finally Ubuntu — and gleefully traded Microsoft for a slightly newer and more exciting company called Google. Google had free food, funky office chairs, and they fit the goal I previously made when I was thirteen. My sense of self was intact, no harm done.

When it came time to interview, though, things were messier than I expected. I was still trying to finish school — to graduate, move out of the dorm and wrap up all the extracurriculars that consumed most of my spare time; and I was, on the more personal side, simply trying to grasp what was happening as I made the transition from college life to real life.

It was all too fast. Not only was I not accepted by Google (that’s a story for another time), but I didn’t know where else to go from there. In magicians terms, I made all but the prestige of my childhood goals, and I was left like a confused audience member contemplating whether I should stand up and applaud.

But, prestige aside, I realized my longstanding goals ended after college, and whether or not I was employed by Big Popular Company mattered little to anything but my ego. Even if I did achieve my goal, I’d still be left with the same question: What happens next?

When I learn, I need to know the big picture. I like to know where I’m going, why I’m going there, and in general, how to get there — but not always. I know the devil’s in the details, but I like figuring out the details as I go along.

Looking back, I had more or less achieved my goal — college, degree, employment — but I didn’t have the big picture telling me where to go next. I still don’t. I’d assume even James’ Buccaneer Scholar, with his exploratory state of mind, would have some type of charter, though I can’t believe he’d always know where he is going. James, I’d like to hear you expound upon that if you haven’t already.

This leads to the more practical dilemma: Of my interests, which path do I choose? The following is a list of career paths I could be interested in taking. Though I run the risk of taking the totally incorrect approach to, well, life, of these career paths, I can’t help but be interested in their prestige. Here goes:

• Independent test consultant.
• Product Manager/Test Manager for large software engineering projects.
• Ph.D in Software Engineering or Cognitive Science, researching and teaching software engineering and/or software testing.
• Software Developer for cutting-edge, Web 2.0 technologies (I am that now).
• Entrepreneur for my own software development company. (I’ll probably need an MBA).
• Entrepreneur for my own software testing firm. (Again, an MBA).
• Tech Journalist for popular blogs and magazines. (I’ll probably need to study journalism).
• Politician. (I’ll probably need to study political science, though this brings me to my next interest…)
• Something in the arts, say, acting.

And as an aside, this whole process feels like one big game of Twister.

James Bach just started a new blog called How I Learn Stuff where he depicts the life of a Buccaneer Scholar. I’ve yet to fully understand his use of the term, but the posts and comments there are quite moving. This is likely a blog to watch.

I’m getting some responses (in comments, and email) that paint ofthewolves.com — or at least my description of it — as segregating the community around software testing. Although I could entertain this perspective (Why have two playgrounds?), I get the impression that the young software testing community is underrepresented, and could use a place where information is targeted specifically to — and possibly from — them. This doesn’t mean the whiz kids will go off on their own and create new testing techniques that we won’t share with the older generation; instead, it’s specifically targeted toward sharing our ideas and cultivating information that may be interesting to us — that is, cultivating it because I’m not sure it exists elsewhere. This could be: What to look for in an entry level testing job. What troubles occur transitioning into the software testing workplace. What skills are needed when beginning a career in the testing world. I’m not sure I can answer all these questions from my experience, but I think there’s value in putting them all in one place.

In some sense, this idea is an attempt on my part to find out if there actually is an “us.” My impression of most people in the software testing world is that they were thrown into it unexpectedly, yet have since chosen to make it a career. I rarely see people who have chosen it early on, or have recently graduated and have found it to be their passion. In my experience, I often get confused looks when I tell people I want to be a tester after having such a rich programming background in school. With ofthewolves.com, I not only want to share this experience with the world, but I want tell others teetering on the edge that this decision is okay, that there are others like you and that it is something you can make a career out of. To me, this is recognition, not segregation, and will eventually help the community as a whole.

Now. Behind this dramatic exterior is an implementation that still needs to be crafted. It may be that this blog presents informal case studies of the troubles new testers face. It may be this blog targets information toward nervous graduates who don’t have the testing equivalent of “Programming websites with Rails.” It may be that this blog inspires highly experienced testers to reflect back on their transition and spotlight the things they went through. I’m not sure, at this point, as the idea is still very young. Yet, I think it’s important to get others involved, to see if this idea has any value and to see if something good can come of it.

Lastly, as a more personal interest: I refuse to believe I’m one of the only up-and-coming software testers in my generation. If there are more out there, I want to give them the spotlight as these are the people I’ll be working with in the future.