One of the Wolves

I sent a small note to the software-testing Yahoo! group asking, “What are some iconic books in software testing that paint a good picture of its history?” I got a huge response, and figured it’d be beneficial to others to duplicate it here.

(I’d link to the thread if there was a public archive, but you must be a member of the software-testing list to see it. Email me personally if you’re interested in joining.)

The list:

• Weinberg, “Fundamentals of Computer Programming (1961)” (Read chapter on testing. Whoohoo! I got the last one!)
• Hetzel, “Program Test Methods (1972)” (Looks like I got the last of this one too!)
• Myers, “The Art of Software Testing (1979)” (This one’s in the mail; it looks like they’re out now, though I remember a different URL…)
• Beizer, “Software System Testing and Quality Assurance (1984)”
• Kaner, “Testing Computer Software, 1st Edition (1988)”
• Beizer, “Software Testing Techniques, 2nd Edition (1990)” (Was recommended the first edition as well, though can’t find info on it.)
• Marick, “The Craft of Software Testing: … (1994)”
• Beizer, “Black Box Testing: … (1995)”
• Schulmeyer, Mcmanus, “Handbook of Software Quality Assurance, 3rd Edition (1998)”
• Kaner, Falk, Nguyen, “Testing Computer Software, 2nd Edition (1999)”
• Kaner, Bach, Pettichord, “Lessons Learned in Software Testing (2001)”
• Weinberg, “An Introduction to General Systems Thinking (2001)” (This book was written much earlier, though it was republished in 2001. I’ve linked to the 2001 version since I can’t find information on the original version.)
• Jorgensen, “Software Testing: A Craftsman’s Approach (2002)”
• Copeland, “A Practitioner’s Guide to Test Case Design (2004)”

This is a daunting list. Some of these — wink, wink — I was supposed to read fully in college. I can’t honestly say I did that, so I’m going to go back to them now.

Just a note: I get the impression some of these books contain ideas that are “obselete” or “out of date,” and may not be directly helpful today. This is a history lesson, and I’m specifically interested in how the thoughts in the industry have changed.

Thanks all for the responses, and I hope the reading list is beneficial to others.

I am happy to announce that my organization, The Open Planning Project, has released version 1.0 of the Livable Streets Network.

The Livable Streets Network is a conglomeration of five websites: NY StreetsBlog, LA StreetsBlog, StreetFilms, StreetsWiki, and LivableStreet’s Groups. All together, they help urban planners and community members alike work together to make our city’s streets a better place to live.

Applause to all for a great run!

After two days of managing — though one day of actually focusing on it — I’m realizing that this time, I’m keeping my hands clean.

It’s different than before. Last time we did this, I kept a close watch of the people I was managing, had long and detailed debriefings, and coached as much as possible. This time, it’s more: “give them what they need up front, and let them go off on their own.”

I can’t say I have enough information as to which approach I like more — so I won’t even begin to hint as to which approach we should do — but I’m gaining some insight here.

Without long and detailed debriefings, I’m seeing:

• A higher bug-duplicate rate than before.
• A lower understanding (in my head) of what’s actually being found.
• Less drive on my part to jump the information hurdle.
• And more of a desire to let bug reports speak for themselves.

On the same token, I’m seeing greater personal drive from testers to self-manage, which could be a big plus.

Though there’s good and bad here, it’s important to note the context changes:

• I have all new testers (i.e., they haven’t worked with me on testing before).
• Reports that they write are going directly to the manager I was supposed to report to, meaning there’s no apparent need for any “high level” reporting.
• Managing is lower on my totem poll time-wise, as I’m only supposed to spend two days a week doing it.

Although it pains me, in general, to not be as involved as I’d like, the outcome of this could be very informative. In simple terms: I think we’re on our way to finding our sweet spot, and by the time this is all over, we’ll have some good insight to share.

Good news! It seems, for the next two weeks at least, I’m back managing a testing group here at TOPP. I won’t be managing every day — only two days a week, due to other work — but the testing group will be going all five.

Our context is a bit different than before:

• We’ve got two (maybe three) weeks until deployment.
• We’re running risk-based charters, focusing more on changes and less on regression.
• We’ve got a few more resources ready than before.
• We’ve already got buy-in on the process.
• We’ve got responsive management who sympathize with the testing effort.

I recently coordinated the group today, and from my initial impression, we’re going to make waves in testing these next two weeks. I’m genuinely excited.

As a side note, it seems the presentation I gave about testing had great effect. It gave me a chance to introduce more testing ideas — along with the performance testing ideas that were the impetus of the presentation — as well as make me more confident when expressing these ideas to others. For example, I’ve had management tell me that “testing is an investigative process,” and then plan work with the general understanding that the testing group’s job is to inform.

I’m not really sure what happened, but the ideas are spreading like wildfire.

No guarantee I’ll give weekly updates like before, but I will keep you updated on the status of the testing group. So far, it seems like testing is a priority from management on down — even if we were only given two weeks — so among other good feelings, I’m starting to feel like we’re headed in the right direction.

We’re getting some Google attention at the geo-side of our organization. Today, TOPP’s work on GeoWebCache made Google’s open-source blog, mentioning GeoWebCache’s lead developer Arne Kepp.

Good work guys!

Update: Alright, full disclosure. I just realized the post was written by Chris Holmes, TOPP’s “geospatial CEO.” It’s still pretty cool, but a little less so.

I just gave a presentation at work about my experience at WOPR. From what I can tell, it went great, and was a good introduction for the organization.

Here are my slides.

They were crafted specifically for The Open Planning Project, though they may be useful elsewhere.

A big thanks goes to everyone at WOPR, and a thank you goes to David Winslow at The Open Planning Project for setting up this talk!

There’s nothing like getting off of a plane, taking a shower, then showing up almost-late to jam with a bunch of software engineering professionals. There’s nothing like it, and career-wise, it was one of the best weeks I’ve had.

I was in Miami two weekends ago for a friend’s wedding. After missing the vows — we got lost! — paintball, and an extremely great weekend, I flew back Monday morning and headed straight to the pre-WOPR event. WOPR, for those that don’t know, is the Workshop on Performance and Reliability, and “pre-WOPR” is a similar workshop held a few days beforehand.

Like all workshops previously, I had a bit of nervousness going in. For this one specifically, I felt I didn’t have much experience related to performance testing, and I felt… well, performance anxiety (pun intended) because I’d have to to participate with folks each having 10 - 30 years experience in the field. To make matters worse — as nervousness is concerned — for WOPR, I was asked to give my views on each experience report, in front of the group, after group discussion finished, in a spotlight that became known as “Tim-bits”. For a newbie out of college, this was some pressure.

Though I had nervousness going in, I enjoyed both workshops immensely. I wrote my Tim-bits down — as “sound bytes” that best describe what I learned — and am providing them for WOPR, pre-WOPR, and the world!

I’ve attributed each bit to the experience report that led to it; however, I would like to stress that many of these bits were the direct result of group discussion afterward. Attribution not only goes to the presenters, but to everyone who attended the workshop.

1. Exercise-centric courses that are meaningful to the audience will be more likely to produce better performance testers in a practitioner-based performance testing course. (Ross Collard, independent consultant)
2. Tools don’t solve problems, they help people solve problems. Like a hammer helps in building a roof, our tools help in doing performance testing. No one tool will do our work for us. (Goranka Bjedov, Google)
3. Performance testing’s monitoring tools are very much like astronomy’s telescopes. Although our tools do more and more things everyday, the best monitoring tools will be those that provide more — and more accurate — information. (Scott Barber, PerfTestPlus, inc.)
4. The same problems in consultant-taught performance testing courses exist in academic professor-taught courses: Fuzzy prerequisites; diverse student backgrounds; variable student interest; turbulent teaching environments. They’re all there in both contexts, and the problems generalize. (Gustavo Vazquez, Uruguay Center for Software Testing**)
5. a) The knowledge of a good performance tester intersects all facets of software engineering. When teaching them — for one course, at least — you must give them a slice of programming, a slice of process, a slice of statistics, and a slice of practical testing environments. Doing so successfully is difficult. b) Bureaucracy, and conflict of interest by those in power (management) can screw up a very effective testing course. (Dawn Haynes, PerfTestPlus, inc.)
6. Graphs, graphs and more graphs! They are — as far as we know — the quickest and most powerful way to give decision makers the information they need to make release decisions. Prepping the decision makers beforehand at the start of the testing process will set expectations and create a healthy reliance on the testing group. (Jude McQuaid, Intuit)
7. Relate performance testing to beer.*** (Andre Bondi, Seimens)
8. Apprenticeship in performance testing has benefits in some contexts, and is more beneficial when testers are newer (avoid the word “junior” ). Also — like a Sith Lord — it’s too hard to manage too many apprentices. (Roland Stens, independent consultant).

** Gustavo: Let me know if that’s the correct translation.

*** His slides are available — and I’m working on getting them — that explain the analogy much better. The group roared with excitement when he showed them, and some said it was “the best performance testing analogy seen yet.”

Though there’s much more detail lurking behind each bit above, the biggest thing that struck me was non-technical — rather, it was how comfortable I felt throughout these two workshops.  I can’t quite pinpoint the exact nature of this comfort, or what made these workshops different than the many others I’ve been to; however, in general, I finally feel like I’m “part of the group.” I probably always was — heck, I’ve been dubbed a “wolf” for a long time now — but I think the realization is finally setting in. Big thanks to all!

Speaking of thanks, I want to thank Rob Sabourin for being content owner; Ross Collard and Scott Barber for co-organizing; Julian Harty and Goranka Bjedov for hosting; Paul Holland and Nick Wolf for facilitating; and all the participants who made WOPR and pre-WOPR great events to attend. I learned a lot — more than what my Tim-bits might show — and I appreciate all the great insights that came from this experience. Thank you!

The Open Planning Project just released a small community website for those in the greater NYC area: http://www.blockpartynyc.org

From here:

Block Party NYC is a new program by the New York City Streets Renaissance Campaign helping neighborhoods around NYC come together and enjoy their street, free from the usual hazards and distractions of automobiles. This summer, we’re providing mini-grants to over 30 block parties throughout the 5 boroughs, [as well as] the services of a professional urban planner to combat community problems like traffic, speeding, noise and air pollution.

Interested in meeting your neighbors? Getting outside? Having some good ol’ fashioned family fun? Well here’s your chance.

I’ve been playing around with the look of the site in order to get more space on-screen. You’ll notice that the sidebar’s gone, and the archives have moved to their own page.

The one problem, which, depending on your perspective, may not be a problem at all: It’s hard to tell the difference between the main page (’/'), and a single post. For instance, go to http://www.oneofthewolves.com (you’re likely already there), and then click on a single post. Can you tell the difference? If you clicked on the first post, were you able to tell the difference without scrolling?

I’m not sure I like it, though I do enjoy the added space. Watch for more dust as I keep making changes.

Since my previous post below, I received three spam email messages that linked to malicious content using Google’s ad servers. The first time was interesting, though the next few times made me think: Is Google the only ad service that allows this?

The answer is no. I went searching the net for online ads similar to Google’s, and I found some pretty interesting results. See a pattern?

Where known — or easy to find through a whois lookup — I’ve listed the company serving the ad.

1. Google: http://pagead2.googlesyndication.com/ …. &adurl=http://www.oneofthewolves.com
2. Interclick: http://a1.interclick.com/ …. click.ic?http://www.oneofthewolves.com
3. EyeWonder: http://www.eyewonderlabs.com/ …. &click=http://www.oneofthewolves.com
4. Etology.com: http://pages.etology.com/gtbclk/ …. /15/http://www.oneofthewolves.com
5. MySpace: http://desb.opt.fimserve.com/lnk/?k=Nz …. href=http://www.oneofthewolves.com
6. http://adc.brandreachsys.com/ …. &Redirect=http://www.oneofthewolves.com
7. Tacoda: http://anad.tacoda.net/ …. //REDIRURL=http://www.oneofthewolves.com
8. ReduxMedia: http://ad.reduxmedia.com/click,JiIAA …. ,,http://www.oneofthewolves.com
9. DoubleClick: http://ad.doubleclick.net/ …. fhref=http://www.oneofthewolves.com

For some reason, it seems these companies don’t care that they’ll redirect to any site on the net. Granted, sites like desb.opt.fimserve.com look like spam anyway… But Google is a household name. Shouldn’t they care?

The one condolence I had in all this was Yahoo! For reasons that you might expect, the following link results in: “This link is not authorized by Yahoo!”

http://rds.yahoo.com/_…. /**http://www.oneofthewolves.com

Woot!