My Testing Proposal
If you haven’t seen my previous post, I’ve been given the chance to lead a testing group for the next few weeks. For this new role, I was told to document my process. This is what I came up with.
Although I think I have most stakeholders already onboard, I wanted to create a proposal that outlined some of my ideas about testing, as well as how we’ll do session-based test management. Even though I could have just “documented my process,” — they’re giving me a lot of free reign here, which I love — it’s a proposal because… well, it’s likely going to change, and I’m not the only one with input. 
I have this fear that it’s too general — which, it likely is — but I think the specifics will come as we flesh out what works best for us. George Box comes to mind: “All models are wrong, some models are useful.”
Tim,
Congratulations on making lead so early in your career. The process you’ve outlined is a great explanation of SBTM - I think I’ll even use part of it to explain what we do at PlateSpin.
It’s great to see you adopting SBTM. I have to ask what are the advantages to using SBTM over some other method? Why use SBTM at all? Why not just go completely ad-hoc? Why not go completely scripted?
One aspect in your plan is a little light - reporting of the results of testing. How you are going to present your findings to the stakeholders. Are you going to use the dashboard from RST? Maybe a modified version of it for your context?
Something extremely powerful I do is keep a written summary throught the release called
“Top 10 things a critic might say”. If we released this software today and gave it to a critic - what might they say about it” This way I draw attention to the things I think should get fixed. Most of the time I can get what I want. Be careful though -you can lose credibility by fighting for the wrong things. I did this a lot in my early career and learned the lesson the hard way.
Any idea if you need to provide metrics? Metrics are popular and I’m sure you’ve heard many debates on the subject. Life can be different in the real world where people want the metrics regardless of the “usefulness” of them and opinions in the testing world.
Good luck with your team!
Adam
[Tim's reply: Hey Adam, thanks for your input. To answer your question about reporting, the answer is, really... I don't know.
I'm not sure of a good way to report this stuff. Management wants me to be the arbiter for all bugs that come through my testing team, and make sure they're actually bugs before sending them off to development. Although this doesn't scale, they primarily want me to write Trac tickets and communicate informally (but at least once a day, and more toward the end of it). All this is up in the air, however, as it's really been, "We need a testing group. Tim knows a thing or two about testing. He says he likes it. Let's throw him in." I wasn't aware of the dashboard from RST; I'll give that a look.
As far as SBTM vs. other methods goes, I had a big long response for you, but finally realized your questions were rhetorical -- they were only asked to show possible additions to my proposal. Thank you for those. I've got answers for you if you'd like to know, but for now, I'll just assume that's what they were intended for.
As far as metrics, my impression is that they don't want them. Well... they haven't asked for them, at least. I think I'll use SBTM's metrics as a model for ours -- just to cover my butt if they ask for them later -- but my guess is metrics aren't their first priority. My take is that metrics can be abused and easily misconstrued, so I'm going to avoid them if I can.
Thanks for your response. It gave me a lot to think about.]
[...] wrote a proposal, which detailed a scaled-down version of SBTM with less bureaucracy (ie., metrics) and less [...]
[...] different than before. Last time we did this, I kept a close watch of the people I was managing, had long and detailed debriefings, [...]