AST Certification Debate
Today marks to date of the Association for Software Testing’s debate on Certification. It was a glorious day, and I believe it deserves some remembering.
To provide full discloser — something I’d like to see more in the industry — I, as of now, am the sole developer on the OpenCertification project. I endorse the project as well as contribute to it. I have a bias.
To my dismay, only about nine audience members (plus or minus two) showed up to participate in the debate. The goal was to have a discussion comparing and contrasting current and future certifications in the software testing industry.
Representing each of the certifications were as follows, in no particular order:
- Dr. Magdy Hanna, IIST
- Joe Gance, ASTQB
- Dr. Cem Kaner, OpenCertification
- Tom Ticknor, QAI
- Dr. Allen Johnson, Hustin-Tillotson University (representing an academic perspective, instead of a certification)
The format was structured in a morning of presentations and an afternoon of debate.
My take on the morning’s presentations:
IIST and QAI seem to have great models for certification — at least, as good as you can get for knowledge based certifications. As far as I can tell, it seems that both IIST and QAI have put a lot of thought into how they certify. This is a good thing. Unfortunately, like ASTQB below, it seems their content is lacking (I’ll get to this in a second).
The ASTQB presentation was considerably troubling. I was not able to discern the value of this certification as described by the presenter. The only thing I took away from it — remember my bias and human fallibility when I say this — is that the ASTQB certification helps keep personal training in the minds of those being certified. That’s it. Although this may be a good thing for those involved, I think it misses the meaning of certification within the industry.
The OpenCertification presentation and the academic presentation said a lot of things that I believe and would (do) encourage. Since I’ve already made my bias clear, there’s no need for me to toot my own horn. (Note however, that there was some debate on a conflict of interest between the certification debate and AST’s sponsorship of the OpenCertification project. I have no details on this sponsorship, and I believe there is a discussion on its implications already in place.)
The real meat of the debate came when the audience was asked to participate. This was only allowed during the afternoon session, as only clarifying questions were allowed during presentations.
In order to stay objective and not blame or belittle those who made comments, the following is a self-recorded (by hand), non-official, and non-complete transcript of my experience of the afternoon debate, with all quoter names stripped from the record. All quotes are provided from the presenters representing IIST, QAI, and ASTQB (they were the only presenters who surprised me), and all questions are provided from one or more audience members.
Q: [Some of the presenters] previously mentioned that each certification was their own standard created to better the industry. In what way would you call these standards “competing standards,” and how do you think this competition may segment the industry? This is question is for all presenters.
Here’s the interesting quotes I received:
“We are each a standard, yes, but we have full agreement within our bodies of knowledge. We are each filling a niche that needs to be filled.”
“I would agree with [the previous presenter]. We are in full agreement within our bodies of knowledge.”
“We are not competing standards. We agree, and are of benefit to each other.”
[Some time passed.]
Q: I was not aware that there was an agreement between the bodies of knowledge within each certification — if there is, I think these agreeing bodies of knowledge will greatly benefit the industry. What way do you think is the best way to promote this agreement? I see it as a conflict of interest to say, “We agree with [that certification over there], but you should take our certification anyway.” What is your take on that? This question is for all who want to answer.
“I don’t think any of us want to answer that question.”
[There was some sifting for answers here.]
“We have agreement, and we show it on our website. Our certification allows courses from [other certifications], and it’s on our website because we want to promote the benefit to the industry.”
“No, I’d guess it’s on your website because it’s a great marketing campaign. I didn’t think of that [for our certification]. We may use that.”
[After pressure from the audience increased: ]
“We’re in it for the money.”
“Ya.”
“I agree.”
Q: How do your certifications assess those who take it?
[There was some discussion.]
“If you sit in front of me in the class room, why do I need to put a question in the [certification] test? I have taught you. I have taught you and you have learned it.”
Q: I’ve worked at [Company X] and [Company Y] for about six years, and both companies are well respected within the industry. It doesn’t seem like any of the well respected companies like Microsoft, Google, Apple, Adobe, Amazon, etc. even pay attention to these certifications. In other words, it doesn’t seem like people with real experience care. Who is your target audience for these certifications?
“People who are currently looking for a job, and can’t find one. Also, people who are unsure if testing is the profession for them.”
“If you’re the type of person who learns from reading a book, I would recommend not being certified because it would be a waste of your time.”
Q: How does your certification program allow for inclusion of new techniques within its body of knowledge?
[The above question was hard for the audience member to ask, because the audience member was interrupted frequently. During the interruption, techniques such as exploratory testing and model based testing were mentioned. Tension increased.]
“Exploratory testing is an ad-hoc process, and is a simple technique.”
“Our courses teach every technique on earth about exploratory testing! It is so basic, it is in the basic course requirements!”
[Remember: There's full agreement between each certification's body of knowledge. Right...]
“I don’t see why you’d want any new techniques within a certification. What new techniques have we had in the last 30 years?”
“If I wanted to teach model based testing in a course within my certification training, I don’t have to get approval from the board because model based testing is not within the body of knowledge.]
“Putting model based testing in the body of knowledge would be a dumb thing to do.”
[This caused great tension.]
Q: So if we haven’t had any new ideas in 30 years, then who could possibly come up with a new idea???
“Glenford Myers.”
[This was not a joke. The presenter was serious when he said this!]
[End.]
I don’t know about you, but I’d really like you to take a good look at the current certifications in software testing, and in related software engineering fields. I believe that every certification (excluding OpenCertification) lost much credibility during this tutorial. My only wish is that more people were there to see it.
In my sole opinion, as a member of an industry that so badly wants to speak for itself: This was a great day for software testing. Whatever your take is on certification — either for it, or against it — there is no doubt that this debate was helpful to the industry.
A big thank you goes out to all who were involved.
To my dismay, only about nine audience members (plus or minus two) showed up to participate in the debate.
I am disappointed that more did not participate, but I am not surprised. I believe that this debate is good and healthy for our industry. I am glad that AST hosted the debate. The problem is that companies tend to pay for employees to go to conferences and tutorials to pick up and bring back practical information. The other tutorials were real tutorials that included practical “how to” information.
I talked to a number of people that were personally interested in the debate but instead participated in tutorials that fit the needs and expectations of their employers that paid to send them to the conference. I also missed the debate so that I could attend a more practical tutorial.
Perhaps Glenford Myers could come up with a new idea for a debate with better attendance.
Thank you for reporting for those of us that missed it.
Ben Simo
http://QualityFrog.com
Thanks for writing this up. Any standards always lag, but it is scary how slow this is. I was part of the IEEE 829 test doco update last year, and one of my first comments was why is there no mention of a test strategy? Apparently it wasn’t in the scope of the revision. I have been using test strategy docs since the early 1990s, yet the standard does still not mention them. All the BOKs use 829, so they also have this issue…..
I liked the reference to Glenford Myers. I actually won the STARwest best paper last year, revisiting Glenford’s testing axioms 30 years on.
Go to http://www.testingspot.net, then search for Glenford Myers to read it. Some of his key ideas have been forgotten, especially around bug clusters…..
cheers,
Erik
Melbourne, Australia
[...] Project (he wrote the question server) and as a result attended the debate. According to his post 9 (+/- 2) people sat in on the debate. That is 4.89% of the 184 total CAST2007 attendees and 7.96% [...]
This link will likely be dead in a couple days as I am re-org’ing my site, but here is my commentary on certification abusing Tim’s attendance numbers. The short version is “let’s rat-hole the whole debate”.
http://www.goucher.ca/blog/?p=243
-adam
Thanks for summarizing Tim. I have to say I was troubled by the event. I think the main reason is that I feel AST got a black eye in the minds of those who don’t know everything the organization is about. I say this because my perception of bias in the room against the three organizations was palpable. The reason for this, I believe was the low attendance of neutral attendees or attendees who were certificate holders and supported one of the organizations. That and the Open Cert AST sponsorship that you mentioned made the debate appear to be a “setup”, in the eyes of many. (A word I heard frequently that day).
Whatever happens with certification, I want AST to continue to grow and be a home for impartial, thought-provoking discussion where anyone who cares about software testing should feel welcome. There are literally thousands of testers out there who are inexperienced and right or wrong, look to certification as a first step towards being someone in the industry. I don’t want AST to appear elitist and shut those kinds of people out.
I really couldn’t care less about getting the heads of the certifying bodies to all agree with each other (though three of them apparently do already?!). What I really care about is bringing in the newest of us and showing them that testing is about thinking for yourself and learning what approach works best for you.
AST must appear more open to “outsiders” if we’re to be a viable organization to the majority of new testers out there. The perception last Wednesday, real or not, amongst some attendees was that we are exclusive of those with differing opinions.
Thanks to put this resume of the event here.
I cant make it to CAST, but I had to deal with certifications every day. Everyone here at Colombia ask you about them, and I think they have no real value.
I would look forward to see where is going the open cert.
Not to completely hijack the tread (I’ll produce my own on my own blog as this has been bubbling around for awhile), but in response to:
AST must appear more open to “outsiders” if we’re to be a viable organization to the majority of new testers out there. The perception last Wednesday, real or not, amongst some attendees was that we are exclusive of those with differing opinions.
I agree in principle to this, and would like see it happen, but I don’t think it will ever happen. At least with the current leadership in place. Don’t get me wrote, I think they’re all smart and dedicated and threw an enjoyable conference, but they are also all from the ‘context’ school and relations between the perceived schools are, shall we say, frosty at best most times. Until such a time as there is some grand unification effort between the ‘schools’ I suspect AST will remain a pseudo-clique of similar thinking people. Much like ASQ and the CMMi crowd have similar cliques. And I’m not sure that that is such a Bad Thing[tm]
-adam
I’m not looking for school unification. There was a long, contentious thread on that in the software-testing Yahoo group.
What I am saying is that if AST is going to grow, we need some way of reaching out to the newbie tester. The certification folks are doing a great job of that, by offering something tangible to just about anyone with no experience. The school of thought that they learn in the process often becomes the basis for their career in testing. I’m saying that AST needs to be deliberately reaching out to these people. I think the Open Certification could be a good way to do that.
Cem Kaner said himself that the Open Certification is a concession to the popularity of these tests (paraphrase from Wednesday’s debate). I think it is such a hotly debated issue because the certifications do tend to represent the schools of thought. It’s almost to the point where if you don’t have a certification test for your school, it might as well not exist in the eyes of outsiders.
[...] regardless of philosophical conflicts on such contencious issues as certification. There is the perception already that we have blown this one for just that reason and it was a relief to see someone share [...]
I’ve done software testing for a little over 4 years. I had the misfortune of taking one of IIST’s International Testing Certification Week (ITCW) course sets. It was far below my expectations for professional training, and this gave me the impression that the “Certifications” were of equally low value. Even though IIST marketed these ITCW weeks with a money-back guarantee if I wasn’t satisfied, they’ve come up with one excuse after another and I can’t get my money back from them. I’m out about $2500. Yes, they’re definitely in business to make a profit!
[...] enjoyed this CAST more than the other two previous CASTs. Though CAST 2007 sported a fun and enlightening certification debate, and CAST 2006 paved ground with an interesting but heated discussion after the first keynote, this [...]