This is my attempt at attracting the very few readers within my age group to get out to the polls.
Please vote tomorrow (or, for whenever you read this, today). Not only is it fun, but all the cool kids are doing it.
Seriously, it’s more important than you may think. Make it out to the polls tomorrow and participate in something that will be of great impact to you and your country.
I have no idea if you can see this. My website and personal email have been going up and down for the last couple days, likely due to a new server upgrade gone wrong. Inquiries haven’t really elicited any response from my host, so if I’m not responding to your email, you know why.
If you’ve been checking the site — though I hear people use RSS nowadays 
— let me know if you’ve seen outages. Not sure about you but it’s been pretty rocky from my end.
Have a good weekend!
I’ve tried to write this post many times. I’ve tried taking different strategies, from telling a story-based account that led me to an idea, to being blunt and direct about what I was trying to say. I even tried using big words that act as “intelligence flags” for those who don’t understand the message. Though I made three tries, I was satisfied with neither.
Instead of trying to talk around a topic that doesn’t seem to have a whole lot of discussion — nothing that I can divine, anyway — I’m instead going to say it very cheaply.
Given these premises: …
… And, given this observation: …
… I’ve come to this conclusion:
What if bug reporting systems had trackbacks, for instance? Or a meaningful RSS feed?* Or maybe, “What’s new?” “What’s interesting?” or “What’s the hottest bug that we know of right now?” What if a bug report displayed who was talking about it (e.g., who linked to it) or performed the mundane task of figuring out which bug reports were related (again, who’s linking to it)?. Wouldn’t that make things easier? If there was a better interface — rather than the top of the bucket — that management could peer into, wouldn’t that solidify the role of the testing organization and help management keep track of their software’s status day-to-day?
Isn’t that bug advocacy?
This all stemmed from our use of bug reporting systems at The Open Planning Project. In short: One Big Bug Reporting System is unmanageable given many users and many avenues of input. Eventually, the silo will become too big. Multiple bug reporting systems are unmanageable given software systems that are tightly integrated. Eventually, a single bug will affect each system, and multiple people — whose hands are somewhere within each bug reporting system — will have to respond to it.
I almost hate to say it, but online bug reporting systems are the oldest Web 2.0 applications I know. So why have they fallen so far behind?
* Our system, thankfully, has an RSS feed, though my impression is most bug reporting systems are behind the times.
PS: I could be convinced that all I’m really suggesting is that Y generation testers will start advocating bugs to management through current Web 2.0 technologies, such as blogs. However, I wouldn’t be disappointed if some bug reporting systems learned from the blogging movement, evolved a bit, and became more open to information sharing across a large, interested audience.
My current open-source favorite is Trac. No guarantees, but there may be a new plugin on the horizon.
Guess what — I’m moving to Brooklyn!
That’s right. I’ve decided that SoHo’s way too expensive for my meager bachelor’s degree wage. Instead of paying the exorbitant amount to be “on location”, I’m packing up and moving out of the city to somewhere much less expensive.
Pros:
Cons:
If you payed attention to my big NYC move a couple months ago, you should notice that my contract in the Manhattan apartment is not up yet. This is true. However, I’ve found someone who wants to sublet my apartment and pay rent for me until the contract ends in August.
The impetus of the move was all about saving money. The trouble was, although I was in a really cool area — SoHo’s pretty hip nowadays, or so I hear — I couldn’t take advantage of it because I didn’t have much disposable income. Now that I do (or, will, once I move) I can do things like play around in SoHo, or go to a friends wedding in Miami without worry.
Official move in date is the 15th of this month, though I could move in as early as the 12th. If, by that time, I haven’t given you my new address, feel free to bug me for it.
Oh, and I’m gonna paint this one, no matter how long it takes me! Colors. Maybe bright ones. Maybe Earth tones. I don’t know, but it’s gonna be a good space.
So, in a totally unexpected turn of events, I’ve been crowned the interim test manager for the next five weeks!
To make a long story short, our organization doesn’t have a structured testing group. From before the time I’ve been here until now, developers were expected to both write code and do testing (in this case, “write code” == write code + unit tests + other styles of automated tests, and “do testing” == do whatever’s needed to find new bugs). Because we’ve got a very important release looming five weeks from now, and because we put out a few large fires after the last release, management decided to enlist a testing group until at least five weeks from now. And — to make this great news even better — they’ve decided to let me take on the effort.
They’re pretty much letting me do whatever I think is best. There’s a few limits to what I’m able to do in this position, but the most exciting questions were “Tim, do you want to lead the testing effort?” and “Tim, how many people do you need?”
I’m still doing research to figure out what would be best in our context, but my inclination is to do some form of session-based test management. My guess is that it’ll turn into the Bach brother’s SBTM with a little bit less paperwork. Although we may not be using James’ test management software, the idea though, is the same: create charters, do sessions, debrief… rinse, and repeat.
If this position turns out to be something longer than five weeks, however, there may be a chance we can keep more records a la the Bach brothers’ SBTM. I’ve talked to one of the stakeholders already, and — if that becomes a reality — we would (could) release all or most of our session sheets into the public domain. What this means is, since NDA’s don’t exist in our organization, we could amass the first publicly available database of session-based testing records. From a research/bleeding edge point of view, this could get very cool very quickly.
Since I’ve needed to do some research in order to assess risk and prioritize the testing effort, I’ve been holding meetings with stakeholders in order to get their point of view. When talking with them, I’ve found these questions to be very helpful:
I’ve asked these questions to five important stakeholders so far — two devs, two managers, and the guy who answers the phones — and I’ve gotten great responses, and some great ideas for initial charters.
All in all, I’m very excited, and I can’t wait to see what comes of these next five weeks.
UPDATE, 3/1/08: When I described who I talked to to get information, I mentioned “the guy who answers the phones.” To his credit, this guy does much, much more than simply answering phones, and he is a real asset around the office. I decided to describe him that way in order to emphasize his user-facing role: he is the sole person responsible for talking with users. Because of this, he is privy to a wealth of information that others generally don’t have, and is therefore extremely valuable to this project.
Today marks to date of the Association for Software Testing’s debate on Certification. It was a glorious day, and I believe it deserves some remembering.
To provide full discloser — something I’d like to see more in the industry — I, as of now, am the sole developer on the OpenCertification project. I endorse the project as well as contribute to it. I have a bias.
To my dismay, only about nine audience members (plus or minus two) showed up to participate in the debate. The goal was to have a discussion comparing and contrasting current and future certifications in the software testing industry.
Representing each of the certifications were as follows, in no particular order:
The format was structured in a morning of presentations and an afternoon of debate.
My take on the morning’s presentations:
IIST and QAI seem to have great models for certification — at least, as good as you can get for knowledge based certifications. As far as I can tell, it seems that both IIST and QAI have put a lot of thought into how they certify. This is a good thing. Unfortunately, like ASTQB below, it seems their content is lacking (I’ll get to this in a second).
The ASTQB presentation was considerably troubling. I was not able to discern the value of this certification as described by the presenter. The only thing I took away from it — remember my bias and human fallibility when I say this — is that the ASTQB certification helps keep personal training in the minds of those being certified. That’s it. Although this may be a good thing for those involved, I think it misses the meaning of certification within the industry.
The OpenCertification presentation and the academic presentation said a lot of things that I believe and would (do) encourage. Since I’ve already made my bias clear, there’s no need for me to toot my own horn. (Note however, that there was some debate on a conflict of interest between the certification debate and AST’s sponsorship of the OpenCertification project. I have no details on this sponsorship, and I believe there is a discussion on its implications already in place.)
The real meat of the debate came when the audience was asked to participate. This was only allowed during the afternoon session, as only clarifying questions were allowed during presentations.
In order to stay objective and not blame or belittle those who made comments, the following is a self-recorded (by hand), non-official, and non-complete transcript of my experience of the afternoon debate, with all quoter names stripped from the record. All quotes are provided from the presenters representing IIST, QAI, and ASTQB (they were the only presenters who surprised me), and all questions are provided from one or more audience members.
Q: [Some of the presenters] previously mentioned that each certification was their own standard created to better the industry. In what way would you call these standards “competing standards,” and how do you think this competition may segment the industry? This is question is for all presenters.
Here’s the interesting quotes I received:
“We are each a standard, yes, but we have full agreement within our bodies of knowledge. We are each filling a niche that needs to be filled.”
“I would agree with [the previous presenter]. We are in full agreement within our bodies of knowledge.”
“We are not competing standards. We agree, and are of benefit to each other.”
[Some time passed.]
Q: I was not aware that there was an agreement between the bodies of knowledge within each certification — if there is, I think these agreeing bodies of knowledge will greatly benefit the industry. What way do you think is the best way to promote this agreement? I see it as a conflict of interest to say, “We agree with [that certification over there], but you should take our certification anyway.” What is your take on that? This question is for all who want to answer.
“I don’t think any of us want to answer that question.”
[There was some sifting for answers here.]
“We have agreement, and we show it on our website. Our certification allows courses from [other certifications], and it’s on our website because we want to promote the benefit to the industry.”
“No, I’d guess it’s on your website because it’s a great marketing campaign. I didn’t think of that [for our certification]. We may use that.”
[After pressure from the audience increased: ]
“We’re in it for the money.”
“Ya.”
“I agree.”
Q: How do your certifications assess those who take it?
[There was some discussion.]
“If you sit in front of me in the class room, why do I need to put a question in the [certification] test? I have taught you. I have taught you and you have learned it.”
Q: I’ve worked at [Company X] and [Company Y] for about six years, and both companies are well respected within the industry. It doesn’t seem like any of the well respected companies like Microsoft, Google, Apple, Adobe, Amazon, etc. even pay attention to these certifications. In other words, it doesn’t seem like people with real experience care. Who is your target audience for these certifications?
“People who are currently looking for a job, and can’t find one. Also, people who are unsure if testing is the profession for them.”
“If you’re the type of person who learns from reading a book, I would recommend not being certified because it would be a waste of your time.”
Q: How does your certification program allow for inclusion of new techniques within its body of knowledge?
[The above question was hard for the audience member to ask, because the audience member was interrupted frequently. During the interruption, techniques such as exploratory testing and model based testing were mentioned. Tension increased.]
“Exploratory testing is an ad-hoc process, and is a simple technique.”
“Our courses teach every technique on earth about exploratory testing! It is so basic, it is in the basic course requirements!”
[Remember: There's full agreement between each certification's body of knowledge. Right...]
“I don’t see why you’d want any new techniques within a certification. What new techniques have we had in the last 30 years?”
“If I wanted to teach model based testing in a course within my certification training, I don’t have to get approval from the board because model based testing is not within the body of knowledge.]
“Putting model based testing in the body of knowledge would be a dumb thing to do.”
[This caused great tension.]
Q: So if we haven’t had any new ideas in 30 years, then who could possibly come up with a new idea???
“Glenford Myers.”
[This was not a joke. The presenter was serious when he said this!]
[End.]
I don’t know about you, but I’d really like you to take a good look at the current certifications in software testing, and in related software engineering fields. I believe that every certification (excluding OpenCertification) lost much credibility during this tutorial. My only wish is that more people were there to see it.
In my sole opinion, as a member of an industry that so badly wants to speak for itself: This was a great day for software testing. Whatever your take is on certification — either for it, or against it — there is no doubt that this debate was helpful to the industry.
A big thank you goes out to all who were involved.
Have you ever had one of those times where you say, “Wow, I actually feel like part of the gang?” Ya, I had mine a few days ago at WHET.
WHET stands for the Workshop on Heuristic and Exploratory Techniques, and was recently held at Quardev Laboratories in Seattle, Washington. The topic was Boundary Testing, a look at how exploring boundaries could lead to rich, interesting bugs.
WHET is a peer conference modeled in the popular LAWST style that seems to be overtaking the industry. The conference is made up of a selected group of 15 to 20 people, each tasked with sharing their experiences with the rest of the group. Although this was the sixth peer conference I’ve been to — which, in my opinion, is a good record for my ripe old age of 22 — I didn’t feel like I totally belonged.
My role, in all it’s glory, felt like a student’s role. I just graduated (so I wasn’t actually a student), but I didn’t have the rich cache of experience the others did. Because of this, I couldn’t confidently sit with the group of attendees and say “Ya, I’ve had those experiences too.”
Throughout the workshop, we talked extensively about boundaries, and argued all the different ways you can think of them, express them, find them, and value them. I was overwhelmed with the mass of information presented to me, and I didn’t know how to take it all in. Because I had little experience, I opted toward staying quiet.
Boy, was this the wrong idea.
While listening to every bit of discussion that I could wrap my mind around, I silently came up with these insights. Some of these are abstractions from the material presented, others are my own. All are up for debate.
I was planning on staying quiet until end of the workshop, but I was forced to talk on the very last day. Rob Sabourin, the co-content owner of the workshop, asked me to give the group my perspective in the very last lightning talk. He wanted to “end it with something cool.”
I wanted to tell Rob that my perspective wasn’t going to be great. I wasn’t nervous, but I didn’t think my experience would be that useful. Heck, I barely understood half of what was said throughout the workshop. What use could it be?
To my own surprise, I was given the “ooh” and “ah” version of a standing ovation. They loved it. This group of experienced people actually had use for what I had to say, and then thanked me for contributing my perspective!
This felt like a huge coming of age. I was a student when I came into the workshop, and was a useful contributor when I came out. A huge hurdle had been jumped over. I overcame a boundary.
This boundary was unknown to me. It was imaginary, and was hindering my ability to contribute to the group. The problem was, I decided not to test it — I decided to stay quiet, and simply fit into the role I thought I was bound by. My own self confidence stopped me from being what I wanted to be.
Thank you Rob, for helping me overcome that.
Update, 7/11/07: There’s someone else who’s actively trying to break the student barrier for me, and I believe he deserves credit here as well. Jon Bach, the other co-content owner of WHET, has intentionally tried to treat me as a member of the group (instead of a student) and has actively encouraged others to do so as well. If there’s any indication of what it’s like to attend a peer workshop, this would be it. Thank you Jon.
UPDATE, 12/22/07: This page is obsolete. Please see my new post here.
So, I’m visiting my parents in Atlanta, and I finally have some free time. (You know how the parents are — they go to bed early. ). I’ve been wanting to get at this blog post for some time, and now I’ve finally found time to do it.
A while back, I found myself wanting to write something like a wiki in Ruby on Rails. Instead of reinventing the revision/diffing wheel needed to support a wiki, I wanted to use the Subversion Version Control system as a backend to my program. I searched far and wide for some documentation, but found nothing; all I found were sites asking for what I’m about to write. Since I couldn’t find any documentation, I looked to the unit tests. This turned out to be exactly what I needed.
From looking at the unit tests, and playing around with Subversion for some time, I found there are two states in which you can access a repository. One state is the client side, where you have a directory of files that should match up with the files in the repository. The other state is the server side; this is the side that receives commands from the client and executes them. Since I wanted to use Subversion as a backend to a program — and I didn’t want to maintain a separate list of files — I had to figure out how to send direct commands to the server. The following is a list of what I learned, and shows how to control a repository through the Ruby programming language.
Installing the SVN Bindings:
There are many sites that tell you how to install the bindings; I won’t be covering that here. This might help: http://collaboa.org/docs/svnbindings/install
(For Ubuntu users, just ‘apt-get install libsvn-ruby’)
Creating and opening a repository through code:
This is easier than printing “Hello World” in assembly. It’s that easy:
Svn::Repos.create(“/path/where/repsitory/should/be/created”)
If you think about the way Ruby works, then the above command shouldn’t be that complicated. Both Svn and Repos are modules; Repos is a child module of Svn. The create method is a method inside the Svn::Repos module.
Opening a repository is just as easy:
Svn::Repos.open(“/path/to/existing/repistory”)
Opening a repository is like opening a file for reading and writing.
Now, both the open and create commands return a repository object (this is not the official name) that you can interact with. To do anything useful, you should store the returned object into a variable, like this:
repos = Svn::Repos.create(“/path/where/repsitory/should/be/created”)
repos = Svn::Repos.open(“/path/to/existing/repository”)
We will use the repository object returned from the above two methods throughout the rest of this post.
Commiting a file:
When I first started writing my wiki-like program, the thing I was most interested in was commiting a file. This isn’t that easy to do, but it’s not hard.
The internals of Subversion are just guesswork to me, but generally, most transfers are done through transactions and streams. I’m not positive what a transaction is to the system (I know what the word means) but it seems to be that, whenever we open a new transaction, Subversion’s internal revision counter increases by one (this is important, as I’ll explain later). A stream, on the other hand, is much like a stream that you’d find in most programming languages; it lets us write to the repository as if we were writing to a file system.
Here’s how you’d commit a file using transactions and streams:
repos.fs.transaction do |txn|
checksum = MD5.new(data).hexdigest
stream = txn.root.apply_text(“/internal/path”, checksum)
stream.write(fileData)
stream.closeend
Note that it’s important we think of a Subversion repository as its own separate file system. All paths within the filesystem start with a forward slash (“/”) and are very similar (i.e., the same) as UNIX filesystem paths.
Getting contents of a file within the repository:
It’s great that we can commit files to the repository, but it doesn’t do us any good if we can’t get that data back out. Here’s how you’d do that:
stream = repos.fs.root(revision_id).file_contents(path)
stream.read
There are two things interesting about the above command: 1) We’re using streams, like before, but not within transactions, and 2) there’s a revision_id variable. Basically, the above command gives you the contents of a file for any revision, just as long as you tell it which revision you want. If you don’t specify a revision id, or you pass in the value of nil, the repository will give you the data corresponding to the most recent revision.
Note that the revision_id variable has some exactness to it, which may bite you in the butt later. If you pass in a revision id, you must pass in a revision id where the file you want was specifically edited. What does this mean? Say you have two files in a new repository, and each one was edited once. The first edit would push Subversion’s internal revision count to 1, and the second edit would push the count to two. If you use a revision_id of 2 to get the data within the first file, nothing will be returned because the first file was not edited on the second revision. If you want the data from the first file, you have to be specific; in this case, you’d use a revision_id of 1.
So how do you know which revision id’s correspond to a given file? Read on.
Figuring out the revisions of each file:
All we need to do is run this command:
repos.file_revs(path, 0, repos.fs.youngest_rev)
All this says is “Give us all the revisions between the initial revision (0) and the most recent revision (repos.fs.youngest_rev) corresponding to the file at path path.” This returns a lot of data.
What you get back is an array of arrays; each of the inner arrays represents one revision within the repository. Each revision has three parts: the path, the revision id, and the date the revision was made. The path is the same path we passed in (it seems redundant, but it’s actually quite handy), and the revision id and date are String representations of what their names imply. Since the revision is represented as an array, these values are at index 0, 1 and 2 respectively.
Note: The date value returned may not be what you expect. Read the next section to figure out why.
Getting the date of a revision (or, getting any property for that matter):
Whenever you commit something — or more importantly, whenever you create a new transaction — Subversion automatically stores the time at which the transaction occurred. It’s easy to get by calling this command:
date = repos.prop(Svn::Core::PROP_REVISION_DATE, revision_id)
The prop() function returns any stored property for the revision specified by revision_id; we’re using a Subversion-specific constant to ask for the date.
Note that the date returned is a String representation of the date; the prop() function only returns strings. However, we can turn it into a Ruby Time object by doing this:
Time.parse(date)
(For you advanced Ruby programmers, and all those who read the last section, the Subversion bindings actually override the Time.parse function to handle the output from the Subversion repository. Cool isn’t it? )
Setting the author for each revision:
Sometimes it’s nice to tell the repository which author made which revision — like, say, when you’re committing files. Since a new revision is created each time we create a new transaction, we can put the following statement into the transaction code given above:
txn.set_prop(Svn::Core::PROP_REVISION_AUTHOR, author)
We again use a Subversion specific constant, but this time, we’re setting the data instead of getting it. The author variable holds the author’s name as a string.
Note that if we wanted to get the author back out later, we’d do the exact same thing we did to get the date, except we’d use the author constant instead:
author = repos.prop(Svn::Core::PROP_REVISION_AUTHOR, revision_id)
Getting the diffs from one revision to the next:
Sometimes, especially in a wiki, it’s handy to show the user the differences from one revision to the next. It’s easy to get the diffs from the repository, but it’s a little complicated to understand what comes back.
To get the diffs, do this:
info = Svn::Info.new(“/path/to/repository”, revision_id)
diffs = info.diffs
Subversion has a separate construct for getting specific revision information. This is the Info construct. We create a new Info construct to get the diffs for a certain revision. The thing that’s different about doing things this way is we totally bypass the repository; or, at least, we don’t use the repository variable we created before. Instead, we pass the repository path and a revision id into the Info constructor, and we get what we want back out.
Well, actually, I lied — we don’t get what we want back out. Instead, we get a lot more than what we wanted. The diffs variable above will contain a hash of all the diffs for the given revision, where the key represents a file path within the repository, and the value is another hash of data corresponding to that file. This second hash — the “value hash” — contains either an :added or a :modified key. These keys are Ruby symbols, and, depending on which one is present, the file was either added or modifed. The value associated with each of these keys is a Subversion DiffEntry object. We don’t care about this object, per se, but we do care about its body instance variable; this variable will give us a unified diff of a certain file showing the differences from one revision to the next.
Can it get any easier?
Now, I know some of the above functions may seem complicated or tedious, and that’s because they are — there’s no getting around that. However, I’ve written a wrapper class that does some of the tedium for us. Download it here: svn_repos.rb
EDIT, 4/2/07:Â The wrapper class linked above is old. A newer one can be found within the acts_as_subversioned plugin project, located on RubyForge.
Note to Ruby on Rails users:
My next post will be a howto describing how to use the wrapper class within a Ruby on Rails project. I’ll be giving a Rails plugin for download, which includes a test harness to test Subversion repositories like you test MySQL databases; that is, using fixtures! Believe me, it’s awesome. (Because I’m busy, I have no guarantee when I can get this out. However, I hope it to be soon.)
I’d love any comments you readers can give me on this. It feels great to finally post again.
