testjutsu

So CITCON is done and dusted. I met a lot of really interesting people, some of them doing some very cool stuff. Saturday turned out to be a long-ish day and quite a fruitful one. I’m going to break down some of the stuff I picked up in the five sessions that I attended in another post, but I thought I’d post a couple of things while they’re fresh in my mind. There was quite a heavy automation focus, I suppose as you would expect at conference for continuous integration. I hadn’t spent a lot of time around people who do more automation than manual testing. Some of the opinions I heard were enlightening.

In a session on testing web apps, there was a demonstration using grails to quickly build a testing framework from which to build automated tests, as well as building a domain-specific language from the method level up to complete actions in the end-user’s natural language. One tester in the group said ‘this stuff blows my mind - are you guys testers or developers?’ - it became clear that the majority of the people in the room were developers. One of them said ‘we wouldn’t expect our testers to get this stuff. We might present them with a domain specific language to help them test it’ - and there was general assent from the other developers gathered.

I didn’t say anything at the time, but I thought it was a real shame that they felt the need to ‘dumb it down’ for the testers. There appeared to be the assumption that testers have not the skill to be able to handle something of this complexity. It made me wonder how widespread the assumption that ‘testing is a poor cousin of development’ goes. Of course, there is merit in being able to group such actions if testing those grouped actions were what you were trying to test, but there just seemed to be this implication that they deemed their testers incapable of doing anything more.

In another session, one participant wondered aloud how to get developers to do testing when testing was like doing the dishes. Developers were the cooks of the analogy and testers apparently the kitchen hands. I put my $0.02 in at this point and suggested that a) they ditch the ‘washing dishes’ analogy and b) that until the developers understood that testing was a highly cerebral task requiring a different mind set, then they probably wouldn’t be all that motivated to learn more about it.

It saddened me a little to think that there was such a wide comprehension gap between what some developers seem to think testing is, and what I understand testing to be. In my experience, good developers like to create things. They like to solve problems. They like the challenge of coming up with an elegant solution to something non-trivial.

Testing is not so dissimilar. We like to be creative too. We may have a different set of problems to create solutions for, but software testing is no less mentally taxing, or challenging. Developers tend to be thinking largely along the lines ‘how do I make this work’, whereas a tester’s mindset is more along the lines of ‘what could go wrong? What might happen that is unexpected and/or unwanted?’.

If you as a developer don’t see the merit in writing or performing tests, try asking yourself what might possibly go wrong with the code you’ve written. Ask a tester to read it - ask them what they think could go wrong with it. See what they come up with that you don’t. Oh, and if the answer you get is ‘Wow, I wouldn’t even know where to begin testing that’, then it might be time to do some refactoring

There has been an interesting thread going on in the yahoo software testing group to which I subscribe. The topic began as a discussion about software development as art, but was quickly shoehorned into a discussion about productivity. Jared Quinert blogged a response to this recently which got me thinking again about tester advocacy. I haven’t made much progress on the code of conduct front (though the principles for the practise of craft resonate strongly with me), but I think being proactive in the education of others is up there on the list, and the discussion around the meaning of productivity helps to highlight it.

For those that are too time-poor, a potted summary of Jared’s post might read ‘productivity - appropriately defined - is the key to defensible testing. Problems arise where productivity is seen as synonymous with output / effort as opposed to the gathering of helpful information’.

I agree with this, and particularly with the qualification ‘appropriately defined’. I think the danger of (mis)applying the term ‘productivity’ to testing is that the audience expects a ‘product’ at the end. Some sort of artefact that they can point to as proof that testing time was time well spent.

In my experience, the test report is often the thing by which tester productivity is measured. Not necessarily the testing that the report represents, but the report itself. Is it then not incumbent upon the tester to ensure the content of the reporting clearly displays the value of the test effort? Both the actual interaction with the code as
well as the thought and planning that goes around it.

In assisting the creation of a software product, we are generally measured by our productivity towards that end. If the people we are reporting to have a misdirected sense of what productivity means in relation to testing, I believe the responsibility is ours to correct it. How that happens really depends on who you’re reporting to and what they’re looking for.

If the thinking goes ‘a developer produces code, then what does a tester produce?’ - then the tester in my opinion is probably being set up for an unfair comparison. If I were forced to give an answer to that question it might be ‘information’. I want to make damned sure
that the information they get out of the testing I have done is valuable to them in some way. If that makes me a producer of information in their eyes, so be it.

Another way to look at it is to ask ‘If there were no testers, what else would you not have?’ If your audience can’t answer that, then I think your problems are more fundamental than their understanding of what you produce.