testjutsu

Jeff Atwood recently wrote a post about phases of software development/release. He wondered where the terms come from and why we rarely use ‘gamma’ – I wondered if these terms actually have any relevance to what actually goes on in software development anymore.

To me, these terms are used to broadly group stages of software stability as simplifiers to people who don’t necessarily understand the finer details of software development (or don’t particularly want to). These labels seem to equate these days to marketing wank for shops that still do big design up front. It’s easier to say ‘we’re at alpha release’ than saying ‘we’re two-thirds feature complete with no current showstoppers, have a bug trendline that indicates a find-rate approaching zero and we’re satisfied that testing to date has adequately mitigated the risks identified so far’.

My issue with these terms is that they oversimplify a process that should be viewed as difficult. Alpha/Beta/Gamma (or whatever other pre-release signifiers you like) imply a linear, production-line progression of software. Anyone that has worked on any project of significance knows that the development of software is anything but. I worry that this oversimplification helps tie us in to a flawed mental model that invites us to fail over and over again.

There is no guarantee that half way through ‘beta testing’ your testing team won’t find a showstopper that requires you to go back to pre-alpha. How well does your ‘A->B->Ship it’ model stack up then? It seems that because we have this simplified mental model of development, decision makers think that a) we can reliably estimate up-front the time and budget required to put together complex systems and b) when, close to completion neither of these estimates prove even remotely accurate, makes them insist on kludging their way through so that the immutable ship date can be met.

The problem with this mental model is that while it probably works well for automatons that churn out identical widgets time and again, software development is not that. Software development is about human beings turning out solutions to difficult problems in a non-uniform manner. Human beings each with their own individual strengths, weaknesses, passions, neuroses and everything else that makes us human. The fact that these people are not producing identical widgets day in, day out means that the production line model is entirely inappropriate, as is the model of big design (and estimation) up front.

This is a can of worms that you could (and some have) write a book about. Entwined with development is software testing. Another very challenging craft done by humans. Testing is an oft-misunderstood profession. The alpha/beta/ship it model doesn’t help. I worry that this model locks people into thinking that testing is something that happens once the developers have finished doing what they do, at which point the testers are left to find whatever bugs they are able to in between that point and the shipping date.

Then, depending on the nature of the project and the people they’re working with, they’re possibly blamed for finding too many bugs, or flaws that, if properly fixed, would extend the length of the project well beyond the date where the PM gets his nice fat bonus for bringing this thing in on time. The PM shoots the messenger because what other choice is there at that point?

This is a stupid way to make software. This is a stupid way to think about making software.

Scrum appears to be intended to make transparent the human issues that crop up in software development and it sounds great in theory, but I wonder how many software shops are prepared to invest in the up-front discomfort of changing over and are prepared to have (on a daily basis) the open, honest and frank dialogue necessary for a framework like that to work. My guess is not many. This is perhaps a subject for another post.

If you want to be an extremely effective software tester, I highly recommend you do something else. Really. I’m not talking about being able to complete Halo3 in world-record time. Specifically I mean you should find something that you are passionate about; that takes practice and perseverance to become proficient in, and then master it.

I’m not talking about ‘I have nothing more to learn from anyone’ arrogance. Anyone who has mastered anything knows there is always more to learn. Always. What I mean is that you should do it at least until you can confidently teach someone else without fear of leading them astray.

Things like playing a musical instrument, speaking a second language, ikebana, playing golf, it doesn’t need to be remotely testing related or even related to computers. It may be better if it isn’t.

There’s something interesting that happens when you master an art. You can recognise that same mastery of something in others, even when what they do is completely different. When you understand the sort of effort required to reach a certain proficiency, you tend to appreciate it wherever else you find it. There is recognition too, of the many stages of learning one goes through on the way to mastering something. You see it in others because you yourself have been there.

How do you recognise that same thing in others? Because once you have mastered something, it reveals itself in everything that you do. Your experience cannot help but inform you about other things, and influence the way you take action.

How will that make you a better tester?

If nothing else, it will make it easier for you to play the simile game. X is like Y. It’s an effective way to learn something new – relate it to something you already know well. It can give you a different way of thinking about things. Paradoxically, any art worth mastering will have concepts that are unique to that art, that don’t directly translate into anything else. Opening your mind to new concepts can help you look at testing problems from angles you may not otherwise have considered.

Recognising various stages in learning allows you to become a better teacher. If you can recognise where someone else is along the path, you can more effectively tailor your guidance to be more meaningful. There’s no point talking about concepts they’re going to have no clue about if what they need is a simple, straightforward explanation that leads them to a deeper understanding.

So I maintain that if you really want to become a great software tester, be great at something else.