testjutsu

I began writing this entry and almost immediately discovered this entry from Michael Bolton on emotions and oracles, which explains things probably more eloquently than I am trying to. I can’t really speak as to why emotion isn’t valued by some testers. I haven’t encountered any situations where this has been the case - I live a sheltered life, apparently. It seems obvious to me that emotion is a useful indicator when testing. I have no problem testing with emotional detachment, but when that state changes, understanding the reason is important. I haven’t encountered a situation where this would not be the case.

In kendo, there is said to be four ’sicknesses’ that will lead you to defeat (or the exploitation of which will allow you to defeat your opponent). The four sicknesses are doubt, confusion, surprise and fear*. One could argue that there are many more emotions that you need to pay attention to. I would argue that the majority of them are a reaction arising from one of these four states. Ultimately, it doesn’t matter all that much as long as you recognise that the emotion being provoked is useful to you.

It is important to recognise these states within yourself (so you can correct it), as well as to strive to engender them in your opponent (so you can take advantage).

In terms of testing, the same is true. Whether you are analysing requirements, evolving your test plan, doing the grunt work, reporting or anything in between.

If you are testing something that an end user will use, the sorts of things that provoke these responses in you are likely to do the same for them. If you can recognise when something provokes one of the four sicknesses, then stop and examine what and why. It might be bugworthy, it might just be one of those things you mip, but at least it’s something that you’re aware of. Recognising emotion in yourself is useful for testing.

It’s unlikely that you’re going to cause actual surprise and fear to your system under test, though you might stretch the analogy as far as doubt and confusion. Do something unexpected and see if the system handles it (or gets confused). It might be for example, renaming a binary file extension then opening it in a text reader, pulling out a network cable halfway through a transfer, I’m sure there are plenty of examples you could come up with.

The point is, not only are emotions a useful indicator that something is an issue, they’re also a useful heuristic for creating tests. How might I cause confusion in this system? What is going to scare stakeholders? How sure are the developers that the functionality maps to the spec (or needs to)? Engendering emotion in others is useful for testing.

If you’re not paying attention to your emotions and the emotions of others, then you’re robbing yourself of opportunities to be a more effective tester.

__
* one nerd point to anyone who thought of suggesting ‘an almost fanatical devotion to the pope’, redeemable for one being hit on the head lesson.

Sorry about the heading, I’m just not feeling particularly clever today.
A friend recently lent me his Nintendo Wii. I wanted to test-drive one to see if ‘wii sports’ would be helpful as a tool for keeping fit or warming up before I get into some real exercise. I was initially hopeful. The boxing game looked good. Reasonably intuitive and quite a good workout if you make the conscious choice to move your body as you play.

Then as the difficulty increased and the basic jabs, punches, ducks and weaves failed to be as effective against the stronger opponents, I discovered I needed to learn the other moves, hooks, uppercuts and combos thereof. That’s when disappointment set in. While the movements for jab, punch and dodging feel ‘natural’ (ie moving my body one way causes the same movement in the sprite), the other moves are more or less a matter of flicking the controllers sideways or upwards with the wrist - there doesn’t appear to be a natural correlation between an actual hook or roundhouse punch performed by me, and the same happening on screen. I worked out that I can flick my arm outward and that requires effort, but doing that for any sort of extended period is going to leave me with screwed rotator cuffs. No thanks. Still disappointed.

I know I can beat this game by working out when to flick my wrists around. That’s just pattern recognition and application of the appropriate response. If I want that, I’ll fire up the xbox and play gears of war. At least the sexy graphics, gunfire, blood and assorted mindless violence will distract me from the fact that I’m engaging in said pattern recognition.

I want to have to move. I want it to be an effort and I want there to be some semblance of parity between what I’m doing with my body and what happens on screen. Essentially, I want it to be shadow-boxing without the tedium. Based on wii sports, Nintendo isn’t getting my money yet. Maybe what they need is multiple motion sensors. Maybe you’d only need two to triangulate movement, I don’t know. One appears to be insufficient.

Since that disappointment, more and more I’m noticing other annoyances with the interface design. Using the controller to point at buttons and such is okay, but not great. Sure there are defaults, and they’re right most of the time, but when they’re not, I don’t want to dick around with pointing at precisely the right button that I do want. I’m not concerned with being particularly dexterous at that point, I just want to get to where I want to be. The buttons are ‘magnetic’, in that the pointer is drawn toward them when it reaches a certain proximity. That’s a nifty little feature, but surely using the corners and edges of the screen would still be faster.

Is the application of Fitts’ law such a stretch? It’s not bloody rocket science and it’s not like they’re new to interface design. It wouldn’t even be that much of a stretch to keep track of the four most commonly chosen options and automatically put those in the corners while keeping the others centered (or even on the edges, where they have unlimited screen real-estate in at least one direction).

Anyway - wii as a fitness tool? Not just yet, at least not with wii sports. Might be time to check out eyetoy kinetic.

katte, kabuto no o wo shimeyo.
When you win, tighten your helmet strings.

This saying is a reminder to remain ever vigilant. In battle, victory over an opponent should not be a signal to let your guard drop. If anything, it should serve to remind you to be even more wary. Defeating one is no guarantee that there aren’t more around.

Have you ever signed off on a product that you were confident was sufficiently tested only to find out there was something you missed that you might have caught if only you’d thought about it just a little bit more (or just a bit differently)? Sure you’re never going to find all the defects, but the ones that burn are the ones that you didn’t find but know you could have.

Time and circumstances permitting, when you’re ready to sign off on your test effort, stop and think about it for a little while. Ask questions, either of colleagues, or of someone completely unrelated - you’d be surprised how often someone with a fresh perspective points out something obvious. After you’ve been testing something for a while, there’s a tendency to do the go-to tests, look at the usual suspects - to maybe coast a little. Especially if your test suite has been coming back green. It’s really at this point you need to tighten your helmet strings.

Are there any new risks you can think of? Can you check the reliability of your oracles? Have you covered the important areas? Sufficiently? How do you know? (and on and on)

I’m not trying to give you the answers. I am saying that when things look like they’re going swimmingly, you need to be asking yourself the hard questions - and making sure you have answers that stand up.

Those of you who have seen ‘Dead Poets Society‘ may remember Robin Williams exhorting his students to stand on their desks in order to show them the importance of having multiple perspectives of the world. I always thought that this was a most excellent scene. If you haven’t seen it, consider checking it out (or check out the screenplay in the link and grep for ’stand on my desk’).

I had my own ’standing on my desk’ moment after I recently took up nito (fighting with two swords), after many years of wanting to. Kendo is traditionally done with one sword from a stance called chudan no kamae. There are four other defined kamae, but chudan is by far the most prevalent.

Chudan is a highly versatile kamae with both strong offensive and defensive aspects. The range of techniques one can execute from chudan is far greater than any of the other kamae. It is also easier to judge the distance between you and your opponent, and execute multiple attacks from chudan. The consensus that I am aware of, seems to promote a high degree of proficiency with chudan before considering moving on to other kamae. This is something I agree with. I’m not saying you can’t be effective if you learn another kamae before having a good understanding of chudan. There are numerous examples to the contrary.

However, I do think that if you want a good understanding of chudan no kamae as well as others (as opposed to being a specialist in one kamae), then having an understanding of chudan before moving on is paramount. If you’re going to use another kamae, you’ll still be coming up against chudan kendoka more often than anyone else. Understand thy enemy.

In the short space of time that I have been practicing nito, I have gleaned numerous insights into my itto kendo that I might otherwise not have seen. For that alone, practicing nito has provided a valuable alternate perspective of my itto kendo. I think one of the keys to having these insights to begin with is the understanding of itto kendo, more specifically chudan no kamae. If I didn’t already have a good understanding of itto kendo, then very likely doing nito would have been more of a hindrance to my learning than a help.

As it stands, I have a decent understanding of distance and timing and what constitutes an opportunity for attack. These things differ subtly (but importantly) between nito and itto chudan. Because I understand one, I can relate them when I encounter them in the other. Each kamae informs my kendo about the other and I believe, makes my kendo better for it overall.

For example, nito uses one long sword, and one short. Typically, the short sword is held in front as a foil whilst the long sword is held overhead. When two itto kendoka engage in chudan, they judge the distance from each other by where their swords cross. That distance is different for nito such that if a chudan kendoka crosses with the short sword, they are actually dangerously close, and the nito kendoka can simultaneously cast their sword aside, and strike with the long sword.

Moreover, the nito kendoka can extend or retract the position of the shortsword to further confound the opponent’s sense of distance. Thinking about it - there’s no reason you couldn’t also do the same with one sword, and there are techniques where this occurs, but I’d never really consciously thought about it before. It makes me wonder what other insights I’ve missed by not allowing myself a differing perspective in other walks of life.

I think there are certainly parallels to be drawn as a software tester. If you think you have an excellent understanding of something, find a way to change your perspective, to challenge the ancient wisdom (such as it is). If you think you understand how a particular programming language works, learn about the underlying architecture that it runs on - see how the performance or the operation of the code differs from one to the other and why. If your product has competitors - check out their product. What does it tell you about yours? You are only really limited by your imagination. If you have trouble coming up with something, try standing on your desk.