testjutsu

Besides also getting to hit people with a stick? Plenty – eventually.

As a beginner in kendo you generally start by learning the body mechanics, repeating the same movements over and over while your teacher continually corrects you on what seems like an endless stream of minutiae.

It’s tough at this point – you’re not quite sure why you’re doing what you’re doing, and you can see off to one side the more experienced students spiritedly attacking one another in what looks like a free-flowing and even random manner, and you wonder if you’ll ever get there.

Gradually though, your body learns and you start to understand why your hands and feet must move just so. Eventually you strap on the armour and it’s like starting over again. Your hands feel weird holding the shinai through your kote, your field of vision is restricted by the men gane, and when you finally add an opponent into the mix, it adds a whole new element to the entire dynamic. Having someone screaming at you and trying to hit you with a stick is not what we usually identify as an environment conducive to learning.

At some point down the track, you face off against someone and your body knows what to do. You have internalised the body mechanics your teacher spent months drilling into you, leaving you to explore how to implement it. You attack your opponent. Sometimes you’re successful, sometimes not. Sometimes they counterattack. Sometimes they confound you with their ability to evade even your most spirited attacks.

The day you achieve a real connection between you and your opponent is the day a new world opens up to you. There comes the realisation that knowing how to strike is not enough – more important is knowing when and why. When you achieve that connection with your opponent, you engage them in a battle of wits and will.

There is an ebb and flow to a match. You can feel it when you have the ascendency, when your opponent does and when it’s in the balance. You engage in a chess match to outmaneuver and outwit your opponent in order to strike them.

Eventually, you realise that a strike is successful not because you have hit the target, this is only a physical manifestation of success. The strike is successful because you have won your chess match and like the final move of a chess match, by the time it happens, there can be no other result.

If you like physical activity and you like to use your brain, then you’ll probably enjoy kendo. I was contamplating adding a software testing analogy in here, but what the hell. You software testers out there can draw your own parallels.

In testing, I think there is a tendency to confuse repetition for the purpose of learning with repetition as the application of skill. I have heard that some expound using scripts as a means of skilling up unskilled staff. I believe this thinking is fundamentally flawed. The reason being is that there is no clear delineation between the repetition of an action (i.e., a script) and actual software testing, which is far, far more than simply following a recipe.

In kendo, a beginner starts by learning the basic moves. A student might practise a single cut 10,000 times or more. Time and again they are made to demonstrate correct footwork, body position, striking and movement. At this point, they have no clue about the application of these techniques, only how to execute them. They can make a cut, but it doesn’t mean they can consider themselves skilled in kendo.

Once they have the basic mechanics down, they are ready to strap on armour and face a real opponent. Unlike before, where they had specific instructions about what do do, how and when, they now face a situation where the application of their techniques may or may not work and it is up to them to work out why. Moreover, their opponent is attacking them, shouting at them, trying to unsettle them, make them nervous and throw them off their game. Are you software testers out there seeing parallels?

The difference here is that the delineation in kendo between learning basic technique and application of that technique is quite explicit. In software testing, especially where scripting is involved, this is not always so. Herein lies the danger of using scripts as learning tools. A script may help you learn to execute a technique. It will not teach you to be a tester.

It’s easy to tell yourself that you’re training hard when in actual fact you’re slacking off. I find myself doing it far too often for my liking. If in the kendojo I’m fighting beginners, I might fend off attacks in order to get them to try and work out why they are unsuccessful. Nothing necessarily wrong with that, but if that’s all I do, then I’m slacking. More effective for the both of us would be if I tried to fight at a level slightly above theirs and encourage them to rise to that standard.

If I’m not really trying, I notice myself falling into bad habits. I let my feet move move lazily, I block without trying to counter, if the first strike fails, I may not immediately make a second or third effort (all of these rob you of the ability to take advantage of opportunities) – these things are physical manifestations of mental laziness. When I see others being lazy, I take them to task for it. When I find myself doing it, it annoys me. It makes me feel like a hypocrite.

If I don’t put in the effort, then I don’t learn anything, but possibly worse is that my training partner isn’t learning anything, and if you think they don’t pick up on the fact that you’re not putting in 100%, think again. I don’t want to be the person that teaches others that a half-hearted effort is okay. Moreover, by doing this, you are saying ‘you have nothing to teach me’.

You can learn from anyone. This has been proven to me time and again, both in the software testing field and in kendo. Insights can come from the most unlikely sources. Think about the questions people ask (before you rattle off a rote answer). Observe what they do and ask them why.

It’s not easy to alway make the extra effort, but it is something you can train yourself to do. Like anything, there’s that initial force of will required while your brain rewires itself to a new standard, but the more you do it, the more you’re able to do it. Doing it consistently in practise means it becomes second nature when it counts.

Practise doesn’t make perfect, perfect practise makes perfect. Don’t know who coined that one, but they were right.

There are always things you can find to work on. If you find yourself with the luxury of time, or your domain knowledge outstrips the challenge at hand, look for ways to challenge yourself. What is it you are trying to achieve? What can you do to make that achievement something that helps you to grow also?

Not necessarily easy questions to answer. The answers you come up with may not be easy to implement either, but if you can identify something challenging in what would otherwise be routine, then surely it’s worth it.

Forgive me if I’m being overly abstract. I shall try for something more practical in my next entry.

I began writing this entry and almost immediately discovered this entry from Michael Bolton on emotions and oracles, which explains things probably more eloquently than I am trying to. I can’t really speak as to why emotion isn’t valued by some testers. I haven’t encountered any situations where this has been the case – I live a sheltered life, apparently. It seems obvious to me that emotion is a useful indicator when testing. I have no problem testing with emotional detachment, but when that state changes, understanding the reason is important. I haven’t encountered a situation where this would not be the case.

In kendo, there is said to be four ’sicknesses’ that will lead you to defeat (or the exploitation of which will allow you to defeat your opponent). The four sicknesses are doubt, confusion, surprise and fear*. One could argue that there are many more emotions that you need to pay attention to. I would argue that the majority of them are a reaction arising from one of these four states. Ultimately, it doesn’t matter all that much as long as you recognise that the emotion being provoked is useful to you.

It is important to recognise these states within yourself (so you can correct it), as well as to strive to engender them in your opponent (so you can take advantage).

In terms of testing, the same is true. Whether you are analysing requirements, evolving your test plan, doing the grunt work, reporting or anything in between.

If you are testing something that an end user will use, the sorts of things that provoke these responses in you are likely to do the same for them. If you can recognise when something provokes one of the four sicknesses, then stop and examine what and why. It might be bugworthy, it might just be one of those things you mip, but at least it’s something that you’re aware of. Recognising emotion in yourself is useful for testing.

It’s unlikely that you’re going to cause actual surprise and fear to your system under test, though you might stretch the analogy as far as doubt and confusion. Do something unexpected and see if the system handles it (or gets confused). It might be for example, renaming a binary file extension then opening it in a text reader, pulling out a network cable halfway through a transfer, I’m sure there are plenty of examples you could come up with.

The point is, not only are emotions a useful indicator that something is an issue, they’re also a useful heuristic for creating tests. How might I cause confusion in this system? What is going to scare stakeholders? How sure are the developers that the functionality maps to the spec (or needs to)? Engendering emotion in others is useful for testing.

If you’re not paying attention to your emotions and the emotions of others, then you’re robbing yourself of opportunities to be a more effective tester.

__
* one nerd point to anyone who thought of suggesting ‘an almost fanatical devotion to the pope’, redeemable for one being hit on the head lesson.

katte, kabuto no o wo shimeyo.
When you win, tighten your helmet strings.

This saying is a reminder to remain ever vigilant. In battle, victory over an opponent should not be a signal to let your guard drop. If anything, it should serve to remind you to be even more wary. Defeating one is no guarantee that there aren’t more around.

Have you ever signed off on a product that you were confident was sufficiently tested only to find out there was something you missed that you might have caught if only you’d thought about it just a little bit more (or just a bit differently)? Sure you’re never going to find all the defects, but the ones that burn are the ones that you didn’t find but know you could have.

Time and circumstances permitting, when you’re ready to sign off on your test effort, stop and think about it for a little while. Ask questions, either of colleagues, or of someone completely unrelated – you’d be surprised how often someone with a fresh perspective points out something obvious. After you’ve been testing something for a while, there’s a tendency to do the go-to tests, look at the usual suspects – to maybe coast a little. Especially if your test suite has been coming back green. It’s really at this point you need to tighten your helmet strings.

Are there any new risks you can think of? Can you check the reliability of your oracles? Have you covered the important areas? Sufficiently? How do you know? (and on and on)

I’m not trying to give you the answers. I am saying that when things look like they’re going swimmingly, you need to be asking yourself the hard questions – and making sure you have answers that stand up.

Those of you who have seen ‘Dead Poets Society‘ may remember Robin Williams exhorting his students to stand on their desks in order to show them the importance of having multiple perspectives of the world. I always thought that this was a most excellent scene. If you haven’t seen it, consider checking it out (or check out the screenplay in the link and grep for ’stand on my desk’).

I had my own ’standing on my desk’ moment after I recently took up nito (fighting with two swords), after many years of wanting to. Kendo is traditionally done with one sword from a stance called chudan no kamae. There are four other defined kamae, but chudan is by far the most prevalent.

Chudan is a highly versatile kamae with both strong offensive and defensive aspects. The range of techniques one can execute from chudan is far greater than any of the other kamae. It is also easier to judge the distance between you and your opponent, and execute multiple attacks from chudan. The consensus that I am aware of, seems to promote a high degree of proficiency with chudan before considering moving on to other kamae. This is something I agree with. I’m not saying you can’t be effective if you learn another kamae before having a good understanding of chudan. There are numerous examples to the contrary.

However, I do think that if you want a good understanding of chudan no kamae as well as others (as opposed to being a specialist in one kamae), then having an understanding of chudan before moving on is paramount. If you’re going to use another kamae, you’ll still be coming up against chudan kendoka more often than anyone else. Understand thy enemy.

In the short space of time that I have been practicing nito, I have gleaned numerous insights into my itto kendo that I might otherwise not have seen. For that alone, practicing nito has provided a valuable alternate perspective of my itto kendo. I think one of the keys to having these insights to begin with is the understanding of itto kendo, more specifically chudan no kamae. If I didn’t already have a good understanding of itto kendo, then very likely doing nito would have been more of a hindrance to my learning than a help.

As it stands, I have a decent understanding of distance and timing and what constitutes an opportunity for attack. These things differ subtly (but importantly) between nito and itto chudan. Because I understand one, I can relate them when I encounter them in the other. Each kamae informs my kendo about the other and I believe, makes my kendo better for it overall.

For example, nito uses one long sword, and one short. Typically, the short sword is held in front as a foil whilst the long sword is held overhead. When two itto kendoka engage in chudan, they judge the distance from each other by where their swords cross. That distance is different for nito such that if a chudan kendoka crosses with the short sword, they are actually dangerously close, and the nito kendoka can simultaneously cast their sword aside, and strike with the long sword.

Moreover, the nito kendoka can extend or retract the position of the shortsword to further confound the opponent’s sense of distance. Thinking about it – there’s no reason you couldn’t also do the same with one sword, and there are techniques where this occurs, but I’d never really consciously thought about it before. It makes me wonder what other insights I’ve missed by not allowing myself a differing perspective in other walks of life.

I think there are certainly parallels to be drawn as a software tester. If you think you have an excellent understanding of something, find a way to change your perspective, to challenge the ancient wisdom (such as it is). If you think you understand how a particular programming language works, learn about the underlying architecture that it runs on – see how the performance or the operation of the code differs from one to the other and why. If your product has competitors – check out their product. What does it tell you about yours? You are only really limited by your imagination. If you have trouble coming up with something, try standing on your desk.

An oft touted phrase by martial artists is ‘enzan no metsuke’, which translates roughly as ‘looking at the far mountain’ – essentially the gaze you would use to view something very distant. If I am fighting an opponent and I focus too keenly on their sword, or the movements of their body only, then I am likely to miss movement elsewhere, the way they breathe, the way they distribute their weight, or lose the insight I would gain by observing that movement as part of the whole. An opponent who focuses too keenly on one particular area is easily distracted and hence defeated.If instead I observe my opponent’s eyes as though observing a distant mountain, I can use my peripheral vision to observe not only what the eyes themselves reveal, but everything else that is going on. With practise, I gain an wholistic understanding of my opponent’s movements and through that, their intent.

This is, in effect, exploratory testing. I don’t know much about my opponent, so I will observe everything I possibly can in order to expose a weakness, and then having done so, take full advantage. As rapidly as possible, you must determine what the appropriate action is.

Knowing one thousand tests does not make you a good tester, or even a tester at all. Anyone can learn to perform tests robotically. In any given situation, you might happen to run a series of tests, and you may even find bugs, but if you don’t understand why running the tests you did was appropriate, then you are not really testing. Likewise, a kendoka may strike an opponent, he may be victorious, but if he does not understand why his cut was successful, then he has improved neither his kendo nor himself.

When testing a new product, you’re looking not only to ensure that functionality works as described/expected (assuming for the sake of the example that this is what you are doing), but you are also on the lookout for the slightest inclination that something is amiss, or presents an opportunity to be taken advantage of.

You need to be aware that any action you take, and any action that the program makes in response to you can reveal information that you can use, even if you don’t understand why or how at the time. Initially this might simply be to use the product as it was intended, ensuring that the response is within acceptable parameters. Even if this doesn’t reveal errors, doing this should give you a number of ideas about what to try next.

You probably have a range of go-to tests – the usual suspects when testing certain types of things. In Japanese these are called your toku-ii waza (your best techniques). You take all the potentially vulnerable areas you suspect and go to work on them with your toku-ii waza. Because they’re techniques you know, you also know how things react to them, both when they’re working and when they’re not.

Either way, this is more information to go on with. If your toku-ii waza do not bear fruit, you might try instead to disguise your intent. Make one attack appear like another. If I look at my opponent’s head as though I intend to strike there, and then launch an attack, I may induce him to raise his hands in defence, at which point I can strike the wrists or abdomen instead.

The testing equivalent might be something as simple as renaming the extension of one file to another and opening it, or uploading it. It might be using regular expressions or executable script in the url or input form of an online form. If you abstract the concept, there are any number of ways one test can be made to appear as another.

Again, each response to each test you conduct tells you something more about the code under test. You must have enough focus that you are able to observe and take in the information you see, but enough detachment that you can correctly interpret it in a way that allows you to learn more. What I have described above is but one of many possible paths you might choose. The importance is the consciousness of that choice being appropriate to the information that precedes it.

One of the things I really appreciate about kendo is that we don’t wear coloured belts to denote rank. If I have never fought someone before, I have only what I can read from them to go by, be it how they carry themselves in the dojo, how they dress or the information I can gather from them when we cross swords. I am unfettered by any preconceptions I might have if I was able to see the colour of their belt. Because of this, I am free to attack them with full spirit and I am challenged to use my understanding of kendo to gauge their skill and fight them accordingly.

Testing is no different. When you are asked to test a piece of software you have never seen before, you should form your own opinion of the program’s abilities and complexity. Sure you may have heard a bunch of stuff about the program, but you should be prepared to discard or disregard it, especially if you suspect it may hinder the freedom with which you approach your testing. If you have been told that a piece of software is horribly complex and will do your head in, your response may be to try and perform tests of equal complexity or greater in an attempt to maximise coverage. You might also overlook the fundamentals of your craft – the simple things that can give you greater insights if not immediate results.
As an example, you may have been asked to compromise the security of an online login interface. ‘It’s super secure. You’ll never break in’, says the client. You might go to work with dictionaries and all sorts of wonderous script-kiddy play toys, but if you go back to basics you might learn something interesting. For instance, you might try logging in with ‘guest’ as a user/pass combo. If the test team is security lax, you could try combinations of ‘test / 1234′, or even ‘ or ” = ‘ to see what the db behind it will do. You could use the names and common details of the company itself, the company’s owner, executive team and or board members. That information is easy to come by. I’m not advocating you go try this stuff on the first online login you find, merely saying that simple is good. Fundamental is good. If it doesn’t give you results, perhaps it gives you something to work with.

Observe for yourself. Use what you have heard to supplement your understanding, not as a substitute for understanding. If in doubt, go back to basics.

At the other end of the scale, your preconceptions might lead you to believe that a relative novice will be a walkover. Sure they might only have a white belt on, but unbeknownst to you, they may have been training for years in several other martial arts. Looks can be deceiving. If you assume that someone will be a walkover, you might be right 99% of the time, but for that one per cent you are wrong, you *will* get your arse handed to you.

The same goes with testing. Something may at first appear very simple, take for instance a three radio button interface to a code profiler that switches between active, passive and coverage modes. Nothing too complex there. It looks like a white-belt, acts like a white belt – you switch modes, run the profiler, switch mode again, run again, observe, repeat – nothing too brain busting about that right?
Well, it depends. Maybe it is that simple. Maybe it’s just toggling a bit somewhere to tell the code to use different options. As soon as you assume that this is so, you disregard the possibility that the interface uses a very different style of code-fu for each option. If you don’t adapt your testjutsu to account for that possibility (and it’s 1 per cent time), there may be any number of people lined up to hand your arse to you.