testjutsu

When I hear this phrase it is occasionally followed by a sort of forlorn sense of resignation (and the urge to say ‘well, your design is shit’), but more often by hope that a healthy, spirited discussion is about to take place - especially when that phrase is subsequently followed by ‘No user would ever do that’.

I’m not saying I’m the gods’ gift to usability design or anything like that, but I do think that the phrase ‘It’s supposed to be like that’ is often code for ‘You might have a valid concern, but it doesn’t neatly fit into the model that works for me, so it’s your problem, thankyounext’. To me, it’s an argument on an intellectual par with ‘eh eh eeeeeeh!‘.

Rather than being helpful, it sets up a contrary position that doesn’t invite further (positive) dialogue. Of course, you might use an answer like this as an educational tool or test to see how well people cope with it, but as a valid answer to a concern that a bug exists, it has limited value that I can see.

I am also not saying that the person who invokes the ‘not a bug’ incantation is necessarily wrong. They may have a perfectly valid point as to why something is not a bug, and that indeed the design does make sense. However, to arbitrarily shut down someone’s question closes them to the possibility that the design is flawed, or could be improved (which is also a possibility, no matter how much they may wish otherwise).

Take for example the ‘maximise vs. snap-to-fit agument‘ - That the Windows OS ‘maximises’ windows, and the Mac OS snaps to fit the content. I don’t necessarily want to get into which is ‘right’, but more to have a look at the presupposition in the argument around it. Specifically, the comments of Alex Chamberlain that Jeff Atwood quotes:

This is a textbook example of how Microsoft’s programmers got the original Mac GUI wrong when they copied it for Win 3.1, and never bothered to fix it: there’s no zoom button on Mac OS windows because it’s unnecessary. What you’re mistaking for a “maximize” button is actually a “snap window to size of contents” button. Far more useful and elegant. Once again, Microsoft has no taste and no clue when it comes to the GUI. All that money and Gates has never been able to hire a decent human factors person.

If this is indeed the entirety of the quote, then it is perhaps one of the most unhelpful stances you could take on the matter. Windows is wrong. Mac is right, and more useful and more elegant, M$ is teh suxxorz. Here’s the dogma I’ve been indoctrinated with, I expect you to swallow it without question, end of story.

The presupposition that MS got it wrong leaves no room to explore whether the difference is an improvement. Different isn’t necessarily wrong. Different is different. Sometimes, different is better whether it’s intended or not.

Why is snap to fit far more useful and elegant? What is it about the windows interface specifically that is clueless and tasteless? There are things that I personally am not a fan of in both environments. Why is maximise unnecessary? What specifically needs fixing?

I can see the relevance of having the window maximised. If like me you’re particularly uncoordinated, and you want to use the scrollbar, you might overshoot the thing two or three times before catching it, but with a maximised window you have unlimited screen real estate in at least one direction.

I don’t immediately see why snapping to fit the content is either more elegant or more useful - I’m about to go and have a look, but the difference is, I’m not immediately dismissing the possibility that it could be.

As a software tester, you’re going to eventually come up against people who are threatened or otherwise upset when you challenge their ideas and opinions. Don’t take it personally. They’re just trying to do their job as much as you are yours. Don’t be intimidated either. If you don’t have many years of experience, it doesn’t necessarily follow that you don’t have anything useful to offer. If you have questions, ask.

If you can show the person you’re asking that you’re on the same side, and trying to provide them with a service, then your audience is likely to be more receptive and arguments like ‘follow the gourd!’ can be avoided.

My regular reader might have noticed that I (re)read a book called ‘Excession‘ recently. Penned by a gent called Iain Banks, who in my opinion, can excrete pure gold.

It is a great read. For those of you already familiar with his work, it probably needs no introduction. For those yet to experience it - if you have any time for sci-fi novels at all, then your life is not complete until you have read this one as well as ‘The Player of Games‘ and better yet ‘Use of Weapons‘.

In Excession, he coins the term ‘Outside Context Problem’ that he describes thus:

An Outside Context Problem was the sort of thing most civilisations encountered just once, and which they tended to encounter rather in the same way a sentence encountered a full stop. The usual example given to illustrate an Outside Context Problem was imagining you were a tribe on a largish, fertile island; you’d tamed the land, invented the wheel or writing or whatever, the neighbours were cooperative or enslaved but at any rate peaceful and you were busy raising temples to yourself with all the excess productive capacity you had, you were in a position of near-absolute power and control which your hallowed ancestors could hardly have dreamed of and the whole situation was just running along nicely like a canoe on wet grass… when suddenly this bristling lump of iron appears sailless and trailing steam in the bay and these guys carrying long funny-looking sticks come ashore and announce you’ve just been discovered, you’re all subjects of the Emperor now, he’s keen on presents called tax and these bright-eyed holy men would like a word with your priests.

It even has its own wikipedia entry.

I’m sure that any tester that has been around the block once or twice has experienced a situation where they were blindsided by a problem (quite possibly after the system under test went to production) that made them think something along the lines of ‘Wow - never in a million years would I have thought of that as a possibility’.

The Outside Context Problem (OCP) is a little different to the SEP heuristic, in that rather than mentally deleting problems that appear to be in someone else’s jurisdiction, OCP describes those things that are nigh on impossible to factor into your risk analysis simply because you have no concept of it prior to it happening.

Of course this has particular relevance to software testing - we’re supposed to be the ones who can think outside the box; who can come up with the oddball issues that the developers and the stakeholders and the analysts didn’t think of. So how do we factor in OCPs to software testing?

Personally, I don’t know that there’s a definitive answer to that question. You can factor in extra testing time for that X factor, you can look at similar projects and products where applicable, to see if there were things that they didn’t account for, you can get advice from your peers, you can do all sorts of end-user testing to gauge how they might (mis)use the product and so on, but depending on what it is you’re testing you might have one, all or none of these things at your disposal. They still won’t guarantee you coverage of an OCP.

We encounter OCPs often when we are still learning and hopefully less and less over time, but you can’t ever completely eliminate the possibility of another one coming along. It’s why a one-size-fits-all standard simply won’t work for the software testing industry.

Beyond there being a multitude of known differences to factor in from one project to another, you have any number of unknowables that no amount of planning will allow you to say ‘I’ve covered all the bases’.

In terms of the OCP, perhaps the best you can do is cover your arse. If you can list the risks you have identified and present them to your audience, get sign off that the set of identified risks is acceptably comprehensive, then as long as you have done the best you can with the resources you have at hand, there is not much more that you can do.

That’s probably cold comfort for people working with virulent strains of bacteria or virii, or any sort of system where lives depend on getting it right the first time. You might be able to legally cover yourself, but you still have to sleep at night and I wonder how many people following a catastrophe (that they didn’t see coming) do not wonder ‘Is there something else I could have done?’