testjutsu

I started this post a few times before I realised I was actually dealing with two subjects rather than one. At CAST2010 I took Fiona Charles‘ workshop on Speaking Truth to Power. It turned out that this was one of the subjects that I was writing about and hadn’t managed to articulate at all (Thanks again, Fiona).

The other subject is the situation where you know that a particular subject, product, action etc is not a good one (at least in your context), but don’t necessarily have the familiarity with it to explain why. In order to do so, you’re going to have to learn at least enough that you can speak from your own experience why such a thing is not good.
I call this the ‘know thy enemy paradox‘ – In order to explain why you shouldn’t need to know/use/do something, you need to learn enough about it so that you can coherently explain why.

It would be nice if we could simply take the advice of our learned peers or mentors. No one I respect in the testing industry is going to pan someone or something without a valid reason. If my interest does not lie in that direction, I am happy to take them at their word. As far as directing our own learning goes, this is entirely possible. It becomes another matter however when your line manager proposes that you use one of these things (say, an expensive testing tool or certification or even an inappropriate testing methodology), especially if its use is to be tied to some KPI. This is where speaking truth to power comes into play.

Let us take a hypothetical situation. Your manager tells you that the company will pay for you to go and get certified as it is something that will help you in your career (now say thank you and get out of my office). You haven’t been testing long, but you’re a keen reader of testing blogs and you’ve read on some of them that some testing certifications may not be all they’re cracked up to be. Of course, you don’t have any direct experience (positive or negative) with the particular certification proposed.

You can find online a lot of information spruiking this certification method. There are also vocal groups saying ‘no, don’t do it!’.

If you need to argue with your manager about the pros and cons of certification, then it is not going to be enough to point at online info and say ‘Read this’. It will come down to ‘pointy haired manager’s glossy whitepaper from salesguy trumps your bit of paper from the internets’. As far as your manager is concerned, it’s just stuff that some dude said. It’s not your own experience (and by extension, not your own argument).

If you find yourself in this situation there are a few things you can do. If you are being pressed for an answer, an effective response is ‘Let me get back to you on that’. This will give you time to put together the information you need to argue your case.

Take the time you need. Study the material and the alternatives (I’ll come back to this). Put it into language they understand (probably money, but this is not always the case). Have whatever facts and figures you need prepared before you present your case.

When you present, use newspaper style. Your first sentence is your conclusion. Your first paragraph is a summary of your story and anything that comes after is as much detail as is required. If you see a room of nodding dogs (I mean agreement, not falling asleep), stop selling.

There’s a danger in the ‘know thy enemy paradox’ for the more experienced tester also, as there seems to be no shortage of stupid trying to pass itself off as smart and taking the time to find out enough to refute the never-ending streams of stupid can leave you less time than you want for learning to be a better tester.

There are a few things that I have found that help. If I’m not gearing up to take them down, I only need to know enough that I can say ‘I disagree, and here’s why’. That’s probably not going to take you long (I’m talking minutes, not hours or longer).If you find yourself spending longer, ask yourself ‘Do I need to be reading this, or am I just getting off on outrage porn?’

If I need to convince someone else, I’ll look for ways to construct a cost/benefit analysis. If you can show you’d get more benefit from a preferred alternative and put that into dollar figures along with a compelling story, then your going to stand a much stronger chance of your voice being heard. If you can, speak to someone with direct experience with the subject in question. If your manager has mistaken beliefs, you may need to spend a bit of time refuting them. Maybe have those in your back pocket in case you need to pull them out.

If you are gearing up for a longer conflict well, that’s probably a topic big enough for a separate post (probably by someone with more battle scars than me).

I spent the last week in Grand Rapids, Michigan, attending CAST2010 (Conference of the Association for Software Testing). It is difficult to describe the value of a conference like this. It is a collection of some of the most respected minds in software testing, of my peers and of people full of passion for the craft.

I attended an awesome workshop by Fiona Charles on ‘Speaking Truth to Power’. There were some fantastic presentations by testers from all over the globe. There were a series of lightning talks (I made one on the use of Japanese language as a learning mechanism). If that wasn’t enough, there were some excellent hallway conversations and exchange of ideas and the opportunity to meet some very talented minds. I met old friends, and many people with whom I had exchanged emails, but never met. It was all over far too soon and I am very much looking forward to the next one.

If you only attend one testing conference in 2011, I urge you in the strongest possible terms to make it CAST2011.

Here is what some other attendees had to say about CAST2010

Michael Hunter
Selena Delesie
Mark Vasco
Carsten Feilberg

Edit:
For those that want a small taste of what CAST is like, the keynote speeches are available online

Tim Lister
Cem Kaner

In the spirit of recent puzzles/challenges, I thought I might throw in one of my own. In high school, my class was given an exercise to come up with a coded message that the rest of the class would have to work out the cipher to.

I didn’t take it that seriously at the time and put in some sort of lame effort at home that afforded me the most time in front of the Commodore-64 instead. Most other kids did the same. Clearly the teacher had been hoping that someone in the class was a secret genius and would put together something brilliant. His disappointment when this did not occur was palpable.

I got thinking afterward that it was a wasted opportunity. I quite like puzzles, I like being creative, but I’d never put the two together. I spent a little bit of time coming up with something that might have been worthy of the assignment. I never did anything with it, or showed it to anyone, but I was reasonably sure that no one in that class at least would be able to figure it out. I should have put my money where my mouth was then. I might have been able to turn a profit betting people they couldn’t solve it in a given time limit. Ah well, hindsight is 20/20 as they say.

Instead, I present my code and the message it contains and invite you to work out the key. If you do figure it out, please don’t post the solution, but  I’d be really interested to hear about how you went about attacking the problem, so by all means leave a comment.

The message: the quick brown fox jumps over the lazy dog

IMPORTANT EDIT: In a moment of paranoia I double-checked my key and noted that there is a small change to the decoded message. In the coded text, ‘jumps’ is actually ‘jumped’. To those of you who have started, I apologise profusely. This means that there is an extra letter to the message than you thought you were dealing with, and rather than an ‘s’, you have a ‘d’ and an ‘e’.

The revised decoded message is:

The message: the quick brown fox jumped over the lazy dog

The coded message:
n g o c w u o y j   r
  f l l l q m v   e k
  o i y k b v   r x a
  k s e a t   d i b o
  v h a v   e u i p l

edit: James Bach has rightly pointed out that I could have worded my challenge a little more clearly.

There are an infinite number of keys that can associate your output with your input. For instance, the key might be simply to look at the first letter in the top left, and “n” means “quick brown fox…”

My intention is that people would attack this problem as though they did not know what the message was in an attempt to resolve the letters presented into the answer provided. I had a particular key in mind when I put it together, so the letters do resolve into the message given.

Enjoy.

Update
Okay, so a few people have asked for hints.
Here’s the deal – I’ll post a few hints, but I’m curious to know what you’ve tried and what you thought. I’ve posted one hint below. I will post more, but I’d like to know – How did you approach the problem? What strategies did you employ? What effect did the hint(s) have?

Hint 1

Hint 2

Hint 3

Congratulations to Rushmila Islam, who has at least worked out the message component, if not the complete cipher (which is trivial once the message is in-hand). Here’s another pangram (one with all the letters this time).

The coded message:
k e i f v a l p q u w
  b m a g n z i e c e

  m g d r w o o t a h
  y j a w u x a g e s

I kinda wish I’d put this one up first now. It looks trickier
Rushmila (or anyone else) – care to post the decoded message?

To borrow from Groucho Marx – QTP, I’ve had a wonderful time, but  this wasn’t it.

So thankfully I’ve been able to step away from QTP for the moment. Given that QTP doesn’t recognise Firefox so well after v3.6, and since we use a firefox plugin for most of our mobile testing (FireMobileSimulator), yours truly gets to switch to Selenium instead. I’m thankful for having had the opportunity to work on QTP, mostly so I have a better understanding of its limitations and shortcomings. I can argue more coherently against what seems to be in most cases, a ridiculous waste of money.

I view QTP a bit like one of those unfortunate bears in a Chinese bile farm. You can see how at birth it had the potential to be something majestic and powerful, but instead it languishes in a cage, irreparably twisted and deformed by years of abuse by the ignorant.

In comparison, Selenium while not without challenges of its own, has been by and large a real joy to use. For starters, having any number of fully fledged languages to work in is almost unbelievable after having toiled in VBScript land for far too long. In fact, the first problem I faced was which language do I go with?

I ended up choosing Java and JUnit, mostly because it’s what the current dev team codes with, and I simply cannot be arsed copping the flak I would get from management for introducing another language into the picture (as wonderful as jruby and jython are, I’m sure).

I didn’t realise quite how much thinking in VBS had stunted my thinking in other languages. I do a bit of coding in my spare time, but as far as automation coding goes, it took me a good few days to get comfortable again remembering the power that a real language gives you. I was all set to start importing test data from excel when a timely tweet (and subsequent blog post) from Adam Goucher reminded me that Java has these nifty things called Properties that you can import. (This is why tools like twitter should be allowed in the workplace btw – okay my RSS reader would have picked it up, but not before I’d wasted a lot of time).

It’s been a bit over 3 years since I last looked at Selenium. It doesn’t ‘feel’ all that different, but it does seem a lot easier to work with than I remember it being. Within a couple of days I’d whipped together a script to cut down some checks that took 4 hours manually to a little under 2 minutes. W00t.

I added loggingSelenium to my test setup and now have some pretty colours that light up for people who are impressed by that sort of thing. I’m now in the process of putting together a few more tests and a framework that will support them and their inevitable expansion.

There are a bunch of setup how-to’s out there, so I won’t be doing that, but I will drag together some of the more useful links that I’ve found. Stay tuned

On the suspicion that my last post amounted to a bunch of word salad for a few, let me see if I can add some clarity by describing the basic hierarchy I’m using for my framework.

At the top level, I have a suite control script that looks like this:

doSetup   ‘Calls to set up data and environmental variables and such like

Call Script1
Call Script2
…
Call Script n

Each script that is called is some sort of major transaction that an end-user would recognise.

If you missed the update I made to my last post, I’ve moved a number of function libraries to classes to lessen the amount of coupling required between the ValidationManager and the scripts. There is a small overhead involved in that the relevant class has to be instantiated at the start of the script. For debugging purposes, I have also added a commented-out debug section that calls the setup function that the suite control script does. Each script looks a bit like this:

‘#begin debug
‘doSetup    ‘uncomment this if debugging this script
‘#end debug

Transaction = loadTransaction    ‘This is the instantiation of the class that holds the Transaction’s functions

Transaction.function1
Transaction.function2
SharedFunction
Transaction.function3
…
Transaction.function n

For functions that are shared across multiple transactions, I have left them as a standard function library. On the plus side, the Transaction namespace aids clarity and lets you know where your functions live. On the downside, you can no longer access them by using the right-click->go to function definition option. No biggie imho.

The transaction classes aren’t that much different from your standard function library other than the fact that they’re wrapped in a class definition and there’s a bit of setup stuff at the start.

Class Transaction

Dim VerificationManager

Private Sub Class_Initialize()
VerificationManager = loadVerificationManager   ‘Here’s where the verification manager is instantiated
EndSub

Public Sub Function1
do.something
VerificationManager.CalculateExpectedState(“someAction”)
do.someAction
VerificationManager.Verify(“someAction”)
do.somethingElse

End Sub

…

End Class

I’ve still not gotten the all-clear to post the verification manager code, but as mentioned previously, it is a class also. There are 4 methods; 1 for setup and 3 relating to verification.

CalculateExpectedState calls a file called “someAction”. someAction calls a setup routine that creates a new dataSheet, also called someAction and populates it with expected data.
Back in the class method, the action takes place, at which point we return to the verification manager when Verify is called.

Verify calls the someAction file which calls a second routine that gathers data from the app and does any calculations required and stores the results in the someAction dataSheet. Verify then calls a diff function that compares the expected and actual results and reports. Finally, verify calls UpdateMasterData, which compares the actual data with the master data sheet and replaces anything that was changed.

At this point, there is a new data sheet created for each different call to the verification manager. This has the potential to get out of hand as the number of verification points increases. I’m thinking of using a single sheet and dumping the results to a file at the end of each verification. I’ll see what else occurs to me as I go.

I promised to talk a bit about what I’m doing with data. I still need to cover that. I’ve been having fun trying to define heuristics to govern where the data should drive the scripts and where I should handle variance programatically. I think I have something that works. We shall see.

One of the things I really like about the Logical Functional Model is the concept of removing verification code from the execution code. Another is updating verification data on the fly to reduce the likelihood of false positives.

These concepts are especially appealing since QTP’s in-built verification method is not worth using. Verification points are script-specific. You can’t update and re-use them multiple times within a script or share them across multiple scripts. Maintenance nightmare.

Instead, what I have done is to write a custom verification layer. Here’s how it works. At the start of my control script, I instantiate the VerificationManager, which is a class I have written to take care of verification code.

If a function that I am executing needs to do any verification, I pass the class to it by reference UPDATE: I’ve switched how I call this now. I’ve turned function libraries into classes. Each class instantiates the VerificationManager as part of its own setup and the functions can then call the VerificationManager without having to handle it as a parameter. There are several reasons for this – I’ll go into these in an upcoming post.

Within the function (now class method), there are 2 calls to the VerificationManager, one to set up the expected data and another to retrieve the actual data.

Where you call these depends on what setup you need to do beforehand. If you’re getting expected values from the application under test, you’ll want to do so before you execute the code you want to verify. If you’re grabbing your expected data from the data table, then you can do it just before you call the code to retrieve the actual data.

That probably looks like word salad on first read through. If you haven’t already, I strongly suggest you take a look at the verification chapter in Michael Hunter’s description of the LFM. It took me a couple of read-throughs to really get it, but once I did, I was impressed.

How does the VerificationManager work?
It’s as generic as I can make it. It has 4 methods.

• CalculateExpectedState(someAction)
• Verify(someAction)
• Diff
• UpdateMasterData

CalculateExpectedData and Verify both make calls to files called StateGenerators. It is the role of the StateGenerators to hold the logic and fetch the data required to do the verification. In simple terms, they are function libraries that are loaded at runtime by the VerificationManager.

You’ll notice that the parameter for each of these methods is the same. They call the same StateGenerator and this parameter tells them what name to look for. The VerificationManager loads the StateGenerator specified and executes one of two functions – either a ‘setExpectedState’ or ‘getActualState’ function.

Each call to the verification manager will use a different name. There will be one StateGenerator for each call. The StateGenerators all have 2 functions, all of them identically named. What they contain differs based on what they need to verify. There might be a series of calls to the global data table to grab data that needs to be checked, or there could be logic to calculate an expected value from a set of other values.

The setExpectedState function creates a new data sheet to hold expected data. If any of this data comes from the GlobalDataTable, then the new data sheet has an identically named column. Likewise if there are any new columns that need to be added to the Global data table for future reference, then this happens here too.

Once the code to be verified has executed, the call to Verify happens. Verify grabs expected values from the application under test and adds them to the second line of the expected data column. It then calls Diff, which (as the name suggests), compares each value of row 1 with the values in row 2.

The diff method logs both successful comparisons and failures by making a call to the standard QTP reporter. Right now, I’m going for simplicity; there’s a lot more you could do with this if you wanted. You could add weights to each verification item and log a severity message based on the weight of the failed item. You could call a custom reporter (which is a good idea, given that any time you tell QTP of a failure, it treats the entire script as failed). You get the idea.

Lastly, the Diff method calls UpdateMasterData. This routine simply loops over any difference and checks the global data table for columns of the same name. If they exist, the global data table is updated with the new value, so that if any comparison is made against it later in the script, it will be made against what we see in the system during this run.

I’ve asked my current employer for permission to post the code, as it’s completely abstract and contains no proprietary data. No word on whether they’ll go for that yet, but if so, I’ll update that here. Hopefully there’s enough here already to be useful.

So I mentioned some of the inherent issues that I dislike about QTP 10. One of those appears to be that reusable actions intermittently fail. An action in QTP is a tool-defined item that collects a number of GUI manipulations (or function calls) into a named action. It sounds handy. It probably would be if it worked reliably, but it doesn’t – at least not for me. The other thing is that if you have reusable actions (let’s say for instance that you keep your common ones in a single file and call them from other files), you cannot modify those actions without first closing your current test file and opening the one where the action is defined.

Instead, I have organised my scripts into a series of user-centric actions, which I have defined as functions in function libraries. To be clearer, each function executes a series of steps in the gui that an end-user would recognise as an action. e.g. log in, navigateToQuotationScreen and so on. This seems to work pretty well. I end up with a QTP screen that looks like a high-level checklist of how to execute a particular workflow. You do not have the editing restrictions that you do with reusable actions, as you can have as many function libs open as you like.

Neato. So I’ve been using the record tool to get something that works. Mostly because I dislike VBScript. I think it’s the Orcish of programming languages (Maybe not even that. Maybe it’s Kobold </nerd>). The recording tool creates a bunch of objects and sticks them in the object repository. That’s fine, saves me having to do it myself. Drawbacks to this are that for dynamically created objects, you may end up with a bunch of identical objects with different names. Not too hard to figure out, given that you should be naming your objects something sensible anyway. You can weed out the dupes when you do this and update your script accordingly. I’m definately going to check out Albert Gareev‘s suggestion of dynamically loaded object repositories. That sounds quite handy. I will say though that at the moment I’m being assisted by some not-so-technical testers who I am weaning away from the keyword view and the active screen.

The recording tool doesn’t always use the best code for the job. It’s the same old story – want the job done right? Gotta do it yourself. After the initial script is in place, I’ve been reading through the code and separating it into user-centric actions. At the same time, I look for:

• places where the code can be optimised
• places where data should be dynamic
• any fluff commands that QTP put in
• potential duplicates in the OR

At this point, what I’ve ended up with is having a series of user-centric ‘tests’ – I’ll call them transactions. Each one is an action (as every QTP test has 1 action by default), but consists of a series of calls to function libraries.  Each function is named for a step that a user might consider part of the transaction. For example, if we were talking about a program that provides a quote for car maintenance, the list of function calls might look like this:

• enterVehicleDetails
• selectServiceTypeAndLocation
• selectPreferredDates
• completeQuotation

Each of these functions lives in an associated function library. These functions may in turn call helper functions (such as date/time functions to enter date ranges in the future). Each transaction hides the ugly details of what is going on under the hood and while you don’t get to use the ‘keyword view’ anymore, you do have a clearly labeled set of steps.

So at this point, I have some scripts. They work. They’re not robust and they don’t do any data verification to speak of, but it is a beginning. Next up – parameterization and making use of the data table.

So I’ve been hacking away at QTP for the last couple of weeks (no, I didn’t make the decision to purchase it, yes I have to use it as the political consequences of not doing so are more painful).

I’m not a massive fan of QTP, mostly because I think that it’s really bloody cheeky for a company to charge six figures for a product because of its feature richness, then charge 5 figures for training because it’s so complex that no one can use the damn thing, not to mention that all the official written how-to information is safely tucked away behind a walled off site, available for those who pay their support fees. For the same price HP charge for basic QTP training, I could fly any world-class tester out here for a week to hang out with me and my team. I know which option I’d get more value from.

Luckily there are plenty of useful sites out there maintained by QTP users, which is handy because if your support fees are not paid up, you get bugger all from the source.

QTP 10 has some inherent bugs that annoy me and I’m still dubious about how well the shared object library will work for me in the long run, but leaving the ranting and bitching aside, I should say that it does seem to be doing what I tell it to most of the time.

I’m attempting to implement Michael Hunter’s Logical Functional Model (which I think is really bloody clever, btw) as best I’m able. It’s still early days yet, but we’ll get there. I do run into problems a lot, largely because if my inexperience with the tool, but perhaps some good will come of it if I share my pain with you I’m going to post bits and pieces that I discover, just in case they’re useful to others. I don’t know how well QTP will fit the LFM, but I’ll see what I can do about documenting my progress with that as well.

For the moment, here are a few links that I have found useful:

• The SQA forums QTP FAQ
• CodeWanker (love the name)
• Sachin Dhall’s QTP blog
• Dmitry Motevich’s blog (vlog?) QTP How-to movies

There are a few others out there, but they’re full of ISTQB ads, so I don’t really feel like linking to them. I’m sure you understand.

I gues I must be feeling punchy lately. No new content here for a little while, but I felt the need to respond to someone who linked to one of my posts. What I probably should have done was commented here and sent a pingback, but does anyone click on those anymore?

I digress.
Marcin Zręda has written an article that describes his conclusions from his experience with ET. The problem is, these conclusions are not presented as conclusions of his experience, they are listed as things that are good and bad about ET (with a line about them being drawn from his experience buried in the first paragraph).

I wrote a lengthy reply which I think should serve to counter most of the the errors he has made in describing ET.  Marcin has mentioned in the comments that the article was to serve as a jumping off point for discussion. In my opinion, this will likely have the desired effect, but misrepresenting experience as fact is one way to get people off on the wrong foot when having dialogue with you.

In this case, rather than wanting to know more about Marcin’s experiences, I wanted to correct what I saw as horribly misleading statements about ET. Now that I see he was trying to engender conversation, I am disinclined to continue.

‘I have these experiences with ET – what do you think? How does your experence compare?’ – sets a much different tone than ‘here are some good and bad things about ET’ (followed by a list of flame bait)’. I think that as testers we have a duty to be better than that. If you want to have a discussion about testing, or even an argument about it, there are plenty of people out there to have those kinds of discussions with, but be up-front about it. Be open about it and you will likely find that you will both learn more and keep your credibility intact.

I am making an appeal to those out there who are managing testers, or are in a position where their decisions influence testers.

I loosely follow several testing groups to see what interesting conversations are going on. I don’t often participate, mostly because I can’t commit to having the time to finish a conversation. If I can drop a comment I think will be helpful, then I will.

Then there are questions or statements that make me a little upset. I try not to respond because I tend to say things that are unkind. There are some people out there that need help and seem to be unaware of it. If you manage a testing team and you don’t have a lot of experience as a tester, you may be one of them.

The questions that get me irritated are mostly stupid questions asked by people that (according to their listed job title) should know better. Maybe stupid is not the right word. Lazy, ill-informed, ignorant – these are probably better adjectives.

Recently I saw the following questions asked:

How can testing be monitored? How can we measure the performance of testing?

That was it. No context, no qualification, no further explanation. I assert that becasuse of this, they are stupid questions. I suspect what this person meant is ‘how do I gauge the efficiency and effectiveness of the work that my testers are doing?’, but that’s one of many different possible interpretations of the original questions.

Not to mention the fact that I suspect that what this person actually wants is a bunch of numbers they can use to reassure themselves that all is well, or possibly to threaten perceived underperformers with.

I don’t really want to get into a rant on metrics. It’s been done to death by far more qualified people than I (see ‘meaningful metrics’). What I do want to say is that if you are managing software testers and you are asking questions like this, you really should know better. Stop being lazy and educate yourself. I feel sorry for the people working for you.

For people who are managing testers – especially if you do not have experience in software testing, let me pose the following questions to you.

How do you measure the effectiveness of a systems administrator? a business analyst? How do you know when you have someone who is ineffective in these roles?

Both of these roles require quite specific knowledge. There is also a massive amount of variation within the definition of these roles. A sysadmin that works on webservers is likely to have different strengths than one that specialises in SMTP admin. Of course there will be overlap in their skills, but they are different animals. Applying any sort of blanket measurement to them because they share the same title is stupid.

One thing these jobs do have in common (and in common with testing) is that they are knowledge and service related. If the people they are providing a service to are not satisfied, then you have an issue. You don’t go setting up metrics to count how many shell scripts and admin has written. You don’t judge a BA on how many use cases they write. You gauge their effectiveness by how their work is received by their peers and their clients.

Why would it be any different with testing? Testing requires specialised knowledge. How do you gauge the effectiveness of the people doing it? Look at the information they produce. Consult the people they produce it for. Speak with the testers themselves. Is the information useful, or not? Why/Why not? There you go. That’s how you measure the effectiveness of your testing.

Without the qualitative information these people can convey, you can collect all the metrics and numbers you like. You can draw all the conclusions from them you like, but those numbers alone are not enough to make any meaningful distinctions.

Worse still, if you’re judging the effectiveness of a testing effort during a project by things like bug counts and test case coverage, you’re potentially ignoring important information. A simple example – we have X open bugs. So what? What does that mean? So you’ve covered 100% of test cases – what does that mean? What other testing have you done? What did that find? What does that mean?

If you are using numbers as a barbiturate so that you can feel reassured about how the project is going – if you look at the numbers and and nothing else, and you accept them at face value, then you are being lazy. If you don’t have time to dig deeper then that’s a bigger problem, not an excuse.

If I sound a bit defensive, it’s because I am. I’m sick of seeing lazy people give testers a bad name because they don’t understand testing, aren’t interested in learning about testing and yet want to have some way to control testing because that’s how they think it works.

If you manage testers but you don’t trust them to be able to do their job, and if you aren’t willing to learn about the difference between effective testing and ineffective testing then have the stones to step aside from someone that is. You are worse than useless in your current role.You are a hindrance and are in all likelihood making people deeply unhappy.

If you are willing to learn, do try and think it through. What is it you are trying to achieve? Are you doing it because it’s genuinely useful, or because that’s what everyone else seems to be doing? Do some research. Plenty has been written down about it. At least that way when you ask a question, it will be an informed one. When I read questions like the above, here’s how it comes across:  ‘o hai, I ar new manager. Plz hope me teh metriks with xamplez kthxbai’.

I think that as managers of testers, it his high time we raised the bar. We have a duty of care to understand what they do and the challenges they face. To support their work and to maximise the effectiveness of their efforts.If you are not constantly seeking sources to help you do this, then you are doing them a disservice.

If you have your heart set on measurement, then please, at the very least learn the difference between first, second and third order measurement and how to apply them.

Additionally, it’d be a bonus if you understood more about testing and how testing fits into software development (please do your testers a favour and read this book).

Hopefully for you this is the tip of a very large iceberg. If you do these things, you will look back at questions like the ones posed above and wonder how you could ever have been that naieve. If you can cultivate a habit of curiosity and the desire to learn then you stand good chance of becoming the sort of manager your testers need you to be.