Posts Tagged ‘michael hunter’

On the suspicion that my last post amounted to a bunch of word salad for a few, let me see if I can add some clarity by describing the basic hierarchy I’m using for my framework.

At the top level, I have a suite control script that looks like this:

doSetup   ‘Calls to set up data and environmental variables and such like

Call Script1
Call Script2
…
Call Script n

Each script that is called is some sort of major transaction that an end-user would recognise.

If you missed the update I made to my last post, I’ve moved a number of function libraries to classes to lessen the amount of coupling required between the ValidationManager and the scripts. There is a small overhead involved in that the relevant class has to be instantiated at the start of the script. For debugging purposes, I have also added a commented-out debug section that calls the setup function that the suite control script does. Each script looks a bit like this:

‘#begin debug
‘doSetup    ‘uncomment this if debugging this script
‘#end debug

Transaction = loadTransaction    ‘This is the instantiation of the class that holds the Transaction’s functions

Transaction.function1
Transaction.function2
SharedFunction
Transaction.function3
…
Transaction.function n

For functions that are shared across multiple transactions, I have left them as a standard function library. On the plus side, the Transaction namespace aids clarity and lets you know where your functions live. On the downside, you can no longer access them by using the right-click->go to function definition option. No biggie imho.

The transaction classes aren’t that much different from your standard function library other than the fact that they’re wrapped in a class definition and there’s a bit of setup stuff at the start.

Class Transaction

Dim VerificationManager

Private Sub Class_Initialize()
VerificationManager = loadVerificationManager   ‘Here’s where the verification manager is instantiated
EndSub

Public Sub Function1
do.something
VerificationManager.CalculateExpectedState(“someAction”)
do.someAction
VerificationManager.Verify(“someAction”)
do.somethingElse

End Sub

…

End Class

I’ve still not gotten the all-clear to post the verification manager code, but as mentioned previously, it is a class also. There are 4 methods; 1 for setup and 3 relating to verification.

CalculateExpectedState calls a file called “someAction”. someAction calls a setup routine that creates a new dataSheet, also called someAction and populates it with expected data.
Back in the class method, the action takes place, at which point we return to the verification manager when Verify is called.

Verify calls the someAction file which calls a second routine that gathers data from the app and does any calculations required and stores the results in the someAction dataSheet. Verify then calls a diff function that compares the expected and actual results and reports. Finally, verify calls UpdateMasterData, which compares the actual data with the master data sheet and replaces anything that was changed.

At this point, there is a new data sheet created for each different call to the verification manager. This has the potential to get out of hand as the number of verification points increases. I’m thinking of using a single sheet and dumping the results to a file at the end of each verification. I’ll see what else occurs to me as I go.

I promised to talk a bit about what I’m doing with data. I still need to cover that. I’ve been having fun trying to define heuristics to govern where the data should drive the scripts and where I should handle variance programatically. I think I have something that works. We shall see.

Jonathan Kohl beat me to the punch on this blog entry by a good margin, but since I was already working on this thing, I’d like to go ahead and basically say exactly what he said in probably a far less cogent fashion.  A lot has been written about the good and bad things about Exploratory testing and the writing and (mis)usage of test cases.

I don’t see a lot written about harmonious use of both. I’m sure people do it. I know I do. Is it like airing dirty laundry? Do context driven testers not want to admit to using test cases in case James Bach reads their blog and decides to tear them a new one? Do factory testers (sorry – what do you BDUF testers call yourselves anyway? Factory tester brings to mind a grey-clad downtrodden peon, which I suppose is appropriate, but not very flattering) – anyway, do you lot not want to admit to doing exploratory testing in case your overseers beat your for not following the script? Am I missing something? Is this just not a big deal?

Johnathan advocates using a blend of both and uses the terms prescriptive and descriptive testing, which quite appeals to me. Michael Bolton’s recent highlighting of the difference between testing and checking as something we should be more conscious of helped me to frame this better in my mind.

There are things that you can plan ahead for. There are risks you can identify and tests that you can design to mitigate against them. I see no reason why you shouldn’t use test cases for these if that is what you are comfortable with.

One of the issues I have with test cases is that some people see them as a magic artefact that covers all possible problems in a given area, because one or more tests have been written that mention it.  They fail to see that many test cases are highly focused on verifying a very specific item somewhere, and just because we go through a bunch of steps to get there does not mean that those steps have also been verified.

This leads me to wonder if I have been thinking about scripts all wrong.

I recently read Michael Hunter’s work on setting up test frameworks. One of the things that was a lightbulb for me was separating the verification from the action. Very very obvious in hindsight, but if you think about it – this is basically a use case and a checklist.

We have a series of steps from point A to point B

We have a list of things that we want to make sure are as we expect them.  Why do we need to mash them together? Why do we need to repeat the list of steps over and over and over so that we can list a different verification point next to it?

An argument I hear for test cases is that if you write them well enough, then anyone can execute them. I really hate the way that thinking goes, mostly because I read it as ‘anyone can do testing, you just follow the scripts’. That being said, a clear set of instructions and a checklist could mean that you could get a non-tester to do checking, even of mission-critical information. That, I might hold with.

Checklists and Check scripts? I think this is something worth exploring further.