Archive for July, 2010

16 07 2010

What I learned from my puzzle exercise

Posted by: Ben Kelly in Software Testing

I’d like to say thank you to everyone who participated in this little puzzle exercise and most especially to everyone who provided feedback. I hope you had fun with it. Well done again to Rushmila Islam who was the first to work out the puzzle.

I have posted .the solution at the bottom of this post. YOU HAVE BEEN WARNED.

I learned a lot more than I thought I was going to. Firstly and probably the most obviously, I learned a lot about cryptography that I didn’t previously know. Thanks to James Bach for the collection of links.
Braingle was probably the best of the bunch. Thanks also for the link to Sebi’s puzzle.

I found Jürgen Plasser‘s answer fascinating. I hadn’t encountered measuring entropy of text as a way to inform possible encryption techniques. It’s something I’d like to look into further.

It was interesting to see how close people got with their responses. Many people had several correct pieces of the puzzle, but not quite enough to solve it completely.

I think most people gave me way too much credit for the amount of crypto knowledge I’d have had as a high-school student. I didn’t have access to teh intarwebs back then, and time that I could have spent in the library was more likely to be spent playing role-playing games or painting miniatures or similar valuable life skills.

One of the things that this puzzle made abundantly clear to me was the value of information before and after it becomes known. Without the key information, it was a real puzzle. Far more so than I had originally anticipated. Having all the information at hand, the puzzle seems obvious and simple in hindsight.

It drove home for me just how difficult it is to convey the value of a tester to those who don’t understand what good testers do. Even the distinction that testers are a source of information (so is a book. so what?) doesn’t necessarily convey the effort that is involved at producing this information.

Some people went to a great deal of effort; tried some very cool things. That was the most gratifying thing for me. The answer was ultimately irrelevant. I got to see how you guys think. As testers, we are often showing others the fruits of our effort, but not the beauty of the effort itself. Having been able to do that here, at least in part, I think is pretty cool. Thank you again to everyone who shared.

SPOILER ALERT – Explanation of the cipher follows.

The message was encoded with direct character substitution with a few added extras. The message reads top-down, right to left. The first two characters provide the character offset – character 1 – character 2 = offset.

The spaces themselves mean nothing, but exist primarily for obfuscation though I did put them in diagonally in the first puzzle to give a clue to the direction of the word flow. Spaces in the actual message are represented by vowels, so in this sense, vowels do double duty in that they can represent their offset character or a space.

Comments 5 Comments »

09 07 2010

Data driven Selenium in JUnit via @Parameters

Posted by: Ben Kelly in Software Testing

I played around a bit with iteration using JUnit today. I have some generic tests that behave differently depending on the values fed to them. I don’t want to have iteration code living alongside each test (maintenance nightmare), so I wanted to use JUnit’s @Parameters tag to pull in my test data via a Preferences file and do the iteration for me.

It took me longer than I expected to get right, mostly because static methods annoy me and trip me up a bit. (Your parameters method must be static in order for it to work).

However, get it working I did. On the off chance that it saves someone else some time, here’s how it works:

import java.io.InputStream;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Enumeration;
import java.util.Properties;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;

@RunWith(Parameterized.class)
public class paramTest
{
	final static String dataFile = "/dataDriver.txt";
	private String words;

	public paramTest(String words)
	{
		this.words = words;
	}

	@Test
	public void verifyThing() throws Exception
	{
		System.out.println("key: " + words);
	}

	@Parameters
	public static Collection data()
	{
		Collection returnList = new ArrayList();
		InputStream dataStream = paramTest.class.getResourceAsStream(paramTest.dataFile);
		Properties dataProperties = new Properties();
		try{
			dataProperties.load(dataStream);
		} catch(Exception e)
		{ System.out.println(e);}

		Enumeration e = dataProperties.propertyNames();
		while (e.hasMoreElements())
		{
			returnList.add(new Object[]{dataProperties.getProperty((String) e.nextElement())});
		}
		return returnList;
	}
}




Interestingly enough, the data comes back in the reverse order that it exists in the preferences file. Something to keep in mind if you want your tests to run in a certain order.

				

				
Comments 1 Comment »


			

				

                                

			        	

                                                07
                                                07
                                                2010
                                        

                                        
My Selenium setup so far

                                        Posted by: Ben Kelly in Software Testing  
                                


				

					
I’m having a lot of fun with this Selenium gig (shh, don’t tell anyone). There have been a few yaks shaved and I’ve seen the inside of some rabbit holes, but there seems to be a lot of good material online to guide you.


At this point, I’m running my tests directly from eclipse, though we’ll be looking to kick them off with Ant in the near future. I’ll jump off that bridge when I get to it.


LoggingSelenium

I’ve set up loggingSelenium to log test execution. I’m not sure what its current development / support status is over at SourceForge. The documentation it comes with is not real flash, but there’s a decent example here (via here). I have made a couple of custom modifications to the source of loggingSelenium as there are a few gripes that I have with it.

Some of the pluses are:


    

  •  it will automatically grab screenshots for you when your scripts bork.
    • 

  •  it color-codes your asserts and failures
    • 

  •  it can wrap JUnit asserts and log them alongside your selenium commands.
    • 



Some of the not-so-good things are


    

  •  it requires a hardcoded path for writing output
    • 

  •  it looks like it’s geared to non-windows systems, given that out of the box, the screenshot links in html output don’t work.
    • 

  •  the color of failed selenium asserts is different to JUnit asserts, the former being an easy-to-miss beige color.
    • 



The mods I made were quick and dirty hacks, one to fix the screenshot link and the other to make the beige color more noticeable. If I find myself with a spare second I want to get rid of the need to use a hardcoded path, or at least make it more nicely configurable. I also want to add a color selector so you can customise your output.


Autodocumentation

I’ve set up doxygen to generate documentation from code comments. I’ve not hooked it up to anything yet, but either an SVN pre-commit hook or getting ant to kick it off shouldn’t be a problem.


Test Structure

JUnit 4 does not require you to extend tests from a test class, which is quite nice, actually. Instead, I have written a parent class for my tests to extend from. The parent holds all of the setup and most of the methods that drive the tests themselves.


Currently I’m writing one test per class. The class contains a single method containing calls to methods in the parent and nothing else, so it ends up looking like a set of test case steps. The parent has the potential to get quite big, so as test case numbers grow, it’s likely that I’ll add further abstract classes between the current parent and the tests to group them more reasonably.


I’m importing data via Properties as a resource stream. I have data that is used to drive the tests and data that is variable, but predictably constrained. For instance, I use a vehicle class number as a driver. The class number refers to a class of vehicle whose data is stored against that number. Depending on the number selected, different vehicle data is populated during setup.


On the subject of Properties – If you’re developing in a language that uses double-byte characters (such as Japanese), if you are generating your Properties files as flat files, you’ll need to run native2ascii on them to convert your double-byte code to unicode. Otherwise, they will not be properly interpreted and things will fall over.


I have just started playing around with the @Parameterised tag and @RunWith(Parameterized.class) to get iteration working with data variation. I haven’t decided quite how I’ll run with that yet. I could do this at a suite level and control iterations of tests by feeding data to them, or I can have each test take care of its own iteration by setting up parameterisation within the tests’ parent class.


Browser setup

I’m doing mobile emulator testing at the moment, so I’ve set up a firefox profile specifically for it. I’m using the user.js file to alter the plugin config so that the browser starts up with the plugin activated and a particular device switched on.


When starting seleniumRC, I specify this browser profile specifically so it is automatically used when firing up the browser. I was hoping to be able to change device type on the fly by mucking around with chrome urls, but I’ve not found an elegant way to do this yet. Instead, I am swapping the user.js file out whenever I need to change. Essentially, you can load up the user.js file with calls to your plugins to switch them on/off and configure them based on what configuration the plugin makes available.


It’ll end up looking something like


user_pref(“<plugin.configItem1″, “value1″);

user_pref(“<plugin.configItem2″, “value2″);


You can hunt down config items by typing about:config into your browser. Any config item that is not set will be invisible however, so I ended up looking through the plugin’s jar file to see what else I could see.


So at this point, I have a loose collection of parts that are not quite all tied together but it’s getting there.

Other things I’m contemplating in the not too distant future is a web front-end so non-technical testers can select test suites and kick them off at will.


While I haven’t gone into code-level specifics, the links should provide a fair amount of detail. Let me know if there’s something I’ve not explained so well. If you have comments or questions re any of this, I’d love to hear it.

				


				
Comments No Comments »