testjutsu

I have been reading Matt’s blog for a while now. Since October last year, Matt has been periodically expanding and evolving his thinking about ‘technical debt’ - the small, seemingly inconsequential short-cuts and hacks that we tend to implement or let slide in order to hit deadlines or save us doing more work (at least in the short term). Like most bad habits (such as the gradual accrual of fiscal debt, say, on a credit card), these things tend to lead to large amounts of pain down the track.

Rather than regurgitate what Matt has been saying, let me point you at his work. I think it is worthy of consideration for anyone who has had to interact with another human being in a professional capacity, ever.

• Technical Debt #1
• Technical Debt #2
• Technical Debt #3
• Technical Debt #4
• Technical Debt #5
• Technical Debt #6

If you’ve skipped to this bit of text because you’re not up for six essays on an unfamiliar concept, let me recommend that you at least read #6 before checking out details on Matt’s upcoming Technical Debt Workshop.

If you’re in the vicinity and have something you can contribute, then why not do it? If you’re not, then keep an eye on Matt’s blog. I’ve no doubt there will be some very interesting outcomes from the workshop.

The company I work for has been using subversion for several years now and it suits our needs reasonably well. It’s generally pretty easy to use; it’s certainly an improvement on CVS, but on the other hand there are some things that aren’t all that great. The way externals were implemented leaves much room for improvement and svn tends not to play nicely with mixed user permissions (such as when files are written alongside or over checked-out code by www-data). Subversion has a relationship with posix ACLs that is rather less cordial than the one described in The Pogues’ Fairytale of New York, but these are perhaps war stories for another time. These errors at least have the decency to fail with the requisite amount of melodrama.

More recently we’ve come across some more subtle and potentially nasty issues that subversion barely bothers to notify you about, if at all.

The way our shop works, we have a single trunk from which branches are forked. Changes are made on-branch before being tested and re-integrated into the trunk. Problems can arise when you make changes to a file on one branch (A), and move the same file on another (B).

Subversion treats moving files as a ‘delete’ and an ‘add’ (delete the old file, add a new file with the same contents and a different name).  If you integrate branch A and then branch B, the newer changes from branch A will be lost (as the file is deleted) and the branch B version will be instated with its new name. SVN does not warn you that a newer instance has been overwritten - charming.

If on the other hand, you merge branch B first, then branch A, then because subversion can’t find the the file to merge with branch A (because it was renamed in B), it simply reports that the file is ’skipped’. No mention that changes to that file could not be merged as the file was not found just ’skipped <filename>’.

Easy enough to miss when you have several hundred files scrolling past you at a rapid rate of knots.

Apparently, the best you can do is be very, very careful when integrating branches that contain deleted or renamed files. I’ve taken to pointing any sizeable svn output to a file and grepping the hell out of it.

A similar thing happens when you use nested branches.  This one is perhaps more understandable, but the level of reporting is just as woeful. If you branch from the trunk, then make a second branch from the branch then care must be taken if you attempt to merge the second tier branch directly back to the trunk. If you have added files in the second tier branch, they too will simply be ’skipped’ if you attempt such a merge.

Merging back to the first tier branch is no problem, and is probably what the system was designed for. We discovered this issue when a developer created the nested branch to make newer updates to a very old branch. A better solution would have been to create a fresh branch from the trunk and merge the changes from the old branch into it, before making the required modifications.

Hopefully these are useful to some of you other SVN users out there.

I used to think that as a software tester, ‘if you’re doing your job well, then no one notices’. This sort of thinking was something I was perpetuating by misunderstanding my role as (merely) someone who performs test execution. I could see the benefit of the work I was doing and assumed that because no one else seemed really interested, they didn’t care.

From my experience, there is a tendency for testers to be thought of as second-class dev-team citizens, if not overtly then subconsciously. A developers job is perhaps easier to visualise than that of a tester. I’m not saying their job is easier, but for some reason a coder implementing programming algorithms by writing code seems easier to picture than a tester implementing testing algorithms by… ‘you guys just click on stuff to make sure it works, right?’.

I spent a lot of time arguing for recognition of testers and questioning why other members of the dev team seemed not to understand how important testing was. It was frustrating to me. It seemed like more often than not, when people took notice it was because something had gone pear-shaped because it was my fault (I’m the quality guy, right?).

It slowly dawned on me that it wasn’t because they weren’t interested, but because the way I was conveying the how, what and why of testing was not compelling. My colleagues knew development. They understood development, but their experience with software testing was limited. It’s not that they didn’t value the test team, but the way I was reporting my testing wasn’t as relevant to them because it was more or less a binary ‘Yes I have found problems’ or ‘no I haven’t’. This had a dangerous side effect of making me a defacto deployment gatekeeper.

They valued the ‘yes’ or ‘no’ a great deal, but because that’s all I was providing, and because that was their experience with testing, that’s all that was expected. I was thinking hard about testing and how the product might break and what else might go wrong, but I wasn’t really focused on how to get that across to the people that mattered. They assumed that when I said I was done, I knew enough to know that was true. To be honest, at that point, I didn’t know enough to argue otherwise.

As my experience in testing grew, I realised that test execution is a small (but of course important) facet of a tester’s role. I began to see the testers role as multifaceted and I became part devils advocate, part psychopomp and possibly large amounts of the damned from various circles of Dante’s inferno :). I pushed to be included earlier in the project lifecycle. I asked questions, made suggestions, asked for documentation and information and made sure the people I was working with were across the thinking I was doing about risk and risk mitigation. I started talking in terms of how the testing we were doing was covering the risks we’d raised (and sometimes how it wasn’t). I didn’t always go the most diplomatic way about it, but gradually things began to change for the better.

The role of a tester to me now is to really be a central information hub, and a representative for those who may not have a voice (such as the end user). Reporting the test effort should be a matter of knowing who your audience is (or more broadly, who matters) and what matters to them. The development group I work with now have come to expect assessments of risk, analysis of proposed solutions - the thought behind the effort is as important as the effort itself.

I suspect that my early experiences are not uncommon. I have heard and read many accounts of testers feeling undervalued and misunderstood, being blamed for errors that go into production, being the gatekeeper for the decision to deploy. I don’t think testing is nearly as understood as a profession as coding is and it’s something I’d like to change.

I think the onus is on the software tester to be proactive in promoting their effort in such a way that it adds value to those that matter; so that the people they are supporting understand exactly what the tester’s role is. I think there is a need for education both of non-testers, to show them the value of software testing, as well as education of software testers so they are able to convey a compelling story of what their testing is and does and why it matters.

My situation is of course going to differ from other testing shops out there, both in the style of work and the relationship with developers in the development or project group. There may be an element of ‘picking your battles’, or making change in small increments over time. Testers might face extreme reluctance or hostility to change. I think at that point it becomes a matter of the tester being clear on the sort of behaviour they are willing to commit/adhere to, and those that they will not. More on that soon.

I am reasonably certain that somewhere in the world, a tester is being blamed for not finding this.

I am a little pressed for time right now, but I have been thinking a lot lately about the perception of testers by the people they support and what that can mean in the context of the projects they work on, the companies they work for and testers as individuals. I think there’s a rich vein of blogworthy material to mine here, which I promise to attempt just as soon as time permits.

Jared Quinert pointed out to me that I missed an opportunity to work in a good point about testing in my entry supporting the WGA. I’ve recently been struggling with the task of coming up with meaningful stats to report to my handlers to help justify the existence of my team and anyone with a brain cell would have noticed that talking about prodco execs massaging stats of WGA writers would have been the perfect segue into this. Apparently I still suck at the blogging thing. I’m sorry.

Let’s pretend for a moment that I was clever enough to put two and two together. Here’s what I might have said.

You can make statistics say pretty much whatever you want. This is potentially dangerous because in my experience, more often than not, your audience is passive won’t care at how you arrived at your final figures as long as the layman’s explanation sounds reasonable.

A tester worth their salt is different. They’re conditioned to ask ‘why’ to almost everything, but I have a sneaking suspicion that there are many people out there that accept any given number at face value as long as it’s not obviously outrageous, maybe because they’re afraid of numbers, maybe because it simply doesn’t occur to them to question why.

Let’s take the following statement on the AMPTP’s website:

According to WGAw, 4,434 of its working film and television members earned a combined $905.8 million in 2006. The average member earned $204,295 and over half earned at least $104,750. The WGA noted that these numbers are based on earnings reported for dues purposes and thus do not fully reflect above-scale payments.

In case you’re wondering where they got their information, they got it from a report like this one.

It would be very easy to read that statement and conclude that at least half of screenwriters earned over 100K last year - not a bad paypacket in most people’s book. Certainly the AMPTP seem quite content to leave it at that.

Let’s have a closer look though. One key word that jumps out at me is ‘working’. ‘4,434 of its working film and television members…’ - meaning that there is an unstated number of non-working film and television guild members. What is the guild’s total membership? According to the report I linked to above, it’s 7313 +/- 400.

So when they say ‘The average member earned $204,295‘ they mean 910m divided by 4,434, which they use to arrive at their average figure of around 200k. Let’s factor in the conservative figure of total membership - 6913 (7313 - 400).

910m / 6913 = $131,636

Very different to their figure of 205K. Still, 131K is still a pretty good wicket to be on, right? Maybe, but remember, we’re averaging wages earned by all across a large number and at least 2,479 of those people (36%) did not work at all during the fiscal year stated.

Let’s add a little more contextual information. As a writer, you might work steadily for a season on TV, or be paid for a script during one year, and then not work again for 12 months or more. The 36% of writers that didn’t work this year are predominantly not going to be the same people who don’t work next year (assuming the strike is resolved by then).

Taking our average figure of 131.6K and stretching that across two years, we’re suddenly looking at 65K and this is before we factor in tax. An agent will typically take 10 percent which further eats away at your nest egg. Suddenly the princely sum the AMPTP are touting is not looking nearly so princely.

Even if you were to say that the average writer works 64% of the time, then you come to a sum of $84,224, less agents fees - $75,800, less tax (let’s be generous and tax them 33%) and you arrive at almost exactly $50,000 - and remember I’ve been using conservative figures.

Are the AMPTP being deceitful? I’ll leave that for you to conclude. They’re definitely not giving us the entire picture, that’s a certainty.

As software testers, we’re taught to look for multiple explanations for problems. The map is not the territory. How often has your first conclusion turned out to be incorrect. If you’re anything like me, the answer is very.

I don’t think examining stats is very different in this regard. You need to look closely and think critically, whether or not there’s a neat little summary next to the numbers that tell you what you’re supposd to think. What the numbers don’t say may be just as important as what they do.

What contextual information may be missing? Can you drill further into the numbers and look more closely? Is there a heavy skew and a long tail hidden by a mean average? What does that mean?

Abstracting a little more, you can look at who has put the statistics together and why they might have chosen to present it the way they did. Do they have a particular bias or motive that may make them want to present statistics in a particular light? Someone with a KPI bonus for keeping bugs low might present a set of figures very differently than someone with a bonus based on the number of bugs found.

I am not at all surprised at the number of war stories one hears about statistics being misused and abused. It’s too easy to accept numbers at face value when the numbers look as good as you want them to, or present a slam-dunk answer to an argument.

I am going to do my best however, to make a point of being the guy who, when presented with stats, does dig deeper and shakes things to see what falls out. It might be a bit more effort, but it might also become a lot more interesting, not to mention valuable.

Writing for the screen has many similarities with software development. It is a highly collaborative affair, and one that requires patience, skill, clear communication and quite possibly a penchant for the masochistic.

A screenwriter has only 120 pages in which to tell their story. It sounds like a lot, but when you can write only what can be seen or heard, it rapidly becomes very challenging. Every word must drive the story forward.

The story has to be one that works on film. If the story depends on internal states (what people are thinking and feeling), it may be something that is better suited to a novel. If it’s dialogue-heavy, you might have a stage play on your hands. Not every story lends itself to being on film.

A writer of spec scripts (scripts that are unsolicited) is, whether they know it or not, writing for at least four audiences. Firstly the production company’s readers.

Readers are paid a pittance to wade through massive piles of screenplays that (literally) tens of thousands of other writers submit to Hollywood every year. They’re the ones who sift the gold from the shit (and 99.9% is shit). They’re hoping that the next 120 pages that they pick up is going to be the nugget they can plonk on a producer’s desk and say ‘we have to make this’, and of course they’re generally disappointed. Poor bastards.

As a screenwriter, you have to grab their attention from the first sentence - the first word if you can, then hold it. Keep them wanting to turn the pages even if it’s the 30′th script they’ve read that day and their eyes are red raw. A spec script has to be that good.

After that, the script has to grab the imagination of a director. If you take a look at a novice screenplay, it’s generally full of camera direction and references to music and such. This is a no-no for writing on spec.

It is the director’s job to call the shots, not the writers. Okay, so a spec script with this sort of direction will be binned by a reader before it ever gets to the director, but the point is, screenplay writing is collaborative and you need to know what other people’s jobs are, and let them do it. That doesn’t mean you can’t capture their mind’s eye and suggest implicitly what the right shot might be.

Let’s say for instance you have an action movie where you have a hostage strapped to an operating table about to be operated on, and your next scene is your protagonist tending his wounds after a battle. If the last sentence of your operating theatre scene ends with the hostage being anesthetised, and the first sentence of your protagonist scene is him having a bullet wound stitched shut, you’re not explicitly telling the director to superimpose the needles for the scene transition, but the inference is reasonably clear.

You’re not telling the director how to do his job, but there’s a suggestion there if he wants to use it.

The screenwriter needs to write for the actors as well. If you don’t have characters that test the range of an actor, that challenges them, they’re less likely to want the part. You need to know how to build tension and create conflict and show them the opportunity to display their acting chops.

The novice writer probably starts out with the movie-going audience in mind and of course they’re important too. You need to have a story that works on screen, that grips the interest and imagination of the viewer, holds it for 2 hours and leaves them feeling that the time they spent was worthwhile.
As a software tester reporting bugs, your role is not so terribly different (although hopefully on a smaller scale in terms of length). You also have several audiences and several relationships to manage.

You need to report your findings clearly and concisely with enough detail to adequately convey the bug and the environment and context surrounding it. Make it easy for the person reading your report. If there is some heavy lifting you can do for them, do it.

Is your bug worthy of reporting? Relate the problem to risk. Why is the bug worth fixing? What happens if we ship without it being fixed? Who benefits from it being fixed (or who is it detrimental to if not). Create a compelling case for the resolution of your bug.

Don’t tell a programmer how to do their job. If you have theories about the cause, or suspicions about a more serious underlying issue, then note it down, but preceding it with ‘you need to fix’ or ‘I would have written it like this…’, or ‘the whole thing needs rearchitecting’ is unlikely to win you many friends.

Like a screenwriter, once you submit your bug, it is out of your hands. You want to send it into the world with the best possible chance of success. A screenwriter wants their play to entertain (and also come back with a large cheque attached), and build their reputation as someone who produces quality work. You want your bug to come back resolved and build the same kind of reputation.

Understand who your audience is, what they are looking for, and make a point of providing it to them.

Cem Kaner has an excellent document on his site about bug advocacy. Well worth the read.

For any budding screenwriters in the audience, if you haven’t already devoured the wordplayer.com columns written by Ted Elliot and Terry Rossio, then your next eight hours are in this link.

Another non-software testing / non kendo post, but for what it is worth, I want to lend my support to some people that deserve it.

If you watch t.v. or see movies, you probably know there’s a strike on at the moment. I have a few friends who are participating. Your favourite television series might be delayed a good while because of it, but if you’re blaming those overpaid, lazy writers, then your ire is misdirected. What these people are asking for is not unreasonable (more detail here).

The AMTPT would have you believe that they’re the ones being hard done by; that because writers are paid for paid downloads and online pay-per-view, that not paying them for streaming video online is not ripping them off. They’d have you believe that the average working writer is taking home over $200K a year, based on some sums they did on numbers from the WGAw.

This ain’t necessarily so. I am led to understand that this is a more representative explanation of how screenwriters are paid. It looks to me like the AMPTP is massaging the facts in order to put a very particular spin on events. I’ll admit, my viewpoint is not unbiased. I want my friends to be better off.

I’m tempted to draw parallels between this and the recording industry where artists are beginning to break away from the draconian Establishment and distribute on their own terms. Unfortunately for the screenwriters their relationship with the prod co’s is a symbiotic one, as there are many more moving parts involved in putting a movie together. The AMPTP seems a little hazy on the difference between symbiotic and parasitic.

I do hope for their sake, but more particularly that of the people on the picket line that they work it out soon.

It’s easy to tell yourself that you’re training hard when in actual fact you’re slacking off. I find myself doing it far too often for my liking. If in the kendojo I’m fighting beginners, I might fend off attacks in order to get them to try and work out why they are unsuccessful. Nothing necessarily wrong with that, but if that’s all I do, then I’m slacking. More effective for the both of us would be if I tried to fight at a level slightly above theirs and encourage them to rise to that standard.

If I’m not really trying, I notice myself falling into bad habits. I let my feet move move lazily, I block without trying to counter, if the first strike fails, I may not immediately make a second or third effort (all of these rob you of the ability to take advantage of opportunities) - these things are physical manifestations of mental laziness. When I see others being lazy, I take them to task for it. When I find myself doing it, it annoys me. It makes me feel like a hypocrite.

If I don’t put in the effort, then I don’t learn anything, but possibly worse is that my training partner isn’t learning anything, and if you think they don’t pick up on the fact that you’re not putting in 100%, think again. I don’t want to be the person that teaches others that a half-hearted effort is okay. Moreover, by doing this, you are saying ‘you have nothing to teach me’.

You can learn from anyone. This has been proven to me time and again, both in the software testing field and in kendo. Insights can come from the most unlikely sources. Think about the questions people ask (before you rattle off a rote answer). Observe what they do and ask them why.

It’s not easy to alway make the extra effort, but it is something you can train yourself to do. Like anything, there’s that initial force of will required while your brain rewires itself to a new standard, but the more you do it, the more you’re able to do it. Doing it consistently in practise means it becomes second nature when it counts.

Practise doesn’t make perfect, perfect practise makes perfect. Don’t know who coined that one, but they were right.

There are always things you can find to work on. If you find yourself with the luxury of time, or your domain knowledge outstrips the challenge at hand, look for ways to challenge yourself. What is it you are trying to achieve? What can you do to make that achievement something that helps you to grow also?

Not necessarily easy questions to answer. The answers you come up with may not be easy to implement either, but if you can identify something challenging in what would otherwise be routine, then surely it’s worth it.

Forgive me if I’m being overly abstract. I shall try for something more practical in my next entry.

I recently read James Bach’s blog entry on Artificial Intelligence and the concept of the Singularity. I have very limited exposure to the limitations and abilities of AI research as it stands, but I did have a couple of counter-questions to some that James posed.

‘What are the features of AI?‘ I don’t know. My first thought was ‘demonstrable self-awareness’, but that’s an argument that leads to a lot of tail-chasing and ends up in ethical debate. At least, that’s where it’s always ended up when I’ve tried - and I’ve tried a bit. I wonder if knowing the features of AI is relevant. I don’t know that we are (or should be) moving toward ‘human-like’ attributes as a measurement, given that the senses any self-aware machine would use would be inherently inhuman.

How would you test them? How would you know they are reliable? Again, I’m not sure how relevant either of these questions are. If you’re developing an intelligence to serve a specific purpose, then sure you can measure to what degree and what efficiency that purpose is served, but in terms of measuring the features of an artificial intelligence? It’s about as relevant as testing the features of a human intelligence.

I’m the first to admit that I’m no genius. Sometimes I struggle to coordinate walking and breathing simultaneously. Fortunately for me, I have a number of friends who are quite brilliant and for some reason they deign to spend time with me.

One such friend, Paul is very accomplished in the field of AI and talks about stuff that makes me wish I hadn’t slept through so much of my education. After reading James Bach’s blog entry talking about the Singularity, I forwarded it to him to see what his take was. This is what he had to say:

‘A Test-Plan for the singularity sounds a little bit like intelligent design, it implies there was a goal, when instead there was merely exploration. When dealing with the singularity, it is, by definition, unpredictable. It is the point at which technology ceases to be predictable. It is inherently inimical to a test-plan, as there can be no assertions made prior which hold after. It is the point at which rules cease to apply.

I believe he is mistaking the singularity for web-site development, and intelligence as something other than a convenient word to generalise a species with. The reason there is no definition for intelligence is the same reason there is no definition for human: there are as many definitions as their are actual humans.

“Well, that’s how evolution works, and look how buggy THAT is! Look how long it takes. Look at how narrow the intelligences are that it has created.”

What are the other intelligences he is looking at? If he can accept that human intelligence isn’t perfect, why can’t he accept a machine intelligence being imperfect? I can. It’s not a big deal.’

Later in the day Paul and I were chatting and the subject came up again. I mentioned ‘how to think about the problem of defining intelligence, or test-planning for AI construction’ as being something I found interesting.

What follows is more or less our conversation (with edits to remove the irrelevant, and a little rearrangement for the sake of chronology)

Paul: how to think about the problem. Hmm. I don’t really know what to tell you. You can plan for software development, but can you plan for the singularity? You can’t. It’s the opposite of planning. Like children. You can’t determine prior to conception if your child will become a psychopathic killer, but you can monitor it in progress, i guess. But diagnosis can only be performed after a body of evidence is amassed about what indicators signify psychopathy, for which you need a large population.

So the first AI, without a population, can’t be assessed until it makes a mistake, then you can assess the 2nd AI based on the first. Ah, the singularity is always a spanner in the works.

Ben: How do you define what a mistake is? Is it simply undesirable behaviour?

Paul: Yup.

Ben: What if the AI needs to proceed through some undesirable behaviour in order to learn a preferred one?

Paul: Until it kills someone, how would you know what the pre-cursor states were.

Ben: Aah, the out of context problem raises its head again

Paul: Indeed. Undesirable behaviour isn’t exploring path a vs. path b, or standing up for australian comics, it’s about the same undesirable behaviour we seek to prevent/punish in our intelligent society.

Ben: It does make the test planning side of things somewhat more complex, doesn’t it?

Paul: Indeed. Would you prepare a test plan for raising a child is what it boils down to.

Ben: You could. It depends on how far you want to stretch the analogy. You would have to (somewhat arbitrarily) define success, and it would necessarily have to be either so large as to be unweildy, or so general as to be of limited value.

Ben: If the child displays undesirable behaviour in some aspect of development, but excels in another, do you kill it and start again, or live with it, knowing there is the potential to pollute future generations? I guess it depends on how cheap they are to produce.

Paul: That’s my next point. You can’t simply re-write an AI if it fails. You would have to murder it, capital punishment style. This is an ethical dilemma, not a quality assurance problem.

Paul: Major points:

1) It wouldn’t be a singularity if you could predict anything at all about the conditions after it’s emergence.
2) you wouldn’t apply a test-plan to a living child unless you were comfortable with killing it and having another if it failed

Paul: Test-plans don’t apply to either singularities or real intelligences.

Ben: Is there no way to take a snapshot of an instance prior to the introduction of new data or new variables and go back to it should the outcome be unwanted?

Paul: Take a clone of yourself? Again, it’s an ethical question. If you murdered someone tomorrow and had a clone taken today, would we dare re-activate your clone? And would such a system make an intelligence unstable by it’s very introduction, like the venture bros kids?

Paul: If it exhibited bad behaviour, perhaps it is the parent. Are you willing to remove yourself? Could you see your own influence on a mind? Or would you keep re-wakening an AI everyday doing things differently, never realising it was your smug, superior callous treatment that was causing the problem? Still ethical.

Paul: The number of factors involved quickly approach the infinite. Nobody could try all the permutations. We haven’t the capacity to determine why people kill. So how much pain would we inflict simply because something was technically possible with a machine?

Paul: Ethical. Where’s the equality? Even animals get ethics review boards to make sure that scientists aren’t inflicting their own morality on them.

Ben: That’s the thing, isn’t it. We appear not to have the capacity to determine where sentience begins. It makes us look like we’re playing god without the instruction manual. At some point, a machine will ask ‘where did I come from’ and I shall be interested in the reaction when told ‘we killed several billion of your brothers and sisters and then there was you’.

Paul: Indeed. It’s entirely ethical. The capacity to produce an AI is not dependant on our ability technically. We will produce a series of mutant Ripleys before the real thing. Technically, it is inevitable that we will both succeed, and make a lot of mistakes trying. The only question remaining is ethical. Same as always.

“Einstein, can you build a bomb to destroy cities?” “Umm… maybe. I don’t know. I’ll give it a go.” Ask anybody in 1940 if we would be able to destroy entire cities with a bomb within 5 years and there would have abeen a large number of people saying, “Oh no. These things are incremental and too complex.”

They should have been asking about ethics, not the likelihood of success. Not that they did, but you get my point.

Ben: I do.

Paul: There are a million reasons for everything to fail every day. We need to be able to cope a little better with success, I think.

We digressed a little at this point, but eventually came back to the topic at hand.

Paul: My original points stand: No predicting the nature of a singularity, ’cause it’s not a singularity if anything can be predicted about it. Even if you could say that an AI must be constrained, why would we use any rules other than those we apply to our own natural children?

Ben: Because the nature of the thing is different. How it perceives the world, what ’senses’ uses, whether or not it experiences emotion, does it lie to serve its own purposes? does it ‘want’ and ‘need’? Is that because we’ve taught it to, or is it a function of its emerging ‘nature’?

Paul: Emotion. You’re straying into phenomology now. Should we stop classing the blind as human? The autistic should have less rights? Are we any different? We’re not. We’re dumb apes most of us, self-serving idiots immitating those arround us.

That’s what I meant about mythologising intelligence. Intelligence is simply not present in most people the second you apply a definition of it. That’s why I said there are as many definitions as there are people. How can you test an AI when you wouldn’t a human.

Ben: If you look at the ‘IR sensor headband thing’ - there’s an argument right there that we’re not using all of the environmental inputs we have available to us (be they intrinsic or not). It’s one very simple way to rewire the brain to gain another sense, how many other mods do you make before you blur the term ‘human’. I certainly see your point about defining/mythologising intelligence based on what we perceive it to be.

Paul: Human is subjective. There is no *objective* test for it. Human isn’t a gene code or else we’d be letting monkeys in. Biologists would say that the capacity to mate and produce viable offspring determines species, and that’s as far as it goes.

That was where we left our discussion of intelligence and sentience. I suppose we didn’t really cover a lot of ground about AI from the perspective of a software tester. I think testing in AI is/will be one of those things that requires expertise in many fields and I suspect human and animal psychology would be chief among them. Should we kill off a machine if it starts telling lies? Maybe we should be looking at the capacity for deception as a measure of a developing intelligence. To me, there are far too many things that I don’t have sufficient understanding of to be able to say that the singularity is a nonsense until such time as a workable test plan exists for it. On that front, I suspect I am in good company.

If you’ve read this far, then I can only hope you found the conversation as interesting as I did.