Sunday, April 17, 2011

Testing Vs Automation Checking

    The argument that, testing is not checking has generated lot of content. While reading the whole debate, I thought, we’re missing some pieces. Hence this post.

See the Testing vs Checking posts here,
http://www.developsense.com/blog/2009/08/testing-vs-checking/
http://www.developsense.com/blog/2009/09/tests-vs-checks-motive-for/
http://thetesteye.com/blog/2010/03/the-testing-vs-checking-paradox/
http://www.testingreflections.com/node/view/8282
http://www.testingreflections.com/node/view/8243
http://www.infoq.com/news/2009/12/testing-or-checking
http://www.stickyminds.com/sitewide.asp?ObjectId=16684&Function=edetail&ObjectType=COL

A simple summary of the discussion. Testing is a learning process, that can’t be automated. Testers can only automate checking not testing. Checking means verifying a button/link/functionality is there or works expected. Other words, human intelligence can’t be automated. Yes, that is the point. However, it missed an other point. Human intelligence can be used in creative areas rather than doing mundane repetitive actions.

Am I saying automated testing can be called as Tool Assisted Testing? Nope.

Let us take Programming. A programming language has control flow, data structures, memory management,error reporting etc. If you look at the abstract level, all the actions done by the programming language is actually checking.
Get an input ---> Check the input –> Do Some actions based on that ---> Show/Pass the output
Which means, if a programmer learns all these things, can he write a great application using this? Nope.
Wisdom of what to put where and how, will differentiate between a good programmer and bad one. Handling errors, designing a UI will make a difference.

Coming back to the point, the purpose of testing starts when tester sees an error and decides what to do with that. That particular error may be a bug or UI needs some improvement or may lead to finding an another bug.

Enter automation. If automation can capture the errors in UI, shows a nice report along with screen shots, log entries etc, tester can analyze it and chart a future course of action. In essence, time spent on producing those errors may be or can be saved. We know the computer is dump, can’t think. So why assume it will assist us in finding the bugs?

Using automation tools in right way and doing more work in less time (productivity) will give us more time to do testing. Human intelligence can be used to do more productive and creative works.

Lets us see a quote from Seth Godin, to understand what I mean.
And email is not a faster fax. And online project management is not a bigger whiteboard. And Facebook is not an electronic rolodex.
Play a new game, not the older game but faster.
In the same way, automation is not faster testing. For example, take Static code analysis. It can find flawed code that may be buggy. Using it to scan the entire repository will make job of the coder easy to find bugs. More over, if he makes some mistake it can be found using this. Code review will be a easier with static code analysis.

Another example is continuous integration tools. It can start the testing when new code checked in.

Test Automation is not faster testing. It enables more testing.

ShareThis

raja's shared items

There was an error in this gadget

My "Testing" Bundle