Monday, August 29, 2011

Test Automation using Tool or API

This debate between Record-Playback  and Scripting is going for long. I have written about that too. Now, after Selenium 2 release, some people asked me, why should one needs a framework such as Selenium? Comments range from, "it is hard to understand", "can this be simple?" or "I want to get started in easy way".

Those comments are seems to valid if you're not burned your finger in testing an enterprise app or writing automation code for thousands of tests. Because, only after automating 10 or 20 % of the cases, you'll come to know that, some actions can't be automated because there is no option in the tool to do that. Then, finding out that there is no way you can do that, will be an another big surprise.

So, if we go in the Selenium way i.e the framework way, it is possible to do that? Yes. Because, as a Framework, it doesn't try to predict the permutations or combinations for all the test automation scenarios.

There are lot ways, I want to test my application. I may use, keyword driven testing, data driven testing or scenario driven testing or some other driven testing. I may want my tests developed in BDD or do a acceptance testing. So can one tool/software give the options for all? It will actually make the product bloated.

Take the analogy of WYSIWYG editors for HTML. Ican write html in a text editor or use those editors to speed up the work(drag and drop). It offers the flexibility and best of both worlds. Same can be said to IDE for programming languages. If something doesn't work in IDE, you can edit the file using text editor. If that possible and good for developing software, why it is not good to do that in Test Automation?

Another example is using BDD. I want to use BDD but I want to use it my way or extenstion of something I already used, not the way like Rspec or JBehave/easyb. So tying these wont work for me.

Now, if we deduce, the issue is "what to do" is combined with "how to do" causing most of the issues in test automation rather than technical issues such locating the elements in the UI.


What do to= Actions done in the UI such as Click, Select etc

How to do= Sequential use of the above actions

Separating "what to do" and "how to do" will bring lot of benefits. I would like to add another example to emphasis the point. Catching the JavaScript errors is a valuable improvement in automation. If I can tweak the Click function or extend, so that, I can check for every click, is there any JavaScript errors.

Here are the requirements for a ideal automation framework

1. All actions should be available independently . No dependencies .

2. Should only do UI actions. Operations such as I/O, connecting to DB should be written by users.

3. Should be extensible.Adding a new action should be easier.

4. Should use standard languages and standard methods to locate the elements. Ex. Java/Python to script. CSS/DOM/XPATH to locate the elements. No vendor scripts.

 5. Running the tests should be independent.


raja's shared items

My "Testing" Bundle