Wednesday, January 26, 2011

Selenium and Choosing a Framework

         After seeing some people asking questions about choosing/developing a framework to use Selenium, I thought, I could offer few tips. This may look like ways to creating a new automation product, however, customizing the tool to the needs of app and the team, will decide the success of test automation. It is like bespoke vs ready-made.
       I've separated this into two categories, one is how to run and other is how to have a robust, reusable Selenium tests. If you choose to go with this way, forget about Se IDE. This is not to discredit the IDE but to understand that record and playback, wont work in this approach. If you think that you can record and put the cases in the script, sorry to bust the bubble. It wont work.

I. Running the Selenium scripts.
      I will list the available choices. These are specific to Java.

  • I can live with the default main method as I do in my app - No need for any existing frameworks. You can decide on your own or use the Java Reflect API.
  • I want run with minimum work. - Choose JUnit or TestNG, again depends upon how you want to organize tests.
  • I want to run using CI - Plug-ins are available for Maven/Hudson
II. For a robust Selenium tests.
     This portion will also cover general test automation best practices, however, I will highlight how to get that using Selenium.

      A. Understand the App you're going to automate:
           I've seen a number of people trying to automate, without understanding the UI/App. It will be a spectacular failure because, that is wrong way to start with. UI can be a heavily ajaxified , may have lot of dynamic ids or app may be in early stages so UI may change infrequently.

          For ajaxified pages using CSS selectors is the best option. Going with Sizzle/jQuery based CSS selectors is nice. If the app has lot of dynamic elements, talk to your developers to understand the underlying pattern. Having a custom locator using that pattern will solve lot of issues. Don't use Xpaths with div or span in the locators, if that div is changed to span, your tests are broken. There wont be any visible changes but tests still wont run. So think before using those locators.

         B. What is testing requirement now and what are the plans for future?
              Now the hardest part. Deciding what you're going to do and what else will be needed in future will decide time and difficulty of the your test automation. Let me list few options.

  • Access to database/csv/excel etc
  • Integration with logging framework if you need to print info message wherever needed
  • Integrating with speed load testing frameworks such as JMeter
  • Integration with HTML/JS validation frameworks such as HtmlUnit. This way you can do unit testing of the page.
Once you've decided on these, taking care of options to handle exceptions/alerts, capture screen shots, generating test results/reports come to play. If the framework/model doesn't handle these things then it will not reduce work or add value to the testing.

III. Page Objects or BDD?
       As you know in Selenium 2, Page Objects come with it. However, by separating element locators from code using properties or xml file you can get benefit of the Page Objects without going for it. You can point out that Page Objects can do much more that, however, my fingers crossed on that issue.
       If it so, why BDD? Using BDD model you can separate flow of actions from the code, that extends the concept of Page Objects. Moreover, creating and maintaining code written using BDD is easy. And if you extend that write your own framework for BDD, you can even create the customizations you need without trying to predict what you will need in future.  

In the end, creating tests which will bring value to testing, app and the team matters. So choose wisely.


raja's shared items

My "Testing" Bundle