“Selenium automates browsers”. So goes the introductory line on Seleniumhq.org. To what extent is this a limitation?
The topic of the blog provokes questions – Can we structure a Selenium-based test framework to test beyond GUI browsers? If not, at the very least, how can we improve test-effectiveness by extending an existing Selenium test-suite to work with other test-actions that do not use browsers?
The first question above needs in-depth technical discussion. We focus on the second one here which can be addressed at a more conceptual level.
We have an example below which illustrates how one can improve the efficiency and coverage of automation by going beyond the Browser GUI and enabling backend operations.
Consider a simple web-application which has a Selenium test suite containing hundreds of automated scripts that need to run daily. In addition, there are test-scenarios that need to be selectively executed during the day using some of the scripts. The framework is robust and the application relatively stable.
All scripts need to necessarily navigate across the web-application to get to specific web-pages. There are often many test scenarios that need to be executed on a specific web-page. The time for the script to get to that specific web-page depends on the performance of the application and latency in loading the intermediate web-pages. This would, in turn, increase the overall time needed to execute the automation suite when the script count is in hundreds. How can this be reduced?
Problem Statement: how do we limit the steps of navigating the web front-end using Selenium scripts so as to reduce the overall test-execution time?
Refer to the illustration below.
Total Execution Time – 2 minutes.
We have the above scenario wherein there are parameters to be set on Web-Page-B which are necessary for validations on Web-Page-C. Every individual script logs-out and logs-on to the application, and total time has taken is close to 2 minutes.
Now if there are 50 scripts that set different parameter combinations, the Selenium suite that runs scripts on the browser GUI would potentially take 50 * 2 = 100 minutes just for navigating back and forth on the web pages, especially if we need to log in and log out after every script.
The actual verification point, however, is only on Web-Page-C for every kind of parameter setting.
The parameter setting could be handled by Python or Perl scripts running in the backend. This would then cut down the navigation on the GUI.
The test flow is handled as below.
· We test the end-to-end GUI navigation one time. The first test scenario covers this part.
· At the same time, we trigger a script that directly accesses the backend. There could be several ways to do this – server-side scripts, API calls, database queries… This depends on the application architecture and what is being tested.
· The parameter is set at the backend, the validation is done by the Selenium script on web-page C, this step is iterated over 50 parameters to be set for the 50 scenarios to be tested
The key here is to enable the automation framework to detect when the backend parameter is set to progress onto GUI validation, and then continue iterating between the two steps.
Total execution time now comes down drastically since webpage navigation is no longer needed.
This is an example of how existing Selenium suite can be extended with backend operations that improve automation efficiency and overall test effectiveness. The concept is proven; the implementation is heavily dependent on application architecture and specific test scenarios.