Selenium Testing beyond GUI Browsers.

“Selenium automates browsers”. So goes the introductory line on Seleniumhq.org. To what extent is this a limitation?

The topic of the blog provokes questions – Can we structure Selenium-based test framework to test beyond GUI browsers? If not, at the very least, how can we improve test-effectiveness by extending an existing Selenium test-suite to work with other test-actions that do not use browsers.

The first question above needs in-depth technical discussion. We focus on the second one here which can be addressed at a more conceptual level.

We have an example below which illustrates how one can improve the efficiency and coverage of automation by going beyond the Browser GUI and enabling backend operations.

Consider a simple web-application which has Selenium test suite containing hundreds of automated scripts that need to run daily. In addition, there are test-scenarios that need to be selectively executed during the day using some of the scripts. The framework is robust and the application relatively stable.

All scripts need to necessarily navigate across the web-application to get to specific web-pages. There are often many test scenarios that need to be executed on a specific web-page. The time for the script to get to that specific web-page depends on the performance of the application and latency in loading the intermediate web-pages. This would in turn increase the overall time needed to execute the automation suite when the script count is in hundreds. How can this be reduced?

Problem Statement: how do we limit the steps of navigating the web front-end using Selenium scripts so as to reduce the overall test-execution time?

Refer the illustration below.

Selenium Beyond GUI

Total Execution Time – 2 minutes.

We have the above scenario wherein there are parameters to be set on Web-Page-B which are necessary for validations on Web-Page-C. Every individual script logs-out and logs-on to the application, and total time taken is close to 2 minutes.

Now if there are 50 scripts that set different parameter combinations, the Selenium suite that runs scripts on the browser GUI would potentially take 50 * 2 = 100 minutes just for navigating back and forth on the web pages, especially if we need to log in and log out after every script.

The actual verification point however is only on Web-Page-C for every kind of parameter setting.

Selenium Beyond GUI img 2

Solution:

The parameter setting could be handled by Python or Perl scripts running in the backend. This would then cut down the navigation on the GUI.

Selenium Beyond GUI -img 3

The test flow is handled as below.

·       We test the end-to-end GUI navigation one time. The first test scenario covers this part.

·       At the same time, we trigger a script that directly accesses the backend. There could be several ways to do this – server side scripts, API calls, database queries… This depends on the application architecture and what is being tested.

·       The parameter is set at backend, the validation is done by the Selenium script on web-page C, this step is iterated over 50 parameters to be set for the 50 scenarios to be tested

The key here is to enable the automation framework to detect when the backend parameter is set to progress onto GUI validation, and then continue iterating between the two steps.

Total execution time now comes down drastically since webpage navigation is no longer needed.

This is an example of how existing Selenium suite can be extended with backend operations that improve automation efficiency and overall test effectiveness. The concept is proven; the implementation is heavily dependent on application architecture and specific test scenarios.

Test Automation in Selenium Using Page Object Model and Page Factory

Writing automated tests is more than just a luxury for any agile software development team. It is a need, and is an essential tool to find bugs quickly during early phases of software development cycles. When there is a new feature that is still in development phase, developers can run automated tests and see how other parts of the system are affected by those changes.

Through test automation, it is possible to lower the cost of bug fixing and bring overall improvement to quality assurance (QA) process. With proper tests, developers get a chance at finding and resolving bugs even before it gets to QA. Test automation further helps us to automate test cases and features that are constantly regressing. This way QAs have more time in testing other parts of the application. Moreover, this helps in ensuring quality of the product in production releases. As a result, we get products that are effectively more stable, and a QA process that is more efficient.

clip_image001

Selenium simplifies test automation for web applications

Although writing automated tests may seem like an easy task for developers and engineers, there is still the possibility of ending up with poorly implemented tests, and the high cost of code maintenance in any agile process. Trying to constantly deliver changes or features in any agile development project can prove to be costly when tests are involved. Changing one element on a web page that 20 tests rely on will require one to go through these 20 test routines and update each one to adapt to this newly introduced change. Not only can this be really time consuming, but a serious de-motivating factor when it comes to implementing automated tests early on.

But, what if we could make the change in one place only, and have every relevant test routine use it? In this article, we will take a look at automated tests in Selenium, and how we can use Page Object models to write maintainable and reusable test routines.

Page Object Model in Selenium

Page Object model is an object design pattern in Selenium, where web pages are represented as classes, and the various elements on the page are defined as variables on the class. All possible user interactions can then be implemented as methods on the class:

clickLoginButton();
setCredentials(user_name,user_password);

Since well-named methods in classes are easy to read, this works as an elegant way to implement test routines that are both readable and easier to maintain or update in the future. For example:

In order to support Page Object model, we use Page Factory. Page Factory is an extension to Page Object and can be used in various ways. In this case we will use Page Factory to initialize web elements that are defined in web page classes or Page Objects.

Web page classes or Page Objects containing web elements need to be initialized using Page Factory before the web element variables can be used. This can be done simply through the use of initElements function on PageFactory:

LoginPage page = new LoginPage(driver);
PageFactory.initElements(driver, page);

Or, even simpler:

LoginPage page = PageFactory.intElements(driver,LoginPage.class)

Or, inside the web page class constructor:

public LoginPage(WebDriver driver) {           
         this.driver = driver; 
         PageFactory.initElements(driver, this);
}

Page Factory will initialize every WebElement variable with a reference to a corresponding element on the actual web page based on configured “locators”. This is done through the use of @FindBy annotations. With this annotation, we can define a strategy for looking up the element, along with the necessary information for identifying it:

@FindBy(how=How.NAME, using="username")
private WebElement user_name;

Every time a method is called on this WebElement variable, the driver will first find it on the current page and then simulate the interaction. In case we are working with a simple page, we know that we will find the element on the page every time we look for it, and we also know that we will eventually navigate away from this page and not return to it, we can cache the looked up field by using another simple annotation:

@FindBy(how=How.NAME, using="username")
@CacheLookup
private WebElement user_name;

This entire definition of the WebElement variable can be replaced with its much more concise form:

@FindBy(name="username")
private WebElement user_name;

The @FindBy annotation supports a handful of other strategies that make things a bit easier:

id, name, className, css, tagName, linkText, partialLinkText, xpath

@FindBy(id="username")
private WebElement user_name; 

@FindBy(name="passsword")
private WebElement user_password;

@FindBy(className="h3")
private WebElement label;

@FindBy(css=”#content”)
private WebElement text;

Once initialized, these WebElement variables can then be used to interact with the corresponding elements on the page. The following code will, for example:

user_password.sendKeys(password);

… send the given sequence of keystrokes to the password field on the page, and it is equivalent to:

driver.findElement(By.name(“user_password”)).sendKeys(password);

Moving on, you will often come across situations where you need to find a list of elements on a page, and that is when @FindBys comes in handy:

@FindBys(@FindBy(css=”div[class=’yt-lockup-tile yt-lockup-video’]”)))
private List<WebElement> videoElements;

The above code will find all the div elements having two class names “yt-lockup-tile” and “yt-lockup-video”. We can simplify this even more by replacing it with the following:

@FindBy(how=How.CSS,using="div[class=’yt-lockup-tile yt-lockup-video’]")
private List<WebElement> videoElements;

Additionally, you can use @FindAll with multiple @FindBy annotations to look for elements that match any of the given locators:

@FindAll({@FindBy(how=How.ID, using=”username”),

@FindBy(className=”username-field”)})
private WebElement user_name;

Now that we can represent web pages as Java classes and use Page Factory to initialize WebElement variables easily, it is time we see how we can write simple Selenium tests using PO and PF patterns.

Simple Test Automation Project in Java

For our simple project let’s automate developer sign up for Toptal. To do that, we need to automate the following steps:

· Visit www.toptal.com

· Click on the “Apply As A Developer” button

· On Portal Page first check if it’s opened

· Click on the “Join Toptal” button

· Fill out the form

· Submit the form by clicking on “Join Toptal” button

Setting Up a Project

· Download and install Java JDK

· Download and install InteliJ Idea

· Create a new Maven project

· Link “Project SDK” to your JDK, e.g.: on Windows “C:\Program Files\Java\jdkxxx”

· Setup groupId and artifactId:

<groupId>SeleniumTEST</groupId>
<artifactId>Test</artifactId>

· Add dependencies Selenium and JUnit Maven in your project POM file

   <dependencies>
        <!-- JUnit -->         
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>${junit.version}</version>
            <scope>test</scope>
        </dependency>
 
        <!-- Selenium -->
 
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-firefox-driver</artifactId>
            <version>${selenium.version}</version>
        </dependency>
 
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-support</artifactId>
            <version>${selenium.version}</version>
        </dependency>
 
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>${selenium.version}</version>
        </dependency>
 
    </dependencies>

Replace Selenium version and JUnit Version with latest version numbers that can be found by searching for JUnit Maven on Google and on Selenium site.

At this point, if auto build is enabled, dependencies should start downloading automatically. If not, just activate Plugins > install > install:install under the Maven Projects panel on the right side of your IntelliJ Idea IDE.

clip_image002

Once the project has been bootstrapped, we can start creating our test package under “src/test/java”. Name the package “com.toptal”, and create two more packages under it: “com.toptal.webpages” and “com.toptal.tests”.

clip_image003

We will keep our Page Object/Page Factory classes under “com.toptal.webpages” and the test routines under “com.toptal.tests”.

Now, we can start creating our Page Object classes.

HomePage Page Object

The very first one we need to implement is for Toptal’s homepage (www.toptal.com). Create a class under “com.toptal.webpages” and name it “HomePage”.

package com.toptal.webpages;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;
import org.openqa.selenium.support.How;
import org.openqa.selenium.support.PageFactory;
 
 public class HomePage {
   private WebDriver driver;
 
   //Page URL
   private static String PAGE_URL="https://www.toptal.com";
 
   //Locators
 
   //Apply as Developer Button
   @FindBy(how = How.LINK_TEXT, using = "APPLY AS A DEVELOPER")
   private WebElement developerApplyButton;
 
   //Constructor
   public HomePage(WebDriver driver){
       this.driver=driver;
       driver.get(PAGE_URL);
       //Initialise Elements
       PageFactory.initElements(driver, this);
   }
 
   public void clickOnDeveloperApplyButton(){
 
       developerApplyButton.click();
 
   }
}
Determining Element Locators

On Toptal’s homepage we are interested about one element in particular, and that is the “Apply as a Developer” button. We can find this element by matching the text, which is what we are doing above. While modeling web pages as Page Object classes, finding and identifying elements can often become a chore. With Google Chrome or Firefox’s debugging tools, this can be made easier. By right clicking on any element on a page, you can activate the “Inspect Element” option from the context menu to find out detailed information about the element.

One common (and my preferred) way is to find elements using Firefox’s FireBug extension, in combination with Firefox web driver in Selenium. After installing and enabling FireBug extension, you can right click on the page and select “Inspect element with FireBug” to open FireBug. From the HTML tab of FireBug, you can copy the XPath, CSS Path, Tag name or “Id” (if available) of any element on the page.

clip_image004

By copying the XPath of the element in the screenshot above, we can create a WebElement field for it in our Page Object as follows:
@FindBy(xpath = “/html/body/div[1]/div/div/header/div/h1”)
WebElement heading;
Or to keep things simple, we can use the tag name “h1” here, as long as it uniquely identifies the element we are interested in:

@FindBy(tagName = "h1")
WebElement heading;
DeveloperPortalPage Page Object

Next, we need a Page Object that represents the developer portal page, one that we can reach by clicking on the “Apply As A Developer” button.

On this page, we have two elements of interest. To determine if the page has loaded, we want to verify the existence of the heading. And we also want a WebElement field for the “Join Toptal” button.

package com.toptal.webpages;
 
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;
import org.openqa.selenium.support.PageFactory;
 
public class DeveloperPortalPage {
   private WebDriver driver;
 
   @FindBy(xpath = "/html/body/div[1]/div/div/header/div/h1")
   private WebElement heading;
 
   @FindBy(linkText = "JOIN TOPTAL")
   private WebElement joinToptalButton;
 
   //Constructor
   public DeveloperPortalPage (WebDriver driver){
       this.driver=driver;
 
       //Initialise Elements
       PageFactory.initElements(driver, this);
   }
 
   //We will use this boolean for assertion. To check if page is opened
   public boolean isPageOpened(){
       return heading.getText().toString().contains("Developer portal");
   }
 
   public void clikOnJoin(){
       joinToptalButton.click();
   }
}
DeveloperApplyPage Page Object

And finally, for our third and last page object for this project, we define one that represents the page containing developer application form. Since we have to deal with a number of form fields here, we define one WebElement variable for every form field. We find each field by their “id” and we define special setter methods for every field that simulate keystrokes for the corresponding fields.
package com.toptal.webpages;
 

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;
import org.openqa.selenium.support.PageFactory;
 
public class DeveloperApplyPage {
   private WebDriver driver;
 
   @FindBy(tagName = "h1")
   WebElement heading;
 
   @FindBy(id="developer_email")
   WebElement developer_email;
 
   @FindBy(id = "developer_password")
   WebElement developer_password;
 
   @FindBy(id = "developer_password_confirmation")
   WebElement developer_password_confirmation;
 
   @FindBy(id = "developer_full_name")
   WebElement developer_full_name;
 
   @FindBy(id = "developer_skype")
   WebElement developer_skype;
 
   @FindBy(id ="save_new_developer")
   WebElement join_toptal_button;
 
 
   //Constructor
   public DeveloperApplyPage(WebDriver driver){
       this.driver=driver;
 
       //Initialise Elements
       PageFactory.initElements(driver, this);
   }
 
   public void setDeveloper_email(String email){
       developer_email.clear();
       developer_email.sendKeys(email);
   }
 
   public void setDeveloper_password(String password){
       developer_password.clear();
       developer_password.sendKeys(password);
   }
 
public void  setDeveloper_password_confirmation(String password_confirmation){
       developer_password_confirmation.clear();
       developer_password_confirmation.sendKeys(password_confirmation);
   }
 
   public void setDeveloper_full_name (String fullname){
       developer_full_name.clear();
       developer_full_name.sendKeys(fullname);
   }
 
   public void setDeveloper_skype (String skype){
       developer_skype.clear();
       developer_skype.sendKeys(skype);
   }
 
   public void clickOnJoin(){
       join_toptal_button.click();
   }
   public boolean isPageOpened(){
       //Assertion
       return heading.getText().toString().contains("Apply to join our network as a developer");
   }
}
Writing a Simple Test

With Page Object classes representing our pages, and user interactions as their methods, we can now write our simple test routine as a series of simple method calls and assertions.

package com.toptal.tests;
 
import com.toptal.webpages.DeveloperApplyPage;
import com.toptal.webpages.DeveloperPortalPage;
import com.toptal.webpages.HomePage;
import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
 
import java.net.URL;
import java.util.concurrent.TimeUnit;
 
public class ApplyAsDeveloperTest {
   WebDriver driver;
 
   @Before
   public void setup(){
       //use FF Driver
       driver = new FirefoxDriver();
       driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
   }
 
   @Test
   public void applyAsDeveloper() {
       //Create object of HomePage Class
       HomePage home = new HomePage(driver);
       home.clickOnDeveloperApplyButton();
 
       //Create object of DeveloperPortalPage
       DeveloperPortalPage devportal= new DeveloperPortalPage(driver);
 
       //Check if page is opened
       Assert.assertTrue(devportal.isPageOpened());
 
       //Click on Join Toptal
       devportal.clikOnJoin();
 
       //Create object of DeveloperApplyPage
       DeveloperApplyPage applyPage =new DeveloperApplyPage(driver);
 
       //Check if page is opened
       Assert.assertTrue(applyPage.isPageOpened());
 
       //Fill up data
       applyPage.setDeveloper_email("dejan@toptal.com");
       applyPage.setDeveloper_full_name("Dejan Zivanovic Automated Test");
       applyPage.setDeveloper_password("password123");
       applyPage.setDeveloper_password_confirmation("password123");
       applyPage.setDeveloper_skype("automated_test_skype");
 
       //Click on join
       //applyPage.clickOnJoin(); 
   }
 
    @After
    public void close(){
          driver.close();
       }
   }
Running the Test

At this point, your project structure should look like this:

clip_image005

If you want to run the test, select “ApplyAsDeveloperTest” from the tree, right click on it and then select Run ‘ApplyAsDeveloperTest’.

clip_image006

Once the test has been run, you can see the results in the lower-left corner of your IDE:

clip_image007

Conclusion

Page Object and Page Factory make it easy to model web pages in Selenium and test them automatically and make the life of both developers and QAs much more simpler. When done right, these Page Object classes can be reused across your entire test suite and to give yourself the opportunity to implement automated Selenium tests for your projects early on, without compromising agile development. By abstracting away user interactions in your page object models and keeping your test routines light and simple, you can adapt your test suite to changing requirements with little effort.

I hope I have managed to show you how to write nice and clean test code that is easy to maintain. I will end the article with my favorite QA quote:

Think twice, code once!

This article originally appeared on Toptal.

Selenium versus HP LeanFT

Selenium has carved a niche in the software testing tools world, and has a dedicated user base with consistently increasing adoption in the last few years. While this tool was always popular with Open Source enthusiasts since the RC days, we now have increased acceptance in enterprises as well. In the last couple of years, quite a few Fortune 500 companies and banks have diversified their skill-base and tool portfolio with Selenium, in addition to traditional HP toolset.


Let’s look at a recent product offering in the market-place, which could be very relevant to continued growth of Selenium user base. This post describes a brief analysis of HP LeanFT with respect to Selenium. Unsure if Learn-Selenium blog is the right medium for comment on HP tool, but it seems that this tool is HP’s response specifically to counter the increasing popularity of Selenium in the testing world.

Selenium – what has worked

1. Cost. Cost. Cost.

Cost by far remains the biggest differentiator for Selenium. Being open-source, this becomes the automation tool of choice for browser based applications in small and medium enterprises. Vibrant user community and strong support base help mitigate concerns around open source usage in enterprises. HP licensing is disproportionately expensive.

2. Object Identification

As web technologies get advanced, we have third party toolsets that cause issues in object identification during automation with HP QTP-UFT. AngularJS, Ajax, Oracle Forms are examples. While HP keeps refining with every version, there are easy alternatives. Selenium uses XPATH, and identifies objects where we face challenges in detecting unique properties using QTP.

3. The Buzz around Dev-Ops

Dev-Ops is the approach of leveraging test assets and automation in Development and Operations. With increasing agile adoption, the lines blur between traditional roles of developer, functional tester, and test automation specialist.

Application Development Leads and Architects are interested in test-automation for continuous integration, build sanity, and unit-testing. This is a community with expertise in Java/C#, and very comfortable with IDEs like Eclipse. They find it difficult to digest that anything worthwhile can be done with VBscript. These stakeholders are often key influencers in management decision-making on Enterprise Tool Usage, leading to increased acceptability for Selenium in large enterprises.

4. Multi-browser Testing

Even today, Selenium is a clear winner in cross-browser testing against UFT. Multiple UFT add-ins have to be tried for different browser versions and we have compatibility issues. Examples – UFT 11.5 does not support Chrome v40, you need to downgrade to Chrome v36 for automating scripts, which would not be in sync with production. HP license upgrades do not keep pace with browser version changes.

Selenium – where it falters

1. End-To-End Automation

Large Enterprises have multiple applications under test and end-to-end testing flows that traverse more than one application. Any tool restricted to browser testing would limit coverage of automation. Example – A very common scenario in banking systems would be transaction initiated on front-end web application that would have validation step on mainframe and backend database.

2. Object Identification and Script Build Productivity

While XPATH usage helps identify problematic objects, we have lot of instances where QTP can easily get unique property index, which may be cumbersome in Selenium. HP QTP/UFT are feature rich and easy to use. Invariably, script build productivity is higher as compared to Selenium, although this would vary based on application under test.

3. Skill-base and staffing

HP QTP has been the industry leader since ages, and sourcing experienced automation testers skilled in coding with VBscript is relatively easier. In comparison, ramping a project team on Selenium skills may be more of a challenge. Note- this is a snapshot as of mid-2015, things change very fast.

HP LeanFT – What’s on offer

Circa 2015 July, HP has introduced LeanFT along with the UFT 12.5 upgrade. Refer the figure below.

image

    Figure: Reproduced from datasheet on HP home-site

We have detailed below features and observations of LeanFT, which bear relevance to the analysis above.

1. LeanFT provides Support for multiple IDEs (Eclipse, Visual Studio) and coding languages (Java, C#).

HP keeps pushing features every few years to retain its dominance (and premium licensing), BPT was introduced to sell the concept of BA–Tester, now LeanFT is built to whet the interest of the Dev-Tester. IDE and language flexibility would make the tool popular with the Application Developers community.

2. Dev-Ops and CI support. LeanFT supposedly integrates well with standard SCMs, build/deploy tools and approaches, as compared to UFT which is heavily ALM centric. Selenium was an easy choice compared to QTP in building CI/CD solutions closely integrated with dev-workflows. This may change with LeanFT and needs to be investigated.

3. Object identification & Multi-browser support. LeanFT has an object identification engine similar to Spy in UFT which is installed as plug-in to the IDE. This is an advantage and could be quite powerful. HP datasheet indicates LeanFT to be light weight tool with good cross browser support. This could potentially address a gap in HP toolset where Selenium has an edge over QTP.

4. Cost: LeanFT licenses are free for HP UFT12.5 users, this helps penetrate the existing user base in large organizations and halts the move to Selenium.

5. We have clean integration of LeanFT with UFT12.5, which aids in end to end automation. This permits automation beyond browser based applications, a clear advantage over Selenium in organizations like banks and insurance companies.

6. QTP-skilled staff have no learning curve to start automation using LeanFT. Existing resources can be used for automation, as against Selenium projects which need well-thought out staffing and training strategy.

To Conclude:

It seems that LeanFT has been specifically targeted at the Selenium user base – would be very interesting to see how this pans out in the marketplace in the next 12 months. Much of the recent increase in Selenium projects has been because of large organizations seeking to diversify their tools portfolios. Technology trends are extremely dynamic and you prepare today for anticipated changes or risk obsolescence. Would LeanFT stem the tide? – We would watch the events and follow up on this in another six months.