Automation Testing With Selenium, Cucumber & TestNG
Bharadwaj Pendyala
Posted On: February 18, 2019
115167 Views
13 Min Read
This article is a part of our Content Hub. For more in-depth resources, check out our content hub on Selenium Cucumber Tutorial.
Cucumber is a test automation framework which leverages Behavior Driven Development for collaboration in between Business and IT teams. It empowers a user to define an application’s behavior in plain English language which makes it easier for non-programmers to understand the acceptance criteria.
The core of cucumber has been developed in Ruby programming language however it supports all the majorly used programming languages for testing such as Java, C#, Python. In this article, I will be demonstrating automation testing with Selenium, Cucumber and TestNG.
If you’re new to Selenium and wondering what it is then we recommend checking out our guide – What is Selenium?
What Is Behavior Driven Development?
Behavior Driven Development is derived from Test Driven Development as a software development methodology. As a part of the Test Driven Development approach, the developer creates tests as a part of the acceptance criteria first. The developer will make sure the test is passed and will commit changes in the code. if required. Test Driven Development is a way to ensure the system meets its requirements by ensuring a 100% test coverage. It provides an edge over other methodologies in terms of finding defects early in the cycle which reduces the cost of finding bugs and refactoring improves the code.
Cons of Test Driven Development are the maintenance of tests created, it is a test-centric methodology, non-programmers might find it hard to understand for performing selenium automation testing. The test or acceptance criteria being targeted makes it hard for team collaboration. To overcome the above trouble heads, Behavior Driven Development was introduced to reduce the time to test, less code with more collaboration.
Advantages Of Using BDD For Automation Testing With Selenium & Cucumber
Collaborative Approach: Bridges the communication gap between stakeholder or business teams who can’t easily understand or read the code. This problem is addressed by defining the behavior of application under test in a ubiquitous language i.e. easily understandable.
Focuses more in terms of end-user experience or inclined more towards behavior specifications while compared to that of traditional Test Driven methodology. Tests are written more from end user’s point of view, making them easier to understand for performing Selenium automation testing.
With TestNG certification, you can challenge your skills in performing automated testing with TestNG and take your career to the next level.
Here’s a short glimpse of the TestNG certification from LambdaTest:
How Cucumber Leverages BDD For Selenium Automation Testing?
As discussed earlier, Cucumber is a test automation tool that supports Behavior Driven Development. It makes use of user-defined specifications to validate the application under test. Before we step into performing automation testing with Selenium & Cucumber, let me tell you about the three major parts of the cucumber framework i.e. Feature file, Step Definitions, and Test Runner file.
Feature File would enable the user to define the behavior of the application in plain English text with help of Gherkin Language. Gherkin is used to define executable specifications with the help of a predefined set of contextual keywords. Keywords involved are demonstrated with the help of the below example:
Feature: High-level description of a software feature or behavior of application under test.
Scenario: Feature file can contain more than one scenario which acts as a test case derived from test scenarios/behaviors. It also symbolizes and acts as one or more user perspectives involved.
Annotations: Keywords which hold a specific meaning and would help and defined the meaning of a scenario. Sample keywords involved are GIVEN, WHEN, THEN & AND.
- GIVEN: Define pre-requisites of the application.
- WHEN: Trigger point for a scenario execution can be related more towards steps to be executed.
- THEN: Defined an outcome more like the expected result.
- AND: Acts as a logical bridge AND in between two statements.
Scenario Outline: It’s a template provided to carry out scenario execution, test data table provided in the Examples section would replace variables/arguments created making each individual row in a table as a unique scenario.
Tags: It paves the way to organize your tests based on your tag definition.
Quoted an example related to the feature file below which involves all the above-mentioned keywords.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
Feature: Place order on the Amazon website. @SmokeTest Scenario: Validate if the guest user is able to add a product to cart. GIVEN user is logged onto the Amazon website as a guest user. WHEN user searches a product on the homepage. THEN user should be able to view product information related to product searched. AND user click on add to cart button THEN user verifies if the product is added to cart Scenario Outline: Validate if a registered user is able to place an order. GIVEN user is logged onto the Amazon website as a registered user. WHEN user logs in with <username> and <password> THEN user should be able to view homepage. WHEN user searches for <productID> THEN user should be on the product information page. AND user tried to add the product to cart THEN product should be added to cart. WHEN user navigates to order confirmation page via express checkout option. THEN order should be placed successfully along with order confirmation id being generated. |
Example:
|
Username |
Password |
ProductId |
|
Username1 |
Password1 |
ProductId1 |
|
Username2 |
Password2 |
ProductId2 |
Step Definition file created would be consisting of a code which defines the steps for annotations created in feature file. An annotation followed by step defined in patterns with the help of regular expression would help in linking the step definition file to steps in feature file. Sample step definition code sample for above code is defined as above.
|
1 2 3 4 5 6 7 8 9 10 |
@Given("^user is logged onto Amazon website as a guest user $") public void user_is_on_Home_Page() throws Throwable { driver = new FirefoxDriver(); driver.manage().timeouts().implicitlyWait(5, TimeUnit.SECONDS); driver.get("http://www.amazon.in"); } @When("^user searches a product on homepage $") public void user_searches_a_product() throws Throwable { driver.findElement(By.xpath(".//*[@id='account']/a")).click(); } |
Test Runner file defined the location of step definition and primarily provides all the metadata information required for test execution. It makes use of ‘@RunWith()’ annotation from JUnit framework for execution. Secondarily you would be making use of ‘@CucumberOptions’ annotation to define the location of feature files, step definitions location through glue, what reporting integrations to use etc.
Below is the sample of feature file created.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
package amazonTest; import org.junit.runner.RunWith; import cucumber.api.CucumberOptions; import cucumber.api.junit.Cucumber; @RunWith(Cucumber.class) @CucumberOptions( features = "src/test/Feature" ,glue={"src/main/stepDefinition"} ) public class TestRunner { } |
We can integrate parallel test execution for automation testing with Selenium & Cucumber framework for saving a considerable amount of time during test execution. To elaborate the above scenario, serve the regression test needs of an application under test you have to execute your entire regression test automation repository which could serve you with both time and resource constraints. The end-user would not be using the same browser with its latest version, there could be plenty of browsers, operating systems and browser version combinations which should be considered while performing out testing.
However, the cons stated above could be resolve with cloud solutions such as LambdaTest which is a cross browser testing cloud. LambdaTest offers an online Selenium grid consisting 3000+ real browsers and operating systems help you achieve maximum amount of browser test coverage. Build time could be drastically reduced by utilizing LambdaTest automation for your test execution. Parallel test execution with the help of various browsers and multiple versions of each browser instance would help you achieve your cross browser parallel testing needs. Automated screenshots and video recording would help you keep track of your test failures and find the root cause quickly providing you a better analytical perspective while troubleshooting.
As we have embarked on our journey of implementing automation testing with Selenium & Cucumber framework using Java, till now we have learned the overview of the cucumber framework and how it leverages BDD. Let’s get into hands-on the mode by creating a sample framework using Cucumber for Selenium automation.
Creating Your First Project Using Cucumber For Selenium Automation Testing
We make use of Eclipse IDE for coding purposes, get the latest version from their official website https://www.eclipse.org/downloads/.
|
1 2 |
“cucumber-archetype Artifact Id Group Id as io.cucumber” |
Archetype would be generated with the help of above statement. If in case above archetype has been missing please add template for the Archetype by entering below info.
|
1 2 3 |
GroupId: “io.cucumber” ArtifactId: “cucumber-archetype” Version: “2.3.1.2” |
After entering above details try navigating further and finishing required steps. Hence maven project would be successfully created.
Replace the dependencies by obtaining them from below pom file as required with below set of dependencies.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>Google_Search</groupId> <artifactId>Google_Search</artifactId> <version>0.0.1-SNAPSHOT</version> <packaging>jar</packaging> <name>Google_Search</name> <url>http://maven.apache.org</url> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> </properties> <dependencies> <dependency> <groupId>org.seleniumhq.selenium</groupId> <artifactId>selenium-java</artifactId> <version>3.14.0</version> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> <scope>test</scope> </dependency> <dependency> <groupId>io.cucumber</groupId> <artifactId>cucumber-java</artifactId> <version>4.2.2</version> </dependency> <dependency> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> <version>27.0.1-jre</version> </dependency> <dependency> <groupId>io.cucumber</groupId> <artifactId>cucumber-junit</artifactId> <version>4.2.2</version> <scope>test</scope> </dependency> </dependencies> </project> |
Creating your Feature file with user defined specifications in Gherkin Language:
Step 1: Create a folder for organizing all your feature files as shown in the image below.
Step 2: Select File from the navigation menu. File → New → File.
Step 3: Name the file using “Login.feature”.
Step 4: As there is no plugin which supports “.feature” extension file created would look as in the image below.
Step 5: Once you create the feature file below alert would be prompted asking you to select the editor to be installed from the marketplace.
Step 6: Select “Show IDE extensions for this file type and let me install them” radio button and select the first option which is Cucumber Eclipse Plugin as shown in the image below.
Click on install. Accept the terms of the license agreement. Click on Finish. You would be prompted with a security alert, click on Install Anyway option, once installation is finished click on Restart Now to view the changes made.
Step 7: To view if changes are reflected observe the Feature file as in below image how different is it from the previous image of the feature file created.
Watch this video to learn how TestNG has become one of the most robust test automation frameworks and all you need to know to get started with TestNG in Selenium.
Running First Script Of Automated Testing With Selenium & Cucumber Framework
Let’s consider a real-world case study of querying a search term in google search by creating a feature file which consists of feature, tags and scenarios and scenario outline as applicable.
For automation testing with Selenium & Cucumber framework for Java, we need to create a GoogleSearch.feature file under Features package with below content under “src/test/java” path.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
Feature: Searching Google should return the name of query Scenario: Google search with scenario Given user launches Google webapp When user searches for a "LambdaTest" And click on search button Then results retrieved should contain the "LambdaTest" used Scenario Outline: Searching google using scenario outline and examples Given user launches Google webapp When user searches for a "<query>" And click on search button Then results retrieved should contain the "<query>" used |
Example:
|
Query |
Lambdatest |
Selenium |
Cucumber |
Once you have completed creating a feature file, created steps would be highlighted with warnings indicating that you haven’t defined the step contained in step definition file yet.
In order to resolve the errors, lets navigate to the step definition file to complete coding the gist which defines our steps.
Below code can be replaced in step definition file which is Stepdefs.
TIP:
While creating your step definitions if you want a template for step definitions file creation just run the feature file along with warnings as discussed previously. It would provide you with the template upon which you can start coding as shown in the image below.
Make sure your step definitions are specified in package “com.googlesearch.testing.steps” under the path of “src/main/java”.
Copy the code and define the path to your chrome driver. Below is the code used for automation testing with Selenium & Cucumber in Java.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
package com.googlesearch.testing.steps; import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.WebElement; import org.openqa.selenium.chrome.ChromeDriver; import cucumber.api.java.en.And; import cucumber.api.java.en.Given; import cucumber.api.java.en.Then; import cucumber.api.java.en.When; public class stepDefinitions { public static WebDriver driver; String baseURL = "https://www.google.com"; @Given("^user launches Google webapp$") public void user_launches_Google_webapp() throws Throwable { // Write code here that turns the phrase above into concrete actions System.setProperty("webdriver.chrome.driver", "C:\\Cucumber_Workspace\\GoogleSearch\\Libs\\chromedriver.exe"); driver = new ChromeDriver(); driver.manage().window().maximize(); driver.get(baseURL); } @When("^user searches for a \"([^\"]*)\"$") public void user_searches_for_a(String arg1) throws Throwable { WebElement searchBox = driver.findElement(By.name("q")); searchBox.sendKeys(arg1); } @And("^click on search button$") public void click_on_search_button() throws Throwable { WebElement searchBox = driver.findElement(By.name("q")); searchBox.submit(); } @Then("^results retreived should contain the \"([^\"]*)\" used$") public void results_retreived_should_contain_the_used(String resultString) throws Throwable { WebElement result = driver.findElement(By.className("LC20lb")); if (resultString.equalsIgnoreCase(result.getText())) { System.out.println("Text is matching"); } else { System.out.println("Text is not matching"); } driver.quit(); } } |
Make sure your Test Runner java file which is named as RunCucumberTest is under “src/test/java” path with code specified below.
|
1 2 3 4 5 6 7 8 9 10 11 |
import org.junit.runner.RunWith; import cucumber.api.CucumberOptions; import cucumber.api.junit.Cucumber; @RunWith(Cucumber.class) @CucumberOptions(features = { "classpath:features" }, plugin = { "pretty", "json:target/cucumber/cucumber.json" }) public class RunCucumberTest { } |
Now you have all the 3 main aspects ready for test execution. To proceed further for running automation testing with Selenium and Cucumber you would need to right-click on RunCucumberTest.java file which is created. Click on Run as JUnit Test. Tests get triggered for both the scenarios specified. If you observe the scenario outline it gets executed thrice with all the sets of data specified in a data driven format.
Interested about JUnit? Don’t forget to read our article on Automated Testing With JUnit And Selenium For Browser Compatibility.
Using LamdaTest For Automation Testing With Selenium & Cucumber
Above test would be running in sequence and nexttest would have to wait till the thread is empty by completion of previous test triggered. This would consume more amount of time and cross browser testing with above methodology would prove to be a troublesome and expensive due to the permutations and combinations of the browsers and operating systems and versions available. In order to overcome the issue we would have to run our automation tests with Selenium and Cucumber in parallel, even if we do so still we would have to configure a lot of machines as the number of tests you can trigger per system would depend on the computing capabilities of each system.
LambdaTest would help us overcome the above problem statements. As a cloud solution it offers scalability to reduces build time, improve browser test coverage, provision to validate responsive test from pixel to pixel accuracy, enhanced reporting and analytical capabilities. Let’s start with configuring Cucumber to enable parallel test execution of Selenium with Java using LambdaTest for automated cross browser testing
Watch this video to learn how to set up and use TestNG with Selenium to automate your testing process. We will also introduce the TestNG Priority method, which allows you to write easy-to-read and maintainable tests.
Configuring Your Test Suite With LambdaTest For Parallel Test Execution Of Selenium With Cucumber & TestNG
Replace your step definition file with below code which includes Remote WebDriver configuration to connect to cloud virtual machines. Currently we have JUnit as our unit testing framework, for provisioning parallel test execution let’s integrate current framework with TestNG. TestNG provides an easier configuration and additional capabilities which makes it more powerful than that of JUnit.
We start with creating a TestNG xml file to run the test suite. Add below xml code to your project root folder. If you are not familiar with TestNG then you can have a look at our guide for running first TestNG automation script.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd"> <suite name="GoogleSearch" parallel="tests" thread-count="3"> <test name="WIN10TEST_FF"> <parameter name="browser" value="Firefox" /> <parameter name="version" value="64.0" /> <classes> <class name="com.cucumber.tests.TestNGRunCucumberTest" /> </classes> </test> <test name="WIN10TEST_CHROME"> <parameter name="browser" value="Chrome" /> <parameter name="version" value="71.0" /> <classes> <class name="com.cucumber.tests.TestNGRunCucumberTest" /> </classes> </test> </suite> |
Replace your step definition file with below code. @Parameters will help us to parameterize the desired capabilities such as browsername, version, platform etc. Have a look at LambdaTest Capabilities Generator for fetching the coded values according to your desired capabilities. If you take a glance at above xml file we have two parameters which are browser and version. Thread count in the xml code represents the number of concurrent session which are possible. We have set it to 2 for demo purposes. Kindly replace the stepDefinition code with below TestNG model.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
package com.cucumber.stepdefs; import org.openqa.selenium.By; import org.openqa.selenium.WebElement; import com.cucumber.tests.TestNGRunCucumberTest; import cucumber.api.java.en.And; import cucumber.api.java.en.Given; import cucumber.api.java.en.Then; import cucumber.api.java.en.When; public class stepDefinitions extends TestNGRunCucumberTest { @Given("^user launches Google webapp$") public void user_launches_Google_webapp() throws Throwable { // Write code here that turns the phrase above into concrete actions driver.get(baseURL); } @When("^user searches for a \"([^\"]*)\"$") public void user_searches_for_a(String arg1) throws Throwable { WebElement searchBox = driver.findElement(By.name("q")); searchBox.sendKeys(arg1); } @And("^click on search button$") public void click_on_search_button() throws Throwable { WebElement searchBox = driver.findElement(By.name("q")); searchBox.submit(); } @Then("^results retreived should contain the \"([^\"]*)\" used$") public void results_retreived_should_contain_the_used(String resultString) throws Throwable { WebElement result = driver.findElement(By.xpath("//*[@id='rhscol'][1]")); if (result.isDisplayed()) { System.out.println("Resutls are retreived"); } else { System.out.println("Resutls are not retreived"); } } } |
As we have trigger the tests from xml file of TestNG we can replace the TestRunner file with below code, as our entry point would be TestNGRunner xml.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 |
package com.cucumber.stepdefs; import org.openqa.selenium.By; import org.openqa.selenium.WebElement; import com.cucumber.tests.TestNGRunCucumberTest; import cucumber.api.java.en.And; import cucumber.api.java.en.Given; import cucumber.api.java.en.Then; import cucumber.api.java.en.When; public class stepDefinitions extends TestNGRunCucumberTest { @Given("^user launches Google webapp$") public void user_launches_Google_webapp() throws Throwable { // Write code here that turns the phrase above into concrete actions driver.get(baseURL); } @When("^user searches for a \"([^\"]*)\"$") public void user_searches_for_a(String arg1) throws Throwable { WebElement searchBox = driver.findElement(By.name("q")); searchBox.sendKeys(arg1); } @And("^click on search button$") public void click_on_search_button() throws Throwable { WebElement searchBox = driver.findElement(By.name("q")); searchBox.submit(); } @Then("^results retreived should contain the \"([^\"]*)\" used$") public void results_retreived_should_contain_the_used(String resultString) throws Throwable { WebElement result = driver.findElement(By.xpath("//*[@id='rhscol'][1]")); if (result.isDisplayed()) { System.out.println("Resutls are retreived"); } else { System.out.println("Resutls are not retreived"); } } } As we have trigger the tests from xml file of TestNG we can replace the TestRunner file with below code, as our entry point would be TestNGRunner xml. package com.cucumber.tests; import java.net.MalformedURLException; import java.net.URL; import org.openqa.selenium.remote.DesiredCapabilities; import org.openqa.selenium.remote.RemoteWebDriver; import org.testng.annotations.AfterClass; import org.testng.annotations.BeforeClass; import org.testng.annotations.DataProvider; import org.testng.annotations.Parameters; import org.testng.annotations.Test; import cucumber.api.CucumberOptions; import cucumber.api.testng.CucumberFeatureWrapper; import cucumber.api.testng.PickleEventWrapper; import cucumber.api.testng.TestNGCucumberRunner; @CucumberOptions(features = { "src/test/java/features" }, glue = { "com.cucumber.stepdefs" }, plugin = { "pretty", "json:target/cucumber/cucumber.json" }) public class TestNGRunCucumberTest { DesiredCapabilities capabilities = new DesiredCapabilities(); String username = " bharadwajpendyala"; String accesskey = " qYlLn1IzVrC2U41zM4kyjv35EvpHxR2tyMB4aEBlkNMmvpnQ5A"; protected static RemoteWebDriver driver = null; String gridURL = "@hub.lambdatest.com/wd/hub"; boolean status = false; protected String baseURL = "https://www.google.com"; private TestNGCucumberRunner testNGCucumberRunner; @BeforeClass @Parameters(value = { "browser", "version" }) public void beforeClass(String browser, float version) { capabilities.setCapability("build", "Cucumber_Project"); capabilities.setCapability("name", "Querying in google engine"); capabilities.setCapability("platform", "Windows 10"); capabilities.setCapability("browserName", browser); capabilities.setCapability("version", version); try { driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), capabilities); } catch (MalformedURLException e) { System.out.println("Invalid grid URL"); } catch (Exception e) { System.out.println(e.getMessage()); } testNGCucumberRunner = new TestNGCucumberRunner(this.getClass()); } @Test(groups = "cucumber scenarios", description = "Runs Cucumber Scenarios", dataProvider = "scenarios") public void scenario(PickleEventWrapper pickleEvent, CucumberFeatureWrapper cucumberFeature) throws Throwable { testNGCucumberRunner.runScenario(pickleEvent.getPickleEvent()); } /** * @return returns two dimensional array of {@link CucumberFeatureWrapper} * objects. */ @DataProvider public Object[][] scenarios() { return testNGCucumberRunner.provideScenarios(); } @AfterClass public void tearDown() { driver.quit(); testNGCucumberRunner.finish(); } } |
After executing the above code for performing automation testing with Selenium, Cucumber, & TestNG. It is important to observe the details of your Selenium automation tests. LambdaTest provides every inch of information, from analytics to video logs for you to observe the output of your automation testing with Selenium.
Automation Dashboard:
Navigate to automation dashboard of lambdatest engine to view number of concurrent sessions being active and browser info as highlighted in the above screenshot.
It provides the logs, metadata information, screenshots and video recording for the tests executed. Click on test to view the details specified above as in below snapshot.
Analytics View:
Analytical view of tests execution information could be viewed by click on analytics tab. This would provide a comprehensive view of builds passed, failed, day of execution, time taken, bugs logged from the testing etc. You can take a glance at it in the below image.
Timeline View:
Timeline would provide the information of test suite executed as a whole. Time and date of execution browser information, failures and error information if any. You can view tests executed in the form of build or test view.
To summarize we have explained what is cucumber, how does it help overcome challenges of traditional test methodologies. We have ran our first test script automation testing with Selenium and Cucumber framework after understanding what cross browser testing is all about. Also, how cross browser testing could be simplified using parallel cloud testing solution like LambdaTest. We have created a cucumber framework and configured it with LambdaTest integrating TestNG for parallel testing. Happy Testing! 🙂
Author’s Profile
Bharadwaj Pendyala
Bharadwaj has more than 5+ year of experience in handling Automation Testing who is also an expert in handling Manual Testing, API Automation, Performance Testing and Certified Agile Practitioner who is currently handling projects in TDD and BDD methodologies with both custom built and hybrid automation frameworks
Blogs: 4
Got Questions? Drop them on LambdaTest Community. Visit now
