Test Automation Using Pytest and Selenium WebDriver

In today’s times, one of the challenges that developers face is ensuring that their website/web-app works seamlessly across a different set of devices, browsers, and Operating Systems (OS)/platforms. This is where cross browser testing plays a very crucial role in testing the website/web-app since it helps in testing across different combinations. Based on the target market, development & product team need to chart out a plan about the various activities involved in browser compatibility testing.

Selenium – Introduction & WebDriver Interface

As far as testing a web application is concerned, there a couple of web frameworks are available that automate the tests performed across different web browsers. Selenium is a very popular framework that is primarily used for automation testing of web applications. It is an open-source tool that can be used to test on Chrome, test on firefox browser, test on Opera browsers, test on Edge browsers, etc. The framework can also be used to perform testing on Internet Explorer (latest version or the legacy versions).

Selenium WebDriver is considered as one of the core components of the Selenium framework. Selenium WebDriver API is a collection of open-source APIs/language specific bindings that accepts commands and sends them to the browser (against which the testing is performed). The individual who is responsible for developing the tests need not bother about the ‘architecture details or other technical specifications’ of the web-browser since WebDriver acts as an interface between the test –suite/test-case & web-browser (achieved using browser-specific WebDriver). Selenium WebDriver supports different programming languages like Python, C#, Ruby, PERL, Java, etc. The diagram below shows the simplified view of the Selenium WebDriver Interface. We have already covered Selenium WebDriver Architecture in depth in our earlier post.

Pytest test Framework – Introduction and Advantages

Python language has couple of test frameworks that ease the task of web application testing, unittest & pytest are the most widely used frameworks. unittest is a part of the standard library (in Python) and comes as a part of the Python installation.

In case you are new to pytest framework checkout this video to know more about pytest framework & its installation.

For test automation using pytest with Selenium WebDriver, you need to install pytest separately, it is more popular than unittest framework, below are some of the advantages of the Pytest framework

• Can be used by development teams, test teams, teams that are practising Test Driven Development(TDD), as well as in open-source projects.
• Can be used to simple, as well as complex functional test cases for applications & libraries.
• Easy to port existing test suites to pytest for performing test automation using pytest with Selenium WebDriver.
• Compatibility with other test frameworks like unittest & nose, hence switching to this framework is very easy.
• Supports Parameterizing, which is instrumental in executing the same tests with ‘different configurations’ using a simple marker. Hence, you can come up with more effective test cases/test suites with ‘less repetitive code implementation’.
• Number of Asserts that provide more detailed information about the failure scenarios.
• Support of Fixtures & Classes. Using Fixtures, it becomes easy to make common test objects available throughout a module/session/function/class. Fixtures & Classes would be covered in more detail in subsequent sections.
• Good & upto date documentation.
• xdist support through which test cases can be parallelized.

Now that you are aware of the advantages of pytest over other test frameworks, let’s have a detailed look at the pytest framework and how it can be used with Selenium WebDriver Tutorial framework in order to perform automated cross browser testing for web applications.

To summarize, Pytest is software test framework which can be used to make simple, yet scalable test cases with ease.

Test Automation Using Pytest – Installation and Getting started

As mentioned earlier, pytest is not a part of the standard Python installation and needs to be installed separately. In order to install pytest, you should execute the following command on the prompt/terminal:

pip install –U pytest
Once the installation is complete, you can verify whether the installation is successful, by typing the following command:
pytest --version

Below is the output when the above command is executed on Linux and Windows machine

PyCharm is a popular IDE that is used for pytest development. You can install PyCharm Edu version for Windows, Linux, or macOS from here. For development, we are using PyCharm for Windows. Once PyCharm is installed, you should make sure that the default test runner is pytest. In order to change the default test runner, you should navigate to File->Settings->Tools->Python Integrated Tools and change Default test runner for performing test automation using pytest with Selenium WebDriver.

Now that PyCharm Edu version is installed and default test runner is set to pytest, you need to install the Selenium package for Python to perform test automation using pytest with Selenium WebDriver. In order to install Selenium, you should invoke the command mentioned below in the Terminal of PyCharm.

pip install -U selenium ( Syntax – pip install –U )

Shown below is the snapshot of the command execution

Now that your development environment is all set i.e. PyCharm to execute test automation using pytest with Selenium installed, we look into some of the features & aspects of pytest.

You can watch this video to learn how to write and run your first test in pytest using Selenium.

Pytest – Usage, exit codes, and compilation

pytest and py.test can be used interchangeably. In order to get information about the arguments that can be used with pytest, you can execute the command mentioned below on the terminal

When pytest code is executed, it results in one of the following exit codes

Exit Code	Description
0	Test cases/test suites are executed successfully and end result was PASS
1	Test cases/test suites were executed, but some tests FAILED
2	Test execution was stopped by the user
3	Unknown error occurred when the tests were executed
4	Usage of pytest command is incorrect
5	No tests were collected

It is important that the file containing pytest code should be named as test_*.py or *_test.py. In order to compile & execute pytest source code for performing test automation using pytest with Selenium WebDriver, you can use the following command on the terminal

pytest <test_file_name.py> --verbose --capture=no

Let’s have a look at some examples of test automation using pytest. We start with a very simple example – test_pytest_example_1.py (Self-explanatory)

In the above code snippet, we create a function named function_1 which takes one argument named var. There are two test cases – test_success() and test_failure(). The test cases are executed in serial order and assert is issued on an execution of the test cases. Compile the code using the command mentioned below

pytest --verbose --capture=no test_pytest_example_1.py

As seen in the output, the result of the first test case is PASS (shown in BLUE) and a result of the second test case is FAIL (shown in RED).

pytest makes use of assert available in Python for verification of results, it gives out meaningful information which can be used for verification & debugging. pytest.raises is commonly used to raise exceptions, below is an example where Factorial of a number is calculated. In one test case, negative number is passed as an input to the factorial function and AssertionError is raised.

In case you want to know more about assertions in pytest you can watch the video below.

factorial_example.py – Contains the implementation that uses recursion in order to calculate factorial of the input number. Before the factorial is calculated, the input parameter check is performed. Assert would be raised in case the input number is negative.

test_factorial_example.py – pytest implementation which use factorial functionality. Three test cases are implemented – test_standard_library (output from factorial_function is compared with output obtained from math.factorial module), test_negative_number (assertion is raised when the input number is negative), and (results of output from factorial_function are compared with specific values).

You can execute the code using the command py.test –capture=no test_factorial_example.py either on the command prompt or on the Terminal of PyCharm IDE. As seen in the snapshot, all the test cases have passed and logs under ‘print statement’ are outputted on the console

Test Automation Using Pytest – Fixtures (Usage and Implementation)

Consider an example where you have to execute certain MySQL queries on a database that contains employee information within an organization. The time taken to execute a query would depend on the number of records (i.e. employees) in the database. Before queries are executed, required operations (w.r.t database connectivity) have to be performed and the ‘returned handle’ would be used in a subsequent implementation involving the database. Database operations can be CPU intensive (as the number of records increase) hence, repetitive implementation & execution should be avoided. There are two ways in which this issue can be solved

1. With the help of classic xunit style setup along with teardown methods.
2. By using fixtures (recommended).

xunit style of fixtures is already supported in unittest but pytest has a much better way of dealing with fixtures. Fixtures are a set of resources that have to set up before the test starts and have to be cleaned up after the execution of tests is complete. It contains a lot of improvements over the classic implementation of setup & teardown functions. The main advantages of using fixtures are

• A developer can set the lifetime & scope of the fixture. The scope of the implemented fixture could be modules, functions, classes, or the entire project.
• Fixtures are implemented in a modular manner; hence there is no learning curve involved.
• Function-scoped fixtures bring the necessary readability & consistency in your test code. This makes the maintainability easy & lesser daunting task.
• Fixture functions leverage the Object-oriented programming design concept termed ‘Dependency Injection’ where fixture functions take up the role of the injector & the test functions are considered as consumers of the fixture objects.
• Each fixture has a name (similar to a function name), which in turn can call other fixture functions.
• Fixtures can be reused and it can be used for simple unit testing to testing complex use cases.

Ever since the launch of version 3.5, the fixtures of higher scope are prioritized above the lower scope fixtures in terms of instantiating. Higher scope fixture includes sessions, lower scope fixture would include classes, functions etc. You can even ‘‘parameterize’ these fixture functions in order to execute them multiple times along with the execution of dependent tests. Fixture parameterization has been widely used to write exhaustive test functions. Below is a simple code for test automation using pytest where setup() and teardown() of ‘resource 1’ is called, even when the test_2 is executed. Since this is a simple implementation (with fewer computations), there are not many overheads even when ‘unnecessary setup & module calls are invoked’, but it could hamper the overall code performance in case any ‘CPU intensive’ operations (like database connectivity) are involved

Execute the test case ‘test_2_not_using_resource_1’ by invoking the following command on the terminal

pytest --capture=no --verbose test_fixtures.py::test_2_not_using_resource_1

As observed from the output [Filename – Pytest-Fixtures-problem.png], even though ‘test_2’ is executed, the fixture functions for ‘resource 1’ are unnecessarily invoked. This problem can be fixed by using fixtures; we would have a look at the same in the upcoming example.

As seen in the example below, we define a fixture function resource_1_setup() (similar to setup in xunit style implementation) and resource_1_teardown() (similar to teardown in xunit style implementation). The fixture function has ‘module scope’ using @pytest.fixture(scope=’module’).

We execute the code by triggering all the testcases i.e. test_1_using_resource_1() & test_2_not_using_resource_1(). As shown in the output below [Filename – Pytest-Fixtures-all-tests-executed.png], ‘setup for resource 1’ is called only for Test 1 and not for Test 2.

Now, we execute only testcase 2 i.e. test_2_not_using_resource_1(). As seen in the output below [Filename – Pytest-Fixtures-only-2-tests-executed.png], setup & teardown functions for ‘Resource 1’ are not called since the only testcase – 2 is executed. This is where fixtures can be highly effective since it eliminates ‘repetitive code’ and ‘execution of unnecessary code’. Official documentation about fixtures in pytest can be found here.

This certification is for professionals looking to develop advanced, hands-on expertise in Selenium automation testing with Python and take their career to the next level.

Here’s a short glimpse of the Selenium Python 101 certification from LambdaTest:

When you are looking out for a Python testing framework, you would probably require a test framework that would suffice all your requirements i.e. unit test, functional test, acceptance tests, etc. The framework should have the ability to log events, generate test reports, and should have good community support. Pytest fulfils all these requirements and test automation using pytest with Selenium WebDriver is highly recommended as it does not involve a steep learning curve.

When you are planning to develop test automation using pytest with Selenium WebDriver, the first concern that you need to look into is when should load the browser. Loading a new browser instance after each test is not recommended since it is not a scalable solution and might increase the overall test execution time. It is recommended to load the browser (under test) before the actual test cases have started & unloaded/closed the browser instance as soon as the tests are complete. This is possible by using Fixtures [discussed below in Automation Testing Using Pytest – Fixtures (Usage and Implementation) section] in pytest. As mentioned earlier, Fixtures make extensive use of a concept termed as ‘Dependency Injection’ where dependencies can be loaded before the actual tests have started.

By default fixtures have ‘function’ scope, depending on the requirements; you can change the implemented fixture’s scope to a module, session, or class. Like ‘lifetime of variables in C language’, the scope of fixtures indicates how many times the particular fixture will be created.

Fixture Scope	Explanation
Function	Fixture is executed/run once per test session
Session	One fixture is created for the entire test session
Class	Only one fixture is created per class of tests
Module	Fixture is created once per module

Once the tests have been executed, you might be interested to capture the test results in a report format (like HTML). You need to install pytest-html module for the same

pip install pytest-html

Below is the snapshot of the command in execution

In this pytest Tutorial, learn how to use pytest fixtures with Selenium and how to set up your test using the pytest.fixture() decorator.

Now that you have knowledge about pytest fixtures, Selenium, and Selenium WebDriver interface; let’s have a look at an example with all these things in action. Before you start the implementation, please ensure that you download Gecko driver for Firefox & ChromeDriver for Chrome from here & here respectively. In order to avoid mentioning the ‘path/location’ where the drivers have been downloaded, make sure that you place these respective drivers at the location where the corresponding browsers are present. In the snapshot below, you can see that we have copied Geckodriver.exe in the location where Firefox browser (firefox.exe) is present

Now that you have the setup ready, let’s get started with the implementation. Import all the necessary modules in the beginning so that you avoid errors. In our case, the modules imported are selenium, pytest, pytest-html. Two fixture functions – driver_init() and chrome_driver_init() have the ‘class’ scope. As seen in the fixture function driver_init(), an instance of Firefox is created using GeckoDriver, whereas in chrome_driver_init(), an instance of Chrome browser is created using ChromeDriver. yield contains the implementation of teardown i.e. code inside yield is responsible for doing the cleanup activity. A class is used to group test cases, in this case, there are two important classes namely Test_URL() and Test_URL_Chrome(). In case you’re wondering how to group tests in pytest, here’s a video you can refer to:

The implemented classes are making use of the fixtures that were implemented using mark.usefixtures [@pytest.mark.usefixtures(“driver_init”)]. The test case performs a simple test of invoking the respective browser (Firefox/Chrome) and opening the supplied URL i.e. https://www.lambdatest.com/ Filename – test_selenium_webdriver-2.py

Since we require the test output in an HTML file, we make us of –html= argument while executing the test code. The complete command to execute test automation using pytest with Selenium WebDriver:

Below is the execution output, testcase test_open_url() is executed for the class Test_URL & Test_URL_Chrome(). The test report is pytest_selenium_test_report.html [Image – PyTest-Selenium-Output-1.png]. Here is a test report for further clarity.

As seen in the above implementation, the only difference between fixture function for Firefox & Chrome browser is the ‘setting up of the respective browser’. Majority of the implementation is same for both the browsers [have a look at Test_URL(), Test_URL_Chrome()], hence it becomes important to optimize the code by avoiding ‘repetition of code’. This is possible by making use of ‘parameterized fixtures’. As seen in the implementation [Filename – test_selenium_webdriver-1.py], the major change is addition of ‘parameters’ to fixtures i.e. @pytest.fixture(params=[“chrome”, “firefox”],scope=”class”). Depending on the browser in use, the corresponding WebDriver is used to invoke the browser.

In our case, we are using the Chrome & Firefox browsers and the test case Test_URL() would be executed for each browser separately. As seen in the output, the test case is invoked once with parameters as ‘firefox’ and ‘chrome’.

Cross-browser testing with pytest, Selenium & Lambdatest

There is always a limitation on the amount of testing that you can perform on your local machine or test machines since thorough testing has to be performed on different kinds of devices (mobiles, tablets, desktops, etc.), Operating Systems (Windows, Linux, macOS, Chrome, etc.) and browsers such as Chrome, Firefox, Safari, Opera, and so on. Setting up a ‘local test environment/test farm’ is not a scalable & economical option. This is where your test team can utilize the power of LambdaTest online browser farm on the cloud capabilities.

You can perform manual as well as automated cross browser testing of your web application/website on different browsers (even old versions), devices, etc. You can also perform real-time testing by using their Tunnel feature which lets you use their test infrastructure from the terminal. LambdaTest Selenium Automation Grid enables you to perform end-to-end automation tests on a secure, reliable, and scalable selenium infrastructure. You can utilize the LambdaTest Selenium Grid to not only increase the overall code-coverage (via testing), but to also decrease the overall time required to execute your automation scripts written in Python.

Conclusion

Test automation using Pytest with Selenium WebDriver is a very favourable option as a framework that has good features with which test engineers can come up with implementation that is easy to implement & which is scalable. It can be used for writing test cases for simple scenarios, as well as highly complex scenarios. A developer/tester who is well versed with the Python, unittest/other test frameworks based on Python would find pytest easy to learn. Since pytest leverages concepts like ‘Dependency Injection’, there is less cost involved in ‘maintainability of the source code’.
If you are looking for testing web-applications or websites, you can execute test automation using pytest with Selenium framework. Since the number of devices are increasing with each passing day, it becomes highly impossible to test your code against different devices, operating systems, and browsers; this is where testers/developers can utilize Lambdatest’s ‘cross-browser testing tool’ which allow you to perform test automation using pytest with Selenium WebDriver, effortlessly.