How To Run Cypress Tests In Azure DevOps Pipeline

When software developers took years to create and introduce new products to the market is long gone. Users (or consumers) today are more eager to use their favorite applications with the latest bells and whistles. However, users today don’t have the patience to work around bugs, errors, and design flaws. People have less self-control, and if your product or application doesn’t make life easier for users, they’ll leave for a better solution.

Before the advent of DevOps, software companies used the conventional Waterfall model, which was effective for many projects like CRM and inventory management. However, it had several shortcomings. DevOps completely overpowered the traditional waterfall model.

Development teams in product organizations are constantly releasing updates, patches, bug fixes, and new features. Before rolling out the updates to all end users, operations teams have their hands full to ensure the new code functions properly. Users dislike downtime and additional errors, which can occur if the rollout process isn’t managed effectively.

In DevOps, code is reviewed and deployed much more quickly due to the collaboration between the development and operations teams. As a result, suggestions for new features and bug fixes based on user data get continuously incorporated into the development cycle. The journey each new piece of code takes, from planning to deployment to monitoring, is known as the DevOps pipeline. Every software development company needs a strong DevOps pipeline to meet customer demands and specifications.

Azure, AWS, GCP, Oracle Cloud Services, and many other cloud service providers have integrations with several open-source and enterprise automated testing tools. Cypress is the most commonly used framework that has integrations with all major cloud platforms. There are also testing platforms like LambdaTest, a cloud SaaS platform for Cypress testing that we can integrate with Azure DevOps.

Cypress Trends on GitHub

As per the Cypress GitHub repository, Cypress continues to gain popularity with:

• Stars: 39.5k
• Forks: 2.5k
• Used By: 599k
• Releases: 282
• Contributors: 400

This Cypress tutorial on how to run Cypress tests in Azure DevOps aims at getting people started with running automation testing on Microsoft’s Azure. We will deep dive into the essentials of Azure DevOps along with running Cypress automated tests with Azure Pipeline.

To make this tutorial more interesting, I have broken it down into different parts. First, I will explain DevOps, the Components/Stages of DevOps then how we Can Set Up Cypress and run Cypress tests in Azure DevOps pipeline. Finally, integrate Azure pipelines with LambdaTest and run Cypress test cases.

What is DevOps Pipeline?

DevOps is a software development methodology that connects the Development and Operations sides of an application. It bridges the gap between developers and maintainers by introducing collective responsibility practices. Traditionally, there was a barrier between these teams, and software packages were metaphorically pushed past the wall. But with DevOps processes, that barrier no longer exists.

A pipeline in software development consists of a set of tools, flows, and automated processes that enable the team to leverage technologies to build and deploy the application/software.

The process of building, testing, and deploying code is known as continuous integration & continuous deployment, or CI/CD. Automating this process can minimize human error and maintain a consistent process for how software is released.

You can develop, test, and continuously deploy new code by creating an effective DevOps pipeline. Building an efficient DevOps pipeline enables organizations to develop, test, and continuously deploy new code rapidly.

One of the main goals of a DevOps pipeline is to automate the software delivery process, thus eliminating the need for manual changes at every step of the pipeline.

Manual work is time-consuming and risks human error, which often delays deployments. Switching from manual to automated changes results in fewer errors and allows developers to release higher-quality code faster than ever.

LambdaTest lets you integrate and run Cypress tests in Azure DevOps Pipelines using an extension that connects the CI server to LambdaTest’s cloud Cypress Grid.

Components of a DevOps Pipeline

In order to run Cypress tests in Azure DevOps Pipeline, let’s discuss the components of a DevOps pipeline.

There are several different approaches and tools that organizations normally use to create a customized DevOps pipeline. Common pipeline components enable continuous delivery by ensuring that code flows smoothly from one stage to the next, automating the entire process and reducing manual work.

A well-constructed pipeline comprises tools that automate processes and permit continuous iteration. Which are:

Continuous Integration

Continuous Integration (CI) means multiple integrations every day. In the past, when the team didn’t have the continuous integration concept, we had to wait for a long time to reflect the actual code into production. Since code is integrated very late, the chance of code conflict is increased due to this delay.

Contrary to the previous methodology, CI encourages developers to submit their code regularly. As a result, they can identify bugs more quickly and spend less time fixing them.

Continuous Integration (CI) is a development practice requiring developers and quality assurance personnel to commit code changes to a shared repository (e.g., GIT repository) several times a day. When we push the code, the CI tool pulls the code from the shared repository and makes the build.

The CI tool builds the code in addition to performing other background tasks like code review, integration testing, and running unit test cases. CI has some best practices in modern DevOps – which detects and notifies developers what has happened. GitHub users can implement CI through GitHub Actions. Azure DevOps users can use Azure Pipelines.

Continuous Delivery

Continuous Delivery (CD) is nothing more than the evolution /or next step of CI. It is about speeding up the release process by encouraging developers to release code for user acceptance testing in the test environment. Once the build is successful, the code is deployed to the test server for user acceptance or end-user testing.

Continuous Delivery (CD) has great advantages. Let’s look at some of the benefits offered by CD. Before CD, developers used to take much longer to locate the issue if there was an error in the build. With CD in place, it became easy to track which particular commit is causing the failure.

The more frequently you can deploy code updates, the better off you’ll be able to use continuous deployment. The workflow determines the release frequency, typically daily, weekly, or monthly. Compared to releasing all changes at once, releasing code in smaller chunks makes it simpler to fix bugs. By doing this, bottlenecks and merge conflicts are prevented, and the integration pipeline continues to flow steadily and continuously.

Continuous Deployment

Continuous Deployment is the next step in automating the software development lifecycle after continuous delivery. Continuous Delivery and Continuous Deployment are similar in many ways, yet there are subtle differences. In continuous delivery, code is periodically committed to the staging branch for user acceptance testing. While continuous deployment allows development teams to deploy software, features, and code updates manually, continuous deployment is about automating the entire release cycle.

In the continuous deployment phase, code updates are automatically released to the targeted users without manual intervention. If something goes wrong, you can roll back the changes without causing the application to stop working. To fully realize the potential of continuous deployment, robust automation testing frameworks like Selenium, Cypress, etc., must be in place to ensure that the new code is truly bug-free and can be deployed to production without delays.

Continuous Testing

Continuous testing is another part of the DevOps pipeline. Using continuous testing, you can run automated tests for the code integrations gathered during the continuous integration phase. To find problems before they affect the production environment, continuous testing involves running tests as frequently as possible at each stage of the development process. Rapid evaluation of the business risks associated with particular release candidates in the delivery pipeline is made possible by implementing a continuous testing strategy.

Functional and non-functional testing should both be included in the testing scope. This entails carrying out the unit, system, and integration tests as well as tests that focus on an application’s security and performance along with the server infrastructure.

Continuous testing encompasses a broader understanding of quality control, including risk assessment and internal policy compliance. Some types of testing that can be automated include end-to-end testing, regression testing, functional tests, integration tests, and performance tests that run automatically as soon as the code is integrated.

Every DevOps cloud platform has some stages. Below are some stages that DevOps pipelines normally have.

Stages of a DevOps Pipeline

There are no strict guidelines for how you should structure the pipeline. DevOps teams add or remove certain phases depending on their specific workflows. Almost all pipelines consist of four core phases: Development, Build, Test, and Deploy.

You can extend the setup by adding two more stages – plan and monitor – as they are also ubiquitous in professional DevOps environments.

Plan

Planning your project, technology, environment, structure, and architecture creates a roadmap for successfully achieving your project goals. In the planning phase, the entire workflow is planned before developers start coding. Product managers and project managers are crucial during this stage. It is their responsibility to develop a development roadmap that will serve as a guide for the entire team.

The work is further broken down into a list of tasks after gathering feedback and relevant information from stakeholders and users. Teams can produce results more quickly, address issues immediately, and adjust to unforeseen changes more readily by breaking the project into smaller, more manageable pieces. In a DevOps environment, teams work in sprints – a shorter time (usually two weeks) during which individual team members work on their assigned tasks.

Develop

Development phase is when developers start writing code. Developers install the appropriate IDEs, code editors, and other technologies on their local computers depending on the programming language to achieve maximum productivity.

Developers submit a pull request to the shared source repository when they are prepared to submit their code. Once the initial pull request has been approved, team members can manually review the recently submitted code and merge it with the branch.

Build

A compilation process is used to extract the source code from the repository, link it to libraries, modules, and dependencies, and create an executable. A DevOps pipeline’s build phase is crucial because it enables developers to find code errors before they get into the pipeline, resulting in a major catastrophe.

Newly written code is merged with the shared repository, and developers run a series of automated tests. The pull request typically starts an automated process that builds the code into an executable or deployable package. Some programming languages do not need to be compiled.

For example, applications written in Java and C must be compiled to run, while PHP and Python do not. Build fails if there are issues with the code and the developer is informed of the issues. To ensure that only bug-free code gets into the pipeline, developers repeat this procedure each time they push code to the shared repository.

Test

The testing stage begins after a successful build. Once ready, the build is then deployed to the test environment where you can perform several types of testing like user acceptance test, security test, integration testing, performance testing, end-to-end testing, and cross browser testing.

Deploy

When the build reaches the deployment stage, the software is ready to be moved to production. When only minor changes to the code are required, an automated deployment method is used. To see how the newly added code functions, the build is first deployed to a production-like environment if the application has been significantly reworked.

Blue/green deployment is a common method of deployment for continuous delivery, where two environments are configured identically — one environment serves end users, while the other is ready for new code to deploy and undergo testing, such as performance testing, end to end testing, and cross browser testing

Monitor

Operations teams put in a lot of effort to monitor infrastructure, systems continuously, and applications during this last stage of the DevOps pipeline to make sure everything is operating as it should. To find performance problems, they gather important data from logs, analytics, monitoring systems, and user feedback.

The DevOps pipeline is more effective using the feedback gathered during the Monitor phase. After each release cycle, optimizing the pipeline is a good idea to eliminate any potential bottlenecks or other problems that might reduce productivity.

Now we know what scenario we will cover in this blog on how to run Cypress tests in Azure DevOps. But before running our scenario, we need to set up and create Cypress code using the below-mentioned steps.

Demonstration: Setup Cypress tests in Azure DevOps pipeline

To explain how we can set up the Azure pipeline for Cypress test cases, I will be testing a typical e-commerce site https://ecommerce-playground.lambdatest.io/.

Project Setup

You can follow the steps mentioned below to create a new Cypress project:

Step 1: Create a folder and generate package.json.

• Create a project by the name Cypress_Lambdatest.
• Create a package.json file using the npm init command in the terminal.

Step 2: Install Cypress.

Run this command in the newly created folder to install Cypress.

npm install cypress —save-dev

OR

yarn add cypress --dev

​​This will install Cypress locally as a dev dependency for your project.

Cypress version 9.2.0 is reflected once this is installed, as seen below. The most recent version of Cypress is 10.11.0

Shown below is the default folder structure of Cypress. You can create test cases under the folder “integration”

Under the integration folder named “cypress_lambdatest,” let’s create a new folder.

Under the folder lambdatest, create the first spec with the name login_searchproduct.spec.js. Second spec with the name ScrollAnd_ClickProduct.spec.js.

Implementation (Test Scenario – 1)

To demonstrate the usage of Cypress, we will first demonstrate the following test scenario.

Test Scenario:

1. Open the URL https://ecommerce-playground.lambdatest.io/index.php?route=account/login.
2. Enter your email address.
3. Enter the password.
4. Click on the Login button.
5. Search for the product “VAIO.”
6. Verify the correct product is searched.
7. Logs out from the application.

Create a login_searchproduct.spec.js file with the script below. This covers login into the application and searching for the product. After the search, verify the correct product should be displayed.

Code Walkthrough:

In the above code snippet, we have opened the target URL using cy.visit(), and Second, it() block is used to log in to the application by entering email and password.

In the third, it() block is used for searching for the product, and the last it() block is used for searching for the product.

In the below screenshot, we can see how we can inspect the element. We have located element by id for the email field

Implementation (Test Scenario – 2)

Test Scenario:

1. Open the URL https://ecommerce-playground.lambdatest.io/index.php?route=account/login.
2. Enter your email address.
3. Enter the password.
4. Click on the Login button.
5. Now, Scroll to the bottom of the application.
6. Click on a particular product.

Code Walkthrough:

In the above code snippet, first, we open the e-commerce site URL using cy.visit(). Second, it() block is used for login into the application by entering your email and password.

In the third, it() block, you will be redirected to the home page after clicking on LambdaTest Logo.

And In the Final it() block, you will scroll to the bottom of the page and click on the particular product.

The Cypress chain starts with cy.[command], where what is yielded by the command establishes what other commands can be called next (chained). cy.get() or cy.contains() methods yield a DOM element. This allows further commands to be chained onto them (assuming they expect a DOM subject), like .click() or even cy.contains() again.

In the below screenshot, we can see how we can inspect the element. We have located elements by type for the Login button.

Looking to try out Cypress? Check out the Cypress on Azure Marketplace

How to run Cypress tests in Azure DevOps pipeline?

Traditionally in the software testing life cycle, once QA creates the test cases, the QA has to wait for the build to deploy in the appropriate environment to test the functionality and to give the final sign-off of the build because, at that time, there was no concept of auto-deployment of the code. However, now there are many cloud computing platforms. One of the most popular is the Azure pipeline for building the code and deployment at regular intervals as code is pushed.

Azure Pipeline helps us to deploy code automatically in prod and test fastly. As the team pushes code within a few seconds pipeline is triggered automatically, and code deployment is started. Azure pipeline helps us ensure consistent and quality code readily available to users.

We can use the Azure pipeline with Cypress for code deployment and execute the test cases.

We have created a Cypress spec file for the scenario. The next step is to set up the Azure pipeline to execute our test cases using Cypress.

Pre-requisites

Let’s look at the prerequisites to run Cypress tests in Azure DevOps Pipeline.

1. Cypress code has already been pushed into the GitHub repository.
2. Azure free account already created through URL.
3. User is already logged into Azure DevOps.

Step 1: Create the pipeline.

1. Click on Pipelines from the left navigation and then click on New pipeline.



2. Click on New pipeline.



3. After clicking on the New pipeline, the below page is open.

Step 2: Select Classic editor.

Click on Use the classic editor to create a pipeline without YAML from the above screen. The classic editor walks you through using a wizard-like process that guides you as you create your build.

Step 3: Select the repository.

1. Click on the GitHub icon to select the repository.



2. Select the repository and branch name from the existing repository.



3. Click on Continue, the below page is open.

Step 4: Select Empty Job.

1. Select the template as an Empty job.



2. After clicking on “Empty job” on the next screen, we can see under the Tasks tab three options: Pipeline, Get Source, and Agent job. By default, the pipeline is selected.

On the right side:

1. Enter the Name of the pipeline, e.g., Cypress Lambdatest Project.
2. In the Agent pool: Select ‘Azure Pipelines’ from the drop-down.
3. In the Agent Specification: Select OS ‘window-latest’ (Test cases are run in this OS).

Step 5: Configure Get Source under the Task tab.

Under Get sources, make sure the correct repository and build are selected.

Step 6: Configure Agent Job under the Task tab.

In the Agent Job section under the Tasks tab, enter the following information.

1. Enter the Display Name of the pipeline, e.g., Cypress Lambdatest.
2. In the Agent pool: Select ‘Azure Pipelines’ from the drop-down.
3. In the Agent Specification: Select Agent ‘window-latest’ (test cases are run in this agent).

Step 7: Add the npm package.

Now Click on the (+) icon against the Project, search for “npm” and click on the Add button.

After adding “npm,” from the left side, click on npm install (See below screenshot). Enter/Select the below data (on the right side) to install the Cypress and other dependencies.

1. Enter the Display name, e.g., npm install.
2. In the Command: Enter ‘install’.
3. In the Control Options: Select Check Box ‘Enabled’ and Check Box ‘Continue on error’.

Step 8: Add npm package to Run Cypress Tests in Azure DevOps Pipeline.

Now Click on the (+) icon, search for “npm” again, and click on the Add button.
Click on npm install from the left side (See below screenshot). On the Right side, enter /select the below data to install Cypress and other dependencies.

1. Enter the Display name, e.g., Run Cypress Lambdatest.
2. In the Command: Enter ‘custom’.
3. In the Command and arguments: Enter run cy:report.

Step 9: Click Save & queue.

Now Click on ‘Save & queue’.

As the user clicks on Save & queue, this will open a dialogue box as below. Select the agent pool, agent, and the branch from which Azure DevOps needs to execute the build

Step 10: Click Save and run.

As we click on Save and run, the Job starts executing.

1. In the first step, the job is initialized.



2. In the second step, code is pulled from the repository. In this case, the code pulls from .github.



3. In the third step, the npm command runs to install the package.



4. In the fourth step, the Cypress test case starts executing. In the below screenshot, we can see using the command cy:report (Which we have given in package.json) test case starts executing.



5. In the fifth and sixth steps, we have to purge the repository, and the job is finished.

How to run Cypress tests in Azure DevOps pipeline on cloud Cypress Grid?

Azure Pipelines is a CI server that automates the build and deploys process of your native web apps. There is an easy way to integrate LambdaTest to run Cypress tests in Azure DevOps pipelines.

LambdaTest is a cloud-based cross browser testing platform that lets you test your website on an online browser farm of 3,000+ browsers, operating systems, and device emulators. It can be used to perform manual web testing and automated browser testing.

Using Cypress automation tools like LambdaTest, you can execute & analyze Cypress test scripts online as a part of the build process. It enables continuous testing, build, and deployment of iterative code changes. Best CI/CD tools help in catching failures ahead of the production stage and mitigate them as they occur. Perform Cypress UI testing on 40+ browsers and browser versions, including headless ones. Other than Cypress, LambdaTest also offers Selenium automation testing on Azure Marketplace.

You can also Subscribe to the LambdaTest YouTube Channel and stay updated with the latest tutorials around automated browser testing, Selenium testing, CI/CD, and more.

To simplify, we have ported the test scenarios mentioned in the earlier section of this blog on how to run Cypress tests in Azure DevOps pipeline so that tests run parallel on LambdaTest Cypress Grid.

Before getting started with Cypress and integrating the Azure pipeline with LambdaTest, we should have the information below information:

1. An Azure CI account.
2. A Git or GitHub repository.
3. LambdaTest Authentication Credentials that can be accessed from the LambdaTest profile page.

There are steps to integrate Azure pipelines with LambdaTest and run Cypress tests in Azure DevOps pipeline, as explained below.

Step 1: Create a project.

The first step is to create the project. Click on “New project” and fill in the required field.

Step 2: Create the pipeline.

Once the project is created the next step is to create the pipeline by clicking on Create Pipeline.

Step 3: Select GitHub.

If you have your code repository in GitHub, then select the option for GitHub from the values.

Step 4: Create a YAML file.

Once you choose an option for code repository, you will be asked to log in. Once you log in, you need to choose the specific repository. Here, the repository will pick the azure-pipelines.yml file.

FileName: azure-pipelines.yml

Yaml file explanation:

• trigger: Under trigger, You can specify the full name of the branch (for example, master).
• pool: Under the pool, we can mention the name of the agent where we want to run your test case.
• variable: Under variable, you can add LambdaTest credential.
• steps: Under steps, you can add various tasks here. We added a task to install the node with the specific version.
• script: Under the script, you can mention the package you want to install using npm and run the test cases using npm.

File: lambdatest-config.json:

In the lambdatest-config file, we can provide the credential, run_setting, and browser with the version in which we want to run our test cases.

Step 5: Run Test cases.

We are using Cypress version 9.2.0 for testing our scenarios. At the time of writing this blog on how to run Cypress tests in Azure DevOps pipeline, the latest version of Cypress was 10.11.0 when writing this blog.

As the build in Azure Pipeline starts, the first step job is initialized.

In the second step, code is pulled from the repository. In this case, code pulled from .github.

In the third step, the node is installed.

In the fourth step, npm install various packages, and finally, the cypress test cases start executing.

In the screenshot below, we can see that the command cy:report (Which we have given in package.json) test case starts executing in the LambdaTest platform.

Execution:

In the below screen, you can see test cases start running in four browsers parallelly (Browser details given above in File: lambdatest-config.json).

In the below screenshot, we can see the test cases are passed in Firefox and in Chrome browser, test cases are running.

In the below screenshot, we can see that both test cases are passed in different browsers.

Here are the logs that are generated when the test cases are run successfully.

In the below screenshot, we can see that VIDEO products are loading.

In the below screenshot, we can see after scrolling to the bottom product is clicked.

If you’re a developer or a tester, the Cypress 101 certification will help you understand the fundamentals of Cypress end to end testing and prepare you to write tests faster, with greater confidence and clarity.

To leverage the benefits of cloud-based testing, you can integrate Cypress with Docker containers. Learn how to set up Cypress docker configurations in our guide.

Wrapping Up

I hope this gives you a great starting point for building your first Azure DevOps pipeline. You will understand how we can run Cypress tests in Azure DevOps Pipelines on cloud-based platforms like LambdaTest.

Stay tuned for the topics on the Cypress Docker Tutorial. We delve into topics like seamless integration with Docker, optimizing container resources, and maximizing testing efficiency.