How To Easily Perform Specflow Parallel Execution With NUnit

One of the prime responsibilities of any automation testers is to ensure that the tests get executed at an optimal pace. This can only be achieved if the implemented tests (or test suites) effectively utilize the automation testing framework (like Selenium) and the infrastructure where the tests are executed. Parallel execution in Selenium is the starting point for speeding up the test execution, as tests can be run simultaneously against different ‘environments.’

SpecFlow, the .NET-based framework for BDD (Behavior Driven Development) and Acceptance-driven test development used by Selenium C# Automation testers also support parallel execution. The support for Specflow Parallel Execution in Selenium was introduced in SpecFlow v.2.0. By executing BDD tests (or scenarios) in parallel, you can achieve faster execution times and accelerated feedback in the CI process.

In this SpecFlow NUnit tutorial, we look at how you can achieve NUnit Specflow Parallel Execution. Hence, NUnit will be used as the SpecFlow test runner for demonstrating parallel execution in Selenium. For a quick recap on SpecFlow with Selenium C#, you can refer to our blog on SpecFlow tutorial for Automation Testing.

Specflow Parallel Execution

SpecFlow+ Runner (or SpecRun) is the default test runner that was introduced in SpecFlow 3.5.x. For a SpecFlow NUnit project in Visual Studio, you have to select the runner as NUnit (instead of SpecFlow+ Runner) since the intent is to achieve NUnit Specflow Parallel Execution.

SpecFlow offers a couple of ways to perform Specflow Parallel Execution in Selenium. However, the effort involved in achieving parallelization with SpecFlow depends on the number of external dependencies in the tests. With Specflow NUnit, parallel test execution is possible ‘only’ at the features (or class) level and not the ‘scenario method’ level.

When performing NUnit Specflow Parallel Execution, the utmost care has to be taken to ensure that the NUnit tests are thread-safe. The primary reason is that NUnit (and other frameworks like xUnit) do not support Memory Isolation. The onus of the tests for thread-safety squarely lies on the developer implementing the test. SpecFlow provides thread-safe ScenarioContext used in parallel tests for injecting the context classes in the binding class.

We cover some of the important concepts essential for implementing Specflow Parallel Execution with NUnit.

Parallel Test Execution using Memory (AppDomain) Isolation

In case the test application (or test scenarios) do not have any external dependencies, and the application depends purely on a static state (e.g., caches), tests can be executed in parallel isolating by the AppDomain. Here, each thread’s memory (e.g., static fields) is isolated, and each test execution thread is hosted in a separate AppDomain.

The SpecFlow+ runner (or SpecRun) has to be used for Specflow Parallel Execution, as the runner supports parallel test execution with AppDomain, SharedAppDomain, and Process Isolation.

Note: Parallel execution with Memory isolation (or AppDomain) does not apply to NUnit tests since NUnit does not support Memory isolation.

Parallel Test Execution without Memory (AppDomain) Isolation

In case the test scenarios are not dependent on static fields, Specflow Parallel Execution can be achieved without AppDomain isolation. Parallel test execution using this approach helps in minimizing the initialization footprint, thereby lowering the memory requirements.

The NUnit test runner lets you run Specflow Parallel Execution without AppDomain isolation. Other test runners like MSTest, XUnit v2, and SpecRun also support this feature.

With Specflow Parallel Execution in Selenium and NUnit, the static context properties like ScenarioContext.Current, FeatureContext.Current and ScenarioStepContext.Current cannot be used. As shown below, the usage of these properties in NUnit Specflow Parallel Execution would result in SpecFlowException:

Instead, Dependency Injection (DI) or Context Injection should be used to instantiate and inject instances for scenarios.

Take this certification to master the fundamentals of Selenium automation testing with C# and prove your credibility as a tester.

Here’s a short glimpse of the Selenium C# 101 certification from LambdaTest:

Sharing data between Steps and Bindings

When performing automation testing with SpecFlow and the NUnit framework, you would come across scenarios where data has to be shared across different step definitions. It could be input data that is initialized in one step, and the same data is used in subsequent steps in the feature file.

This can be done using the ScenarioContext or FeatureContext. As the name indicates, ScenarioContext is created for individual scenario execution, and the same is disposed of when the scenario execution is complete. On the other hand, FeatureContext is created when the first scenario from a feature is executed and disposed of when the execution of that feature’s scenario is complete.

Here are some ways for sharing data between bindings (or Steps/Features) in SpecFlow:

• Using Static Members (or Private Fields)

The easiest way for sharing data across steps is via creating private fields in the step definition class. Here, the data to be shared across steps is stored in different fields in the binding class. However, this approach’s major drawback is the state will not be shared when the steps in a feature are implemented across different binding classes.

• Using ScenarioContext

In SpecFlow, the ScenarioContext class holds the shared state during the execution of a Scenario. ScenarioContext can be used to store values in key-value pairs between different steps. It is useful for sharing data between different steps and binding classes, as it is accessible by all the involved binding classes.

The properties ScenarioContext.Current and FeatureContext.Current are marked as obsolete in SpecFlow 3.0 since they could not be used when scenarios are executing in parallel.

In SpecFlow, the automation that connects the Gherkin specifications to the source code is Binding. The Binding class is a public class that is marked with the [Binding] attribute. Without the Binding class, SpecFlow will not be able to detect the class with Step Definitions. Hooks and Step definitions are the two kinds of bindings available in SpecFlow. We would look at Hooks at a later point in time in this blog.

For accessing ScenarioContext in Bindings, we use Context Injection by acquiring the instance of ScenarioContext through a request via a constructor. In SpecFlow, Context Injection works out of the box. To get the current context instance, we declare a ScenarioContext parameter for the step definition class’s constructor and save the same in an instance field.

Unlike the now-obsolete ScenarioContext.Current & FeatureContext.Current, which were not useful with NUnit SpecFlow parallel execution, ScenarioContext can be used in parallel execution scenarios.

Here is a sample usage of ScenarioContext in the CalculatorSteps example of SpecFlow:

As shown above, the instance field _scenariocontext of type ScenarioContext can be used across different properties and methods.

ScenarioContext also helps in storing values in a dictionary (i.e. < Key >, < Value > pairs) between steps. It is much better than using private variables in step definition classes.

Here is an example where we store (or set) data in ScenarioContext and retrieve (or get) the corresponding data from the same instance.

FeatureContext can be acquired similarly to ScenarioContext. It has to be acquired via a function parameter in the Before/After Feature Hooks.

ScenarioContext.ScenarioInfo lets you access information such as title, tags, description, etc., related to the currently executing scenario.

• Using Context Injection

The SpecFlow framework has to take care of the step definitions and the binding classes. It also configures the dependencies of its infrastructure classes. This is where Context Injection – a simple dependency injection shipped along with SpecFlow can be extremely useful.

Context Injection is used for instantiating and injecting class instances for scenarios. This helps group the shared state in context classes and inject them into the binding classes that require access to the shared state. For sharing data across bindings using Context Injection, we have to do the following:

1. Create classes that represent the shared data.
2. The shared data is defined as Constructor Parameters in the Binding classes that intend to use the data.
3. The constructor arguments are saved as instance fields to be used to inject the same context across different step-definitions.

Below are some of the important rules about Context Injection:

• The lifetime of the injected object is limited to the scenario’s execution.
• In case the injected object implements IDisposable, the same would be disposed of after the scenario has been executed.
• The SpecFlow framework has built-in logic to detect if the instance of a scenario object is created or not. Hence, the same instance of the object is returned within a particular scenario execution.

Further rules are available in SpecFlow’s official documentation on Context Injection. Here is a simple example of the usage of Context Injection:

In the above code snippet, we have registered the Web Driver object (or browser instance) through a constructor (i.e., WebDriveContext). The constructor is used for initializing the Web Driver object.

For using the property in the step bindings, the Context needs to be injected into the constructor. As mentioned earlier, we create an object for the Context, further included in the constructor.

For invoking the Web Driver in further implementation, we use the webDriverContext.webdriver from the newly created webDriverContext object.

Sample – IWebDriver Injection

Hooks in SpecFlow

In the last few sections, we have been referring to SpecFlow Hooks. Let’s look at what Hooks are and how they can be used for Specflow Parallel Execution. Hooks (or event bindings) are used for performing additional automation logic at times, such as setup necessary for executing a test scenario. For using hooks, the [Binding] attribute should be added to the Class.

Hooks are global and can be restricted so that they run only for specific scenarios or features by defining scope binding, filtered by tags. The execution order of Hooks for the same event is undefined unless it is explicitly specified.

You can add SpecFlow hooks to an existing SpecFlow project by performing the following steps:

1. Add a new item to the project.



2. Select SpecFlow Hooks (event bindings) and assign a logical name to the file (e.g. Hooks.cs)

On successful addition of the Hooks file (i.e. Hooks.cs), the following class is automatically generated:

Akin to NUnit attributes, there are predefined hooks in SpecFlow that are executed after certain events occur during the test execution. Here are the available orders in SpecFlow hooks and the order in which they are executed:

[BeforeTestRun]	[BeforeFeature]
[BeforeScenario]	[BeforeScenarioBlock]
[BeforeStep]	[AfterStep]
[AfterScenarioBlock]	[AfterScenario]
[AfterFeature]	[AfterTestRun]

Here is the detailed description of the available Hook attributes:

Attribute	Description
[BeforeFeature]/[AfterFeature]	The Automation Logic has to run before/after the execution of each feature. The method under the attribute must be static.
[BeforeTestRun]/[AfterTestRun]	The Automation Logic has to run before/after the execution of each test run. The method under the attribute must be static.
[BeforeScenario]/[AfterScenario]	The Automation Logic has to run before/after the execution of each scenario.
[BeforeStep]/[AfterStep]	The Automation Logic has to run before/after the execution of each scenario step.
[BeforeScenarioBlock]/[AfterScenarioBlock]	The Automation Logic has to run before/after the execution of each scenario block (i.e., between ‘given’ and ‘when’).

In case a Hook throws an unhandled exception, the subsequent hooks of the same type are not executed. We demonstrate the usage of hooks along with IObjectContainer in the subsequent section.

The IObjectContainer

Do you know SpecFlow uses a special dependency injection framework called BoDi for realizing Context Injection? BoDi is a simple IoC (Inversion of Control) container that is also embeddable as a source code. It is an open-source mini DI (Dependency Injection) framework that is available on GitHub.

The Container used by SpecFlow is customizable, which means that there is flexibility for modifying the resolution rules or including object instances that are already created. The Container can be customized from a SpecFlow Plugin or in the [Before Scenario] hook in SpecFlow.

For customizing the Injection rules, the responsible class should obtain an instance of BoDi.IObjectContainer – an instance of the scenario execution container. Constructor Injection is used for realizing the same.

The below code snippet registers the LambdaTestDriver (which is used for invoking the Chrome browser) instance to SpecFlow’s default IObjectContainer:

Demonstration of Hooks and IObjectContainer

This SpecFlow Tutorial for beginners and professionals will help you learn how to use SpecFlow framework with Selenium C# for performing Selenium automation testing.

Demonstration: Specflow Parallel Execution with Selenium and NUnit

To get started with NUnit Specflow Parallel Execution, we first create a .NET project with SpecFlow.

By default, the Test Runner selected is SpecRun (Or SpecFlow+Runner); hence, the runner (or test framework) should be changed to NUnit. When creating a SpecFlow project, select ‘.NET Framework 4.8’ for the framework and select ‘NUnit’ as the Test Framework.

The newly created project would require reference to the following libraries (or packages):

• Selenium.WebDriver
• NUnit
• Specflow
• SpecFlow.Tools.MsBuild.Generation
• Specflow.NUnit

The packages should be installed from the Package Manager (PM) console, accessed through the ‘Tools’ -> ‘NuGet Package Manager’ -> ‘Package Manager Console.’

For installing the packages, run the following commands on the Package Manager (PM) Console:

To confirm whether the packages are installed or not, run the Get-Package command on the PM Console:

SpecFlow and SpecFlow.NUnit are the base packages that are necessary for a SpecFlow NUnit project. With the base project ready, we look at the three test scenarios that would be executed in parallel against different browser and platform combinations.

Test Scenario – 1 (GoogleSearch.feature)

1. Go to Google.
2. Search LambdaTest.
3. Open the first search result. Assert if the page title does not match with the expected title.

Test Scenario – 2 (ToDoApp.feature)

1. Go to LambdaTest ToDo App.
2. Select the first two items in the ToDo list.
3. Add a new item, ‘Happy Testing At LambdaTest’ to the list.
4. Confirm if the new item is added to the list.

Test Scenario – 3 (LambdaTestSearch.feature)

1. Go to the DuckDuckGo search page.
2. Search LambdaTest Blog.
3. Open the first link on the search page.
4. Confirm if the page title matches with the expected page title.

Here is the overall project (or directory structure):

• Features: As there are three test scenarios, we would have three feature files in the project (Location – <parent-project-folder>\Features)
• Step Definitions: The ‘three’ step definition files (*.cs) corresponding to ‘three’ feature files are located at <parent-project-folder>\Steps.
• Hooks – The SpecFlow Hooks file (Hooks.cs) is created in the <parent-project-folder>\Hooks folder.
• App Configuration – An App File (App.config) is created in the project’s root folder (i.e. <parent-project-folder>). This XML file will store the test configuration (i.e., browser and platform combinations) against which the tests would be executed.

This is how the overall project structure looks after creating the necessary feature files (*.feature), step definition files (*.cs), Hook file, and App configuration files.

The test scenarios mentioned above would be executed on the cloud-based Selenium Grid by LambdaTest. The desired capabilities are generated using the LambdaTest capabilities generator. For demonstrating NUnit Specflow Parallel Execution, the input parameters (or examples/scenarios) for the feature file ‘LambdaTestSearch.feature’ would be passed in the form – <Build>, <Name>, <Platform>, <BrowserName>, and <Version>.

The browser and platform combinations for the scenarios in other feature files (i.e., ToDoApp.feature and GoogleSearch.feature) are fetched from App.config.

Implementation

We first have a look at the three feature files in the SpecFlow NUnit project:

FileName – GoogleSearch.feature

FileName – ToDoApp.feature

FileName – LambdaTestSearch.feature

FileName – App.config

FileName – Hooks.cs

FileName – GoogleSearchSteps.cs

FileName – ToDoAppSteps.cs

FileName – DuckDuckGoSearchSteps.cs

Code WalkThrough

We would not dig deeper into the usage of locating web locators, Selenium APIs, etc., which are extensively used in the Step Definition files in the project. We would look into the two major aspects of the tests:

• Registration of browser instance, usage of Context Injection for handling browser and OS combinations.
• How SpecFlow parallel execution is achieved in the above implementation.
• How different browser and platform combinations are handled through App.config and using ‘Examples’ in Feature files.

Here are the important steps that are instrumental in achieving Specflow Parallel Execution with NUnit project:

Context Injection & registering the LambdaTestDriver (or IWebDriver) Instance

The Hooks class in Hooks.cs is responsible for customizing the injection rules. Hence, we obtain an instance of the scenario execution container, an instance of the IObjectContainer. This is achieved through constructor injection.

_objectContainer, the IObjectContainer instance, will be further used to add the LambdaTestDriver to the container. The Binding classes can specify the dependencies of LambdaTestDriver.

For accessing ScenarioContext in Bindings, we make use of the Context Injection technique. In [Before Scenario], we first save the ScenarioContext in the _scenarioContext field so that ScenarioContext is accessible from all the Bindings.

The object of the LambdaTestDriver class (that contains methods for initializing the RemoteWebDriver using a different browser and OS combination) uses ScenarioContext (or _scenariocontext, which is the Current ScenarioContext) as the input argument. The ‘LTDriver’ key of ScenarioContext is set to the newly created object of the LambdaTest class.

From [BeforeScenario] in Hooks, we register the LTDriver object in the IObjectContainer (i.e., _objectContainer) instance that we received from the constructor.

The current ScenarioContext can be used in the Bindings of the respective Step Definition files. As seen below, the ScenarioContext is updated in the constructor of LambdaTestDriver.

As the ScenarioContext is accessible across Binding classes (i.e., classes under the [BINDING] attribute), the constructor in each Step Definition uses the key “LTDriver” in ScenarioContext, which was set in [BeforeScenario] of Hooks. This is where we are using IWebDriver (or, in our case LambdaTestDriver) injection where the LTDriver object (from Hooks) is injected in the Step definition files.

Handling Browser & OS combinations and realizing SpecFlow parallel execution in NUnit

An instance of the LambdaTestDriver class (defined in Hooks.cs) is created in every step definition file constructor. Since the ScenarioContext, which was set in Hooks, is available across Bindings of classes (and step definition files), the LambdaTestDriver instance (i.e., LTDriver) is set to ScenarioContext[“LTDriver”].

Now that the instance of LambdaTestDriver is created, the next step is to read the corresponding browser and OS combination from the Feature file and instantiate a remote WebDriver object using the Initialization methods (i.e., Init and InitLocal provided by the LambdaTestDriver class.

We have created two separate IWebDriver initialization methods. The step definition files corresponding to the feature files – GoogleSearch.feature and ToDoApp.feature uses the browser & OS combinations from App.config. On the other hand, the step definition file corresponding to the feature file – LambdaTestSearch.feature uses the browser & OS combinations from the feature file itself.

Browser & OS combination from GoogleSearch.feature:

As seen above, the test scenario takes two inputs – profile and environment. Let’s take the case where the profile is ‘parallel’ and the environment is ‘chrome.’ In the step definition where the two inputs are used, the Init method of LambdaTestDriver class is invoked with the profile and environment as input arguments.

In the Init method of LambdaTestDriver class, the GetSection method in ConfigurationManager class is used for accessing the section (“capabilities” + profile) (e.g. “capabilities” + “parallel”) in App.Config. The same method is used for accessing the section (“environments” + environment) (e.g. “environments” + “chrome”) in App.Config.

The entries returned in the NameValueCollection format are parsed for reading the respective entries in the collection.

As seen in the snippet of App.Config, the key values used in the < environments > tag are inline with the Selenium capabilities (i.e., browserName, browserVersion, and platformName) used in Selenium 4.

LambdaTest Capabilities Generator for Selenium 4

The browser and platform capabilities are set using the SetCapability method offered by the DesiredCapabilities class in Selenium.

Once the desired capabilities are set, a Remote WebDriver object is instantiated using the credentials of the remote Selenium Grid by LambdaTest.

On successful execution, the IWebDriver object is returned by the Init method. The same IWebDriver object is used across different steps in the corresponding Step Definition file.

The same steps are repeated for all the profile and environment combinations available in the corresponding feature file.

It is important to note that the ScenarioContext is disposed of once the Scenario is completed. Hence, ScenarioContext will be different for every Scenario executed against different input values.

Browser & OS combination from LambdaTestSearch.feature:

Unlike the other two feature files, we have taken a different approach to handle browser and platform combinations for the DuckDuckGo search test scenario. As shown below, five input values (build, name, platform, browserName, and version) are used for each scenario. The sequencing is inline with the capabilities generated by the LambdaTest Capabilities Generator.

Once a particular input combination in the feature file is encountered, the InitLocal method in LambdaTestDriver class is invoked with those input combinations from the feature file.

As shown below, the Selenium 4 test’s desired capabilities are set using the input values obtained from the feature file.

On successfully executing the InitLocal method, you would get an IWebDriver object used in subsequent step methods in the step definition file (i.e., DuckDuckGoSearchSteps.cs).

The intention of passing browser and OS combinations using two approaches is to demonstrate how SpecFlow NUnit tests can handle cross browser testing scenarios.

SpecFlow parallel execution With NUnit

The NUnit v3 test provider only supports the [Parallelizable] attribute only on Fixtures. It does not generate the [Parallelizable] attributes on feature classes or scenario methods. Doing so would result in an exception (shown below):

Since SpecFlow parallel execution in Selenium with NUnit is only available at Fixtures level; hence, Scenario Injection also would have sufficed for executing tests in parallel. In order words, we might not have required adding the LambdaTestDriver object to the instance of BoDi.IObjectContainer.

For realizing SpecFlow NUnit parallel execution, we set the Fixtures level parallelism to 4. We have set the Parallelism in the step definition file TodoAppSteps.cs.

Execution

NUnit Specflow Parallel Execution is only applicable at the Fixtures level. Though we have set the LevelOfParallelism to 4, a maximum of three ‘different’ scenarios will be executing at a time. In Visual Studio, also enable ‘Run Tests in Parallel’ in the Test Explorer.

Here is the execution snapshot from LambdaTest, where we see that three tests are executing in parallel at any point in time.

Shown below is the execution snapshot from Visual Studio and LambdaTest automation dashboard, which indicates that all the 13 tests have executed successfully:

This NUnit Tutorial for beginners and professionals will help you learn how to use NUnit framework with Selenium C# for performing Selenium automation testing.

Wrapping up

SpecFlow parallel execution with NUnit tests is possible by using the NUnit test runner that supports the Memory Isolation feature. All scenarios in a particular feature are executed on the same thread. However, each thread has a separate and isolated ScenarioContext. Also, Scenarios and the associated hooks do not block each other since they are isolated in different threads during execution.

NUnit SpecFlow parallel execution is only possible at the Fixtures level. This means that Parallelism cannot be done at the Methods or Feature classes level. Hence, it is not possible to achieve parallel execution in Selenium when the SpecFlow tests have a single feature, and the steps have to be executed against different browser and platform combinations. For SpecFlow NUnit parallel tests, Context Injection is the preferred approach for state sharing since it is thread-safe. You have to ensure that the shared ScenarioContext is accessible to the binding classes and step definitions.

The true potential of NUnit Specflow Parallel Execution can be exploited by running the tests in parallel across different browser and OS combinations provided by a cloud-based Selenium Grid like LambdaTest. Such an approach is instrumental in making the most of the features offered by NUnit, SpecFlow, and the cloud-based Selenium Grid.