Structural testing: A Detailed Tutorial [With Examples]

OVERVIEW

Structural testing is the technique that tests the system's structure or component. It determines what is happening inside the system or application and is often referred to as glass box or white box testing. In this type of testing, the testers are required to know the internal implementations of the code, like how the software is implemented and how it works.

For example, a structural technique wants to understand how loops in the software work. Different test cases can be derived to exercise the loop once, twice, or more. This may be done irrespective of the functionality of the software.

Structural testing applies at all testing levels. Developers use it in component testing and component integration testing when there is robust tool support for code coverage. It's also used in the system and acceptance testing. However, the structures are different.

Structural testing is a software testing approach that tests the code structure and intended system flows. For example, verifying the actual code to test whether the conditional statements are correctly implemented and every statement in the code is correctly executed. It is also known as structure-based, glass-box, clear-box, and white box testing.

To perform this testing, one needs to understand the code thoroughly. That is why this testing is usually performed by developers who write the code requirements, as they are well aware of the overall system functionality and user flows.

Let's consider a scenario where you wish to test a specific error message in an application. You need to test the trigger condition for it, but there can be many triggers.

It is possible to overlook one while testing scenarios mentioned in the software requirement specifications (SRS). This is where a structural testing technique will handle the trigger, aiming to cover all the nodes and paths in the code structure.

By running structural tests, the test cases written by the system requirements can be analyzed first, and more test cases can be added to expand the test coverage.

In this section, let’s look at some of the characteristics of structural testing.

• It requires an understanding of the software's internal code. Hence, members of the developer team can perform who understand how the software was built.
• This testing is focused on how the system performs tasks rather than how users perceive it or how it functions.
• Better coverage as it thoroughly tests the entire code, and defects are quickly fixed. The likelihood of overlooking an error is pretty negligible.
• Can be performed at many levels, from high to low, involving the end-to-end testing of the system. It can also be used to augment functional testing.

Following are the advantages and disadvantages of using the structural testing approach.

Pros

• It does not require a lot of manual intervention as it is an automated process.
• Not a time-consuming process.
• Helps to identify all the defects in an early phase.
• Removes redundant code or statements easily.
• Provides easy coding and implementation.
• Delivers detailed testing of the software.

Cons

• In-depth knowledge of programming languages is mandatory.
• Training is required for the testing tool.
• Expensive in terms of money as resources are required to perform it efficiently.
• There is a probability of some commands, statements, or branches being missed unintentionally.

Structural testing is divided into the following four main types.

• Mutation Testing
• Data Flow Testing
• Control Flow Testing
• Slice-Based Testing

Let us understand each of these testing types in more detail.

Mutation testing is where errors are purposely introduced into an application under test to verify whether the existing test cases can detect the error. In this type of testing, the mutant of the program is created by making modifications to the original program. Minor changes are made to the source code to see if the defined test cases can detect code errors.

It is a type of white box testing, which is also called a fault-based testing strategy, where you can create a fault in the program. The expectation is that none of the test cases should pass. If the test case passes, there is an error in the code, and the mutant (the modified version of our code) lives. If the test case fails, there is no error in the code, and the mutant is killed. The end goal is to kill all mutants.

In addition, mutation testing also helps to test the quality of the defined test cases or the test suites with the scope to write effective test cases. The more mutants are killed, the better the quality of the test cases.

Data Flow Testing

Data flow testing is a software testing technique based on the data values and their usage in the program. This software testing verifies that data values are correctly used and that they generate the correct results. This testing also helps to trace the dependencies between data values on a particular execution path.

It uses the control flow graph to detect invalid things that can interrupt data flow. Anomalies in the data flow are detected at the time of association between values and variables for the following reasons.

• If the initialized variables are not even used once.
• If the variables are used without initialization.

Data flow anomalies: Data flow anomalies are the errors encountered in a software application. These are classified as Type 1, 2, and 3, respectively.

Let us understand the different data flow anomalies:

Type 1: A variable is defined, and the same value is assigned twice.

list_1 = [1,2,3,4]
list_1 = [5,6,7,8]

for var in list_1: 
print(var)

list_1 variable is defined, and two different values are assigned to it. The first value is ignored.

Note: Type 1 anomalies do not result in application failure.

Type 2: The variable’s value is referenced before the variable is defined.

The loop in the above example has no values to iterate over.

Note: Type 2 anomalies cause the application to fail.

Type 3: A data value generated but never used.

list_1 = [1,2,3,4]
list_2 = [5,6,7,8]
for var in list_1: 
print(var)

list_2 variable is not referenced.

Note: Type 3 anomalies may not cause the program to fail.

Types of Data Flow Testing

The following are the types of data flow testing:

• Static data flow testing: It means that you go through the code and control flow graph to identify data anomalies without executing the code.
• Dynamic data flow testing: It helps you identify the specific paths and then create test suites to validate the paths to identify anomalies that you may have missed during static data flow testing.

Data Flow Testing Process

The data flow testing process defines the dependencies between data values. You are required to define the different paths that can be followed in a program. The process involves the following steps.

• Draw a control flow graph

You need to draw a control flow graph, which is a graphical representation of the paths you will follow in your program.

cost = 20

a = int(input("How many visitor seats did you reserve? ")) b = int(input("How many member seats did you reserve? ")) if a>b: bill = cost -1 else: bill = cost print(bill)

In the above example, a member should get a discount if they invite a visitor.


• Definition and usage of variables and associated data values

A variable is defined or used in a program. In the control flow graph, variables are defined at each node. Each node is named as per the variable type it contains.

• If a variable definition is done at a particular node, the defining node is created.
• If a variable is utilized at a node, a usage node is created.

Considering the variable cost in the control flow graph, these are the defining and usage nodes described in the table below.




Node	Type	Code
1	Defining node	cost = 20
5	Usage node	bill = cost -1
7	Usage node	bill = cost



• Define definition-usage paths

The following are the two types of definition-usage paths:

• du paths - These definition paths start with a definition node and end with a usage node.
• dc (decision clear) paths - The dc path has more than one definition node even though it ends at a usage node.

An example of a dc path about the bill variable is shown below.




Node	Type	Code
5	Defining node	bill = cost -1
7	Defining node	bill = cost
8	Usage node	print(bill)



• Create the test suite

This is all about adding the desired inputs that help you to create a comprehensive test suite.

Note: You need to have a different test suite for each variable. The test suite will help you identify data flow anomalies.

Control Flow Testing

Control flow testing is the basic model of structural testing. This testing technique is part of white box testing. It contains the following features:

• The purpose of this technique is to determine the execution order of statements or instructions of the program through a control structure. The control structure related to the program helps in the program's quick and easy test case development.
• In this testing type, the test engineer selects a specific part of an extensive program to set up the testing path.
• Generally, this testing technique is used during unit testing.
• Complete information on all the software features and logic is required to execute the control flow testing.

Slice-Based Testing

Slice-based testing can be defined as a software testing technique based on slices – executable parts of the application or group of statements that impact certain values at particular points of interest in the program.

For example, parts of a program where variables are defined or an output for a group of statements. It contains the following features.

• The purpose of this testing is to split the complete code into small chunks or slices and then closely evaluate each chunk.
• This testing type is beneficial for software maintenance along with fixing the software.
• This testing is useful for software debugging, program understanding, and quantification of functional cohesion.

Slice-Based Testing Process

Example: Following is the code to print even and odd numbers using Python

num_list = range(1,12) 
even_nums = [] 
odd_nums = []
 for var in num_list:
if var%2==0:
even_nums.append(var)
print(f"Even numbers: {even_nums}") 
elif var%3==0:
odd_nums.append(var)
print(f"Odd numbers: {odd_nums}")

Following are the two ways to look at a slice:

• Follow the variable’s path of your interest or
• Follow the code’s part that impacts the output

In this example, if you look at the odd numbers output, you can trace the code’s part that gives you this output.

As per the slicing criteria by Mark Weiser, in the above example, to get the slice, you start from the output from line 10, which becomes the n. The variable is defined as var.

So the slicing criteria are defined as

The output will be: 10,9,8,4,3,1

So, the slice in this code is:

num_list = range(1,12) 
odd_nums = []
  for var in num_list: 
   elif var%3==0:
   odd_nums.append(var)
    print(f"Odd numbers: {odd_nums}")

Types of Slice-Based Testing

The following are the two types of slice-based testing:

• Static slice-based testing: It focuses on a particular variable. If you consider the output in the above example as var, you can trace the slice that impacts it as 10,9,8,7,6,5,4,3,2,1.

Here it is verified that the code is correct in terms of syntax and requirements, and then you do not execute it. Static slice-based testing is also known as verification testing.



• Dynamic slice-based testing: The example explained above, where you looked at the statements that impact the printing of the odd numbers, is dynamic slice-based testing. Here, the focus is only on what directly affects the individual output.

The code is executed and uses test data to ensure it works as expected. The range can be increased to range(1,50) to see whether it generates only odd numbers. Dynamic slice-based testing is also known as validation testing.

It is important to note that dynamic slice-based testing is ‘smaller’ when compared to its static counterpart.

The following are the factors to determine the slicing criteria.

• Statements where values are defined, assigned, and reassigned.
• Statements where values are entered from outside the program, for example, user inputs.
• Statements that print or give output.
• The program’s last statement, for example, a function call that can define values or specify values to arguments.

To test different code aspects, you need to understand the control flows.

The visual representation of different code sections helps define the paths that can be followed during execution is called a control flow graph. It has several components like nodes, edges, paths, junctions, and decision points. The graph can be created manually or automatically, where software is used to export the graph from the source code.

Let us understand these components in more detail.

• Process block: This component is used to represent a section of code that is executed sequentially. This means that the code is executed the same way every time, and no decisions or ‘branching out' need to be made.

The process block comprises nodes with one entry and exit path.

Example:



The process block is not an important part of the control flow graph; you can test it only once.



• Decision points: These are the critical components in the code control flow. The decisions are made within these nodes. This component of the control flow graph is made up of only one node with a minimum of two outputs.

The decision can be based on conditional statements such as if-else statements with two possible outcomes and case statements with multiple output values.



In the above diagram, a decision point (Age=18) is followed by either ‘yes’ or ‘no’ options based on the condition.



• Junction points: In the above diagram, you can quickly identify the junction points from where the decision points join. Junction points contain multiple entry paths but only one exit path.

A few things to consider while constructing a control flow graph:

• Keep the control flow graph simple. You can do this by combining components that may be less significant, for example, process blocks.
• Ensure that at the decision points, only one decision is made. In more complex control flow graphs, some consequences come after making a decision. In the above example, you could also add that if an individual is 18 years or older, they are eligible and need to pay for a ticket. If they are not, entry is free. The other decision must skip a few nodes, and all those steps must be shown in the control flow graph.

After defining the control flow graph, proceed to the next phase in the control flow testing procedure, describing the extent to which the code needs to be tested.

Define the extent to test the code

Trying to cover all paths in the tests is practically impossible. You need to find a middle ground to determine how much testing can be done.

Let's say that you aim at testing 50% of the code, which means that you will define all executable code statements and aim at testing at least half of them. However, the question is, ‘do you need to define all possible executable paths?’ This, again, might be practically impossible.

A better approach might be aiming to test 50% of the paths you can identify at each code section. There are different levels of coverage, statement, branch, and path coverage, explained later in this section.

Create test cases

The next step is to create the test cases you will execute. The test cases in structural-based testing are determined based on the following factors:

• The executable statements.
• The decisions that need to be made.
• The possible paths that can be tracked.
• The conditions that need to be met (can be multiple or boolean).

The above factors give you an idea of the types of test cases you need to create. You can also use a structural test generation tool to create test cases.

You can run the test cases and enter input or data to check how the code executes it and verify if you get the intended results. For example, enter an array in a function call to check the results you get after looping through it or whether the decision points are making the correct decisions.

Here, all you need to do is check whether you get accurate results after execution. For example, when you enter an array where all the values are above 18, you should have all the decision points resulting in eligibility.

To carry out control flow testing, the following assumptions are applicable.

• The only bugs present are the ones that can affect control flow.
• All variables, functions, and elements are defined correctly.

Following are the different levels of coverage in control flow testing.

This technique aims at implementing all programming statements with minimal tests. In structural-based testing, executable code statements have a critical role in deciding the methods of designing the tests. The aim is to achieve 100% coverage, so every executable statement should be tested at least once. The higher the coverage, the lesser the likelihood of missing the bugs and errors.

This technique runs a series of tests to ensure that all the branches are tested at least once. This technique involves testing the points in the control flow graph branches (where decisions are made). The outcomes are boolean. Even if a switch statement has multiple outcomes, each case block compares a pair of values.

The aim is to achieve 100% branch coverage, where you must test each outcome at least once at each decision level. Since you are dealing with boolean outcomes, you should aim to run at least two tests per code section.

This technique aims to test all possible paths, which means that each statement and branch is covered. This level of coverage is more exhaustive when compared to decision and statement coverage.

The purpose is to discover all possible paths and test them at least once. This can be extremely time-consuming. However, it can help find bugs or errors in the code or even aspects that need to be defined, for example, user input.

Here are the basic formulas you can use to measure the effectiveness of your structural tests.

• Statement Testing = Number of statements executed x 100% / Total number of statements
• Branch Testing = Number of decision-related outcomes tested x 100 % / Total number of decision outcomes that are undertaken
• Path Coverage = Number of paths executed x 100 % / Total number of paths in the program

Following are some of the tools used in structural-based testing.

• JBehave: It is a framework executed using Java programming language that allows behavior-driven development.
• Cucumber: It is a framework to develop test cases for automation testing to evaluate system behavior. You can use Cucumber to create a testing script that helps you to perform system acceptance and automation testing.
• JUnit: It is an open-source unit testing framework written in Java. This helps improve the developer's overall productivity by improving the code requirements.
• cfix: It is a specialized structural testing tool supported by the C/C++ programming language.

For automation testing needs, always make sure you test your software applications on real world environments for accurate testing. Continuous quality cloud platform like LambdaTest lets you perform manual and automated testing of websites and applications on an online device farm of 3000+ real browsers, devices, and platform combinations.

Also, subscribe to LambdaTest YouTube Channel and stay updated with the detailed tutorials around Selenium testing, Cypress testing, and more.

Following are some of the best practices that need to be followed while performing structural-based testing:

• Label and name the tests correctly so that if someone else is running the tests, they can locate them quickly
• Before code refactoring and optimization are used in different environments, make sure that its overall structure and flow are ideal
• Run tests separately so it will be easier to identify and fix bugs
• Generate tests before making any changes to them. The tests should run as expected. If something breaks, it is easy to trace and fix the problem
• Keep the tests of each section separate so that if there are changes down the line, it is not required to change a lot of tests

In this tutorial, we explored structural testing, its advantages, disadvantages, best practices, tools, and more. It is always recommended to combine different testing types and approaches. For example, structural testing and requirement testing complement each other, as there can be features that might not have developed when structural testing was carried out.