Creating Browser Compatibility Matrix for Testing Workflow

Browser compatibility testing is a complex problem. From chrome’s latest version 69 to the oldest one, from Firefox’s latest version 61 to the oldest one there are approximately 130 browser versions. This is just the case for two browsers, if I include every browser and browser version, they’ll add up to to hundreds of browser versions. And if I check for their combinations with different windows and mac operating system it will add up to thousands. I have still not completed 😉

Adding devices like mobile, tablet, and desktop will make the number even more gigantic. Lots of combinations to develop for and lots of combinations to test for.

One will lose his mind if he go on for testing all the possible combinations of browsers, browser versions, devices, and operating systems. This gives the need of formalizing a list of browsers in order to make sure that you cover all the possible browser and operating system versions along with all the devices.

This formalized list of browsers is called as Browser Matrix or Browser Compatibility Matrix.

“When you have hundreds and thousands of devices and browsers available, you need to formulate a proper cross browser testing strategy to make sure your users get a seamless browsing experience on all devices. Based upon your audience, geo, and usability you need to formalize a proper browser matrix which covers all the browser based on their usability. This formalized list of browsers is called a ‘Browser Matrix’. It ensures that you cover all the relevant browsers and decrease the development efforts and testing efforts”.

No Number of Browsers is Enough

When a tester starts testing for cross browser testing, he can’t say that he has completed the cross browser testing of the website / web application. If he says that, I can bet he has not done enough testing. With the number of different combinations available and hundreds and thousands of functionalities available it becomes an infinite matrix to test upon. And cross browser testing is not something, which you can say that is completed and assured.

And yes, no number of browsers tested is enough.

So, how many browsers and what all browser you need to test?

How To Find Browsers To Add In Your Browser Testing Matrix

Finding the perfect browsers and devices to test upon, requires a bit of playing around with data and gathering the most relevant and used browsers by the audience. You can utilize some of the tools to make a hold of this data.

1. Google Analytics

Google analytics helps you track your user data like from where your users are entering your website, what platform are they coming from, what OS are they using the maximum. So, you can figure out the maximum used browser and OS to involve them at the highest priority in the browser matrix.

2. Data from similar sites

If you’re a newcomer, you won’t be having much data on Google Analytics to set up the browser matrix, so you can use the data for the similar sites. Like what is their main audience, where are they coming from.

3. Stats counter

One of the reliable sources(that I use personally) is stats counter. You can use it to get data according to your requirements. You can also filter out data like: location wise, area wise, OS based, browser based, time based, and much more.

This is one way for figuring out these stats however, you can also use some public data sources like: netmarket share, mixpanel trends, android dashboard, net applications, w3counter, statowl, statista, internetworldstats, etc.

Read on: Tips for choosing right browser and device to test upon.

Once you have the data source defined for your browser matrix, it’s time you figured out the data points you’ll need to cover for your browser matrix.

Required Data Points for a Perfect Browser Compatibility Matrix

When drafting a best browser compatibility matrix, you need to get certain data points around which your whole strategy will revolve.

1. Platform

Platform defines the means by which your user is accessing your website. It can be either laptop, desktop, mobile, or a tablet. You need to be sure what is the most liked platform by your audience and ensuring its compatibility is utmost important.

2. Desktop Browser Usage

If the user is using desktop for the same, better note what browser is the maximum used browser for a specific geo. Check for all combinations of mostly used browsers with different windows and mac operating systems.

3. Mobile Browser Usage

If the users love to surf on mobile, you need to ensure the mostly used browser there and again the browser, and mobile devices combinations.

4. Compare different platforms

Figure out what your users are preferring over the other and take the preferred ones into immediate attention. You must know what are the best combinations for your users in every aspect.

Building a Browser Compatibility Matrix

Once you’re done with the detailed analysis of what all the users are loving, their favorite browser, their favorite platform, their favorite food (just kidding! If you’re truly interested in that, find that too 😉 ), the time is to find out the most supported browsers for your website and create a browser matrix.

Categorize the browsers into:

A: Fully supported and mostly loved browser.

B:Fully supported and not so favorite browser.

C: Partially supported but loved browser.

D: Partially supported and not a favorite browser.

E: Unsupported but favorite browser.

F: Unsupported and non-favorite browser.

The Traffic and conversion, you need to find it using your analytics results. Once you get them, you need to rate it according to the data already present. This will help you to decide for yourself which browser you need to support first and include in your matrix. By support I imply the browser support for your website.

You can make the traffic and conversions rate liike:

And with the traffic-conversion relationship and browser support for website you can figure out what step you need to take for your website to make it support in the likely browsers and how to create a test strategy.

You can also download the template for creating a browser matrix.

With the increasing browsers and increasing need of browser compatibility testing it becomes necessary to act and work smartly. Strategizing and making use of data helps us to solve these problems to the earliest. Make sure you reduce your testing efforts while maintaining the quality.

Now with LambdaTest, You can perform cross browser testing on various mobile devices such as test on MotoG 2nd and test on Xperia Tipo 4.

Hope this might help you in that.

Happy Testing!