What is A/B Testing? (And How To Win at It)

If you want to grow your Shopify or Woocommerce stores, you really need to understand what is A/B testing.

Remember the experiment you did at school by growing runner beans in jam jars to see how different elements i.e., water, light, and temperature affect the growth of a plant?

A/B testing is quite similar, but more precise – you test how two variants affect the performance of a webpage or a marketing campaign.

Known as one of the simplest and most effective ways for conversion rate optimization, A/B testing is a method of gaining an insight into your audience’s behavior and preferences, which then helps in improving the conversion rate.

Simply put, it has taken the guesswork out of digital marketing by enabling digital marketers and business owners to take data-backed decisions. What makes this methodology even more useful is that it can be applied to a number of variables and can be used across a range of communication channels.

Although the technique has been around for some time, it is not as commonly used as many other digital marketing tools. But, this doesn’t reduce its significance.

The importance of A/B testing can be determined from the fact that it has now become the standard method used by tech giants (including the ones in Silicon Valley) for improving online performance. Former US President Barack Obama used this very technique while raising funds for his 2012 election campaign and guess how much he collected? $600 million!

This is the power of A/B testing!

How Does A/B Testing Work?

A/B testing is the process that is used to identify how small changes in marketing campaign influence audience behavior. This is done by creating two versions of a webpage, an email of any other marketing content that only have one varying element. It could be a headline, CTA, subject line of an email, a slogan on an advertising banner, or the layout of a webpage.

The test is conducted within a controlled group of audience and the purpose is to determine which version receives a greater response and is more successful among the public.

It can also be conducted multiple times to fine-tune the content and improve its effectiveness until you achieve your desired results.

Let’s say, you want to test the headline, so you create two separate pages with only the headline changed; the rest of the content is going to be the same. Now you test run both the variations by presenting each to half of your visitors for a certain period of time. At the end of the testing period, you collect some specific metrics, such as the average time spent on the page or conversion rate, to determine which version of the page performed better.

To help you better understand the process, let’s take a look at what A/B testing looks like:

Things to Consider for Successful A/B Testing

For those who have never done it, here are a few things you should remember to make sure it serves the purpose:

• The first thing you should do when you decide to conduct the test is to analyze all the metrics of your business and figure out the ones that you think are the most crucial and define the success of digital marketing campaigns. Depending on the nature of your business and the goals of a campaign, these could be one or more of the following:

1. Click-through rate
2. Leads/ sign-ups
3. Downloads
4. Number of sales

• While these are the most valuable metrics to measure the success of a digital marketing campaign, experts also recommend tracking the bounce rate, exit rate, and engagement metrics to get a deeper insight into audience behavior, which will help you improve in the long run.
• When setting up the marketing campaign, it is important to remember that you should only make one variation at a time. It’s common for people to feel tempted to make multiple variations, but that will kill the purpose of the testing process as you will not be able to figure out which factor (variation) actually made the difference.
• Testing both the versions simultaneously, in identical circumstances, and among the same number of people are the key factors in getting accurate results. While this can be easily done when testing a webpage, it requires a good amount of homework when you are testing a newsletter or an email. In this situation, you have more control over who’s going to receive your content. Therefore, make sure the test groups you prepare are as similar as possible, if not identical. For that, you not only need to have an equal number of people in each group, but, ideally, you should also have an equal number of men and women in each. Depending on how detailed information you have, you can also consider the geographic locations, age group and other factors that you think may affect people’s behavior and choices.
• When running the email test, you should only do it on a small group, let’s say 10% of your leads, and make sure the emails are sent simultaneously in order to reduce (if not eliminate) the chances of variance.
• It is also important to decide the duration of the test and the number of responses you will need to quantify the results, before you begin. Running the test for a too short or too long period increases the probability of other factors impacting the results.

What Forms of Marketing Communications Can You Use A/B Testing for?

While A/B testing can be used across all digital marketing channels, it particularly provides useful results when applied to the following forms of communication:

• Websites
• Email marketing campaigns
• E-newsletters
• Digital advertising, such as AdWords, PPC, and banners
• Applications

What Are the Best Elements for A/B Test?

It is common knowledge that not all elements of a marketing asset make equal contributions to conversions; some contribute more than others. While you can run an A/B test for every element on a website, newsletter, email, or any marketing material, it is wise to do it for elements that are known to make the most impact. To give you an idea, let’s take a look at some of the most effective and impactful A/B testing elements:

• Headlines

The headline is the first thing visitors see when they land on a webpage and if it fails to grab their attention, they are not going to stick around – it’s as simple as that!

The advertising expert David Ogilvy had advised the writers on the importance of headlines long before the internet revolution. This is what he said:

“On average, five times as many people read the headlines as read the body copy. When you have written your headline, you have spent eighty cents out of your dollar”.

A headline that doesn’t hit the right chord could cause you to lose a potential customer. Therefore, it is important to come up with different variations and test them to improve the performance of your website.

• CTAs

Unless you have been living under the rock, you would know about the importance and power of CTAs in converting visitors into leads and/or customers. A powerful call-to-action is one that clearly tells visitors what you want them to do next, but in a way that it becomes difficult for the visitors to ignore it. It has to be enticing and offer value that visitors won’t be able to resist.

Many times, just changing one word can significantly improve the performance of a CTA. Apart from the text, the shape, size, text color, button color, and contrast are some other elements that have a significant impact on the performance of a call-to-action. However, remember to only change one element at a time or you won’t be able to identify which factor made the difference.

• Visuals

Images have the power to instantly grab the attention of visitors. A webpage or a marketing material that has high-quality and powerful images is highly likely to entice people to read and find out more about the business or offer.

In addition to images, infographics and videos are also known to contribute significantly in attracting and engaging the audience.

In today’s digital marketing world, where videos have been established as one of the most powerful forms of content, you can test how it affects the performance of your website. A great way to begin is by using testimonials. Create two versions of a webpage, one with written testimonials while the other with video testimonials and see how it improves your traffic and/or engagement rate. In case, you didn’t know, videos are good for SEO too.

• Subject Lines

Again, it is common knowledge that the subject lines of emails have a huge impact on the click-through rates. Therefore, A/B testing them is a great way to increase the chances of the open rates. Try out statements, questions, power words and perhaps, emojis too.

The Final Word

Gone are the days when business owners believed they couldn’t do much about improving the conversion rates of their websites. In today’s highly advanced digital marketplace, there are multiple ways you can make an impact on your audience and convince them to do what you want. A/B testing is one way of doing that.

By helping you determine the behavior of different segments of your audience, the technique helps you streamline and improve your marketing campaigns, so you can get desired results. Although it isn’t as widely used as some other techniques, it is worth your time and effort. There’s a reason why all the tech giants do it (as discussed above). Try it out to see the results yourself and do let us know how it benefited your business!

Next Steps...

1 See which tools I use & recommend to clients:

2 Get My FREE Funnel

3 Book your FREE 30 Min strategy call: ​