The Ultimate “A/B Test” Guide for your Ecommerce Site Search

A/B testing your site search is really important as it is one of the most important element of any eCommerce Store.

Many sites that fail at helping visitors find the right product fail at selling them.

When it comes to website development and creating user experiences, ecommerce businesses often place aesthetics before speed, clarity, and ease of use.

But, this is often not the right approach because the most important factor that determines user experience is how quickly and easily a consumer can find their desired products on your website.

When I talk about finding the desired product, I am not referring to site navigation. While having great navigation is important, you cannot just rely on it to provide a great user experience and website usability.

You also need to have a good site search.

According to research, about 50% of digital shoppers prefer to use a site search to find their desired products on a website. 

And you wouldn’t want to put off or frustrate such a huge number of your potential buyers by not having an internal search function on your ecommerce store, would you?

Why Should You A/B Test the Site Search?

Most marketers simply put a search form in their header and forget it, thinking it is enough to satisfy their visitors.

It's often not.

You should A/B test elements of it, which then can lead to more conversions.

When optimized, your internal site search makes it easier for users to find what they are looking for, which then increases the chances that they would make a purchase.

But, here is the catch – to do this, you first need to understand user behavior.

Understanding how your prospective buyers tend to behave online and what they expect from a site’s search function will help you fine-tune the search element in a much better way.

Since every business attracts a unique set of audience, your potential buyers may act a bit differently or have unique requirements and expectations. But on average, digital buyers have one thing in common – they want quick and relevant results.

Like it or not, Google has spoiled us. Being used to searching on Google, we all have become accustomed to the great search experience that the platform offers and we expect every search function to work the same way. We not only expect quick results, but also expect the site search to understand what we are looking for even if we do not enter the exact term for or name of a product, just like Google does. And when this does not happen, we get frustrated.

Unfortunately, many ecommerce businesses do not yet understand the importance of on-site search. As a result, site searches often do not perform as well as the users had expected, which then makes businesses lose customers and money.

But, the problem can easily be resolved with A/B testing.

While A/B testing is not the only available method for improving the search function on an ecommerce store, it is definitely the most versatile and valuable. It is also the best way to determine the impact of a change on your website performance.

Just like with all other elements, A/B testing allows ecommerce businesses to continuously improve and optimize their internal search to make sure it caters to their users’ requirements and expectations.

But, since on-site search is still undervalued in the ecommerce industry, not many businesses are utilizing A/B testing to improve it.

In a study conducted by Econsultancy, only 32% of the respondents said their companies test the search functionality of their websites.

What to Measure when A/B Testing your Site Search?

There is no one way to perform A/B testing for site search or for any element per se. It is a long and continuous process that involves testing various features and how they impact on click-through rate, conversion rate, sales, and other metrics that an ecommerce business use to measure success.

However, an important thing to remember here is that you may not experience a significant increase in average order value or items per order because users who run site searches generally have specific requirements.

Here are some success metrics for site search; you don’t necessarily have to measure them all, just figure out which ones are important for your business and hence, need to be improved:

• Add to Cart rate
• Conversion rate
• Average Revenue per visitor

Secondary KPIs:

• Product page click-through rate – this indicates the relevance of search results and the usability of filters.
• Search results exit rate
• Engagement with sorting and filtering tools
• Engagement with pagination links
• Engagement with auto-suggested terms and products

Tips and Ideas for A/B Testing the Search Function of Your Ecommerce Store

For those who are new to it, here are some tips and ideas for A/B testing on-site search:

The Size, Style, and Placement of the Search Bar

In order to search for their desired product(s), your visitors must first be able to locate the search bar on your ecommerce store.

To ensure greater visibility, test different sizes, styles, and positions of the search box on the website.

For example, you can create a variant of the webpage with a slightly bigger search bar and test it against your existing version. Similarly, you can test a sticky search bar versus a non-sticky one, expandable search bar versus the non-expandable one, and a search box placed on the top corner of the webpage (like you find on most websites) against a center position.

One way to find if you should test the placement of the search bar is to look at heatmaps or do user testing. Try to find out how your users are interacting with the element and if a lot people use it, test making it larger and more prominent.

I have clients where we've literally removed everything on the home page to make place for a search form (like Google) and that won by a lot.

It's just a matter of giving your users what they are looking for already.

Search Field Copy

In the above example, Wallmart are limiting the user's interactions by using a bland term like “search”.

You'll typically want to guide your visitors in a better way by specifying what they can search or even suggesting popular products. In the Best Buy example further above, they are suggesting searching by keywords or SKUs.

That's ok, but you should test suggesting your most popular sections or products and using these as copy within the field. Like: Search products, like umbrellas or raincoats.

Search Icon vs. CTA

The use of an magnifying glass icon helps save space and is widely known by now.

However, the appeal of using a copy based CTA is that you can use it to add a benefit or future-pace your visitors.

Something like “Search Our Best Flower Arrangements” is much more powerful than an icon. But obviously much more suited for a home page search, than a header search because of available space.

And, as always, what works for other ecommerce stores doesn’t necessarily mean it would also work for you. And this is exactly why we need A/B testing.

In addition to testing the search bar with the search icon versus the search bar with a relevant CTA, you can also test different CTAs.

‘Search’ and ‘find’ are the most commonly used CTAs for search bars, but you should try a different one. Make sure it is simple and clear; don’t use ambiguous words by assuming that your visitors would understand them.

To make it more attractive, you can also choose a bright and contrasting color for the CTA. Always revert back to the principle that your primary CTA should get the color that pops the most. If that happens to be search, then that's ok. But if you are asking visitors to buy now on the page, then the search button color should be secondary in both size and color compared to the primary objective CTA.

The Number of Results

The reason why online buyers use search function is that they want to narrow down their options and this is exactly what you should do for them. Search results with endless options and/or too many pages are generally a turn-off for potential buyers.

Think of it this way – why would they have used the search function had they been willing to browse through countless results?

Run a few quick searches on your website to see the number of results it is showing on average, and then create a variant with a lesser number of results.

Test both the versions and you are likely to get a better response on the one with lesser (most relevant) results.

Make it simple for them.

One trick is to the most popular products at the top. These are most likely the ones your users want anyway, and this helps funnel them through easily.

Grid or List? How Results Are Displayed

Is it better to display search results in grid format or as a list? Does the display format of search results even make any difference in user behavior?

In my experience it really does. but you won't know until you test it.

Grids provide more results to users, but limit the information your can provide on each products. Both options can be attractive, and testing it will let you know which is preferred by your visitors.

While you can include both the display option on the search results interface, it is not likely to make a noticeable difference because most users stick to the default display.

Therefore, it is better to test both display formats to figure which one you should opt for or choose as default.

The Relevance of Results

There are no two views about the fact that the results have to be relevant to the searched term.

To further improve this feature, you can rank or display results in order from the most relevant to less relevant ones and test it against your existing format to see if it brings any changes in the buyer’s journey or improve conversion rates.

You can also test adding certain filters (specific to your business) to allow consumers to further narrow down their options according to their preferences.

Auto-complete

Let’s take Google’s example once again. Isn’t it amazing how the search engine always shows several options related to your search as soon as you begin to type something in the search bar?

Okay, I understand it can be a bit annoying sometimes, but it is actually useful in many cases, such as in instances when you can’t exactly recall something and try to look it up on the search engine.

What makes it even more brilliant is the fact that the search engine can even pick up wrong spellings and also provide spelling suggestions along with the search results allowing you to further refine your search, if needed.

This is how your on-site search should be!

But, before you actually bring these changes, A/B test them to make sure they work with your audience and are worth implementing.

No Results vs. Suggestions

No matter how well you have optimized the search function of your ecommerce store, there will always be searches with no accurate or highly relevant matches. For example, if you sell sweaters, it is easy for people to assume that you would have them in all styles and you can get searches for ‘over-sized sweaters’ which you don’t sell.

In such cases, you have two options. One, your search result can show ‘No Results’ and two, you can provide uses some suggestions on the ‘No Results’ page.

The second option has been found to work better for most businesses, so it is worth testing. With regards to suggestions, however, you can try a few different things, such as displaying your best sellers, items on sale, or the latest additions.

Showing just the ‘No Results’ page without any suggestions is like putting a stop to further navigation and can bring the session to an end.

Example of Site Search A/B Testing

Here is a successful example where ecommerce businesses improved their website performance by optimizing on-site search through A/B testing:

Pinterland

Pinterland, a company that operates in the SMB market, experienced a four times increase in the conversion rate of users who make use of site search by incorporating the auto-complete feature with image thumbnails.

The small change in their site function also resulted in increasing the per visit value of site search users by six times.

Maximize the Performance of Your Site Search through A/B Testing

An optimized search function is one of the integral elements of a well-performing website.

But, optimizing and fine-tuning your on-site search is not a one-time task. Rather, it is a continuous process.

Regular testing of your site search ensures that you continue to meet the expectations of your potential customers, which then causes an increase in the per-visit value, conversion rate, and per customer value or revenue in the long run.

So, keep on A/B testing and don’t hesitate to contact me if you have any questions or confusion!

Next Steps...

1 See which tools I use & recommend to clients:

2 Get My FREE Funnel

3 Book your FREE 30 Min strategy call: ​