5 Benefits of Ecommerce Testing That Show It’s Not Just About Winning

The benefits of A/B testing, whereby two variants of a website are tested in parallel, can be huge for an ecommerce company, so it’s no surprise that A/B testing is reported to be one of the most popular methods used by marketers to optimise conversion rates.

However, testing programmes which run with no clear defining strategy, or with the restricted aim of testing to win, could be missing out on some of the biggest gains available.

At Swanky, we see it as our responsibility to recommend to our clients an approach of “testing and learning”. This means a couple of things. Firstly, that no test is viewed in isolation. Secondly, we hold the mentality that a test we learn from, even a ‘losing’ test, is far more valuable than one we don’t (such us a test which results in a draw).

For many, this might take an uncomfortable shift in perspective, with the clearer commercial value of ‘winning’ tests always likely to be the most attractive driver for many businesses to enter an ecommerce testing programme. However, there is so much more on offer if you can adopt a test and learn approach!

Still not convinced? Have a read through these five ways A/B testing delivers value beyond winning:

Benefits of a test and learn approach in ecommerce

#1 Learning from losses

The beauty of A/B testing is that no solution is final, and, in fact, it shouldn’t be –  there’s always room for innovation.

The average industry CRO test win-rate is generally agreed to be around 12-15%¹ … but what happens when you don’t get a winner? Moving on from losing tests without digging into their learnings just accounts to wasted time, particularly as we would expect the majority of tests to result in ‘losing’ outcomes.

Instead, there’s huge potential value to be found in investigating further. By diving into the data and understanding user behaviour in the testing period, we always look to recommend iterative improvements to our clients’ tests.

Whilst this sometimes means knowing when to draw the line, in some cases swapping out a link or piece of copy and re-testing can make all the difference in the success of the experiment variant.

Additionally, it might be that we need to rethink how we solve the initial pain point identified. With running a first iteration, even if this loses it tells us what didn’t work and gets us closer to that final goal – it doesn’t mean abandoning hope on resolving a key pain point.

Taking a controlled, iterative approach to ecommerce testing sees our clients’ win-rates increase steadily over time. In fact, one of the brands we provide ecommerce growth services for finished 2021 with an impressive average win-rate of 60%, well above the 12-15% industry standard!

#2 Refining learning through segmentation

Too often, a test to win approach can lead ecommerce testing programmes down the route of solely chasing for ‘big wins’. However, looking deeper into segment analysis can provide some of the most valuable insights on how to successfully target select groups of users.

It might be that iterative recommendations are different from one segment to another. For example, perhaps the original test worked really well for your desktop users but this didn’t transfer to mobile device users, or vice versa.

By running tests, we take a magnifying glass to these users and in some cases pick out more problems or blockers which potentially need to be resolved before we continue with an iteration.

The more you learn about your customers, the more granular your segmentation will become, and the more laser-focused your targeting can be.

#3 Personalising customer experiences

Talking of targeting, we know now more than ever that customer groups are more and more specific about their requirements and looking for deeper connections with brands through personalised ecommerce experiences.

An unsung benefit of A/B testing is the ability to reach out and personalise the digital experience for specific users. This could be through tests that experiment with specific landing pages for paid campaigns, or by targeting users with personalised homepage content to make their return to site a seamless experience.

You can test ways to present value to refined segments of users, using your insights to build up a more loyal and valuable segment –  your VIPs.

#4 Validating larger opportunities

In some instances we’ll see a lot of data indicating a pain point on site, however the complete solution to a problem isn’t always clear, or there are multiple opportunities which would require heavy investment and we need to prioritise a specific opportunity first.

In this case, we consider simpler tests we can run to help us validate a larger opportunity.

For example, it might be that users are struggling to find the right product, but we don’t know yet how users want to shop because there is a lot of struggle on the site. Here, we could run a test which gives users options to filter through products in various ways, such as a quiz or filters on the collection page. Based on which filtering options are more successful in the initial test, we can then look to deploy a more purposeful route through the site which helps users find the right information to match their needs.

A Swanky example

The Growth Accelerator Team here at Swanky have recently used this sort of test and learn approach to validate a larger opportunity in our conversion rate optimisation (CRO) engagement with Loop Earplugs.

Our detailed analysis showed us that the website’s situational landing pages were underperforming in terms of conversion rate – they weren’t optimised and didn’t meet the needs of users in their product discovery phase. We therefore identified a clear opportunity to help users find the products they need quickly and easily.

We started simple with our first on-site test, experimenting with the placement of product recommendations on these situational landing pages. The variant, which elevated the profile of the product recommendation so that it was above the fold, generated a much higher conversion rate to transaction than the control (which had minimal information above the fold). We believe this was because users were presented with visible and relevant information, and felt confident to make a purchase directly from the landing page. 

Our variant in the first A/B test for Loop Earplugs.

The insights and confidence that we gained from this initial test provided a foundation for a series of successive tests on the same theme, validating a larger opportunity around the promotion of use cases at high-exposure points on the website.

We went on to develop a multifaceted testing campaign that addressed other touch points in the funnel, experimenting with a product finder quiz, the ability to shop by use case in the site’s navigation, and the promotion of product use cases on product pages.

Find out more about our CRO campaign with Loop (which created a €1M growth opportunity!) in this case study.

#5 Generating learnings for internationalisation

Last but not least, another opportunity to think about is the international potential of A/B testing. Here there are several different benefits, but most prominently, testing allows you to trial a change to your site on just one store before rolling it out across all markets.

The ease at which you can swap out and change experiments within a testing platform also allows you to run concurrent tests and to different traffic sources. This is helpful in situations where traffic is low and testing significance is harder to reach, allowing you to combine learnings from other markets and build a better picture of the success of the test. For ecommerce stores with larger international markets, more defined region-specific testing strategies can then be deployed.

Plus, A/B testing allows you to learn more about new markets which your store enters, as opposed to merely making assumptions about how different markets behave.

You can read more about adopting a test and learn mindset in foreign markets in Swanky’s free guide to ecommerce internationalisation.

Summary

Even with the best intentions, any new-to-launch or existing site can’t expect to optimise successfully unless they look to learn from and adapt to their audiences. With a test and learn approach to ecommerce testing, you can identify pain points and create experimentation programmes which seek to resolve these problems in meaningful ways, overall resulting in better user experience and better performance of your online sales channels.

The opportunities opened up by testing are pretty much endless, but hopefully the five benefits outlined above showcase the significance of prioritising a data-led approach to ecommerce testing.

Award-winning conversion rate optimisation agency

As the UK’s CRO Agency of the Year (UK Digital Growth Awards 2021), we know what it takes to deliver a successful ecommerce testing programme that drives business growth. We combine our extensive experience providing digital consultancy to established market leaders with detailed site analysis, technical innovation and the latest technology to boost the metrics that matter. And, unlike most CRO specialists, we conduct all campaign consultation, analysis, development and testing in-house!

So, contact us today to find out how we could unlock your brand’s potential with ecommerce testing and other conversion rate optimisation services.

For reference:

[1] https://www.invespcro.com/blog/ab-testing-questions-answered/