Case Study: Using A/B Testing to Increase Average Order Value by 32%

The challenge: Rising acquisition costs

Even before the pandemic hit and increased the number of brands competing online for customer attention, acquisition costs were on the rise.

In a bid to adapt to the challenges of today’s marketing landscape, brands are taking steps to improve the performance of their ecommerce stores and make the most out of the customers they acquire (you can read more about this in our comprehensive guide on how to reduce customer acquisition costs).

Faced with increasing CPA (cost per acquisition) across their marketing channels, our client’s primary concern was ensuring that the customers they were acquiring would spend more.

With their primary storefront based in Australia, and having recently launched a presence in New Zealand and the United Kingdom, increasing average order value (AOV) would afford them the budget to accelerate their marketing in these new markets.

Following a referral from Shopify Plus, we collaborated with our client to set goals, strategise on how to achieve them, and test the Australian market before iterating our findings across all three storefronts.

How Swanky helped to increase average order value

Gathering data insights

First thing’s first, how much are customer spending?

By exporting all of the business’ orders over a 30-day period (January 2022), Swanky’s data team were able to segment customers into order value-based cohorts. Doing so allowed us to see that only 29% of users were spending under the required shipping threshold ($30AUD) to qualify for free next-day delivery, whilst 39% of orders fell in the next cohort, spending between $30 and $50.

From this insight, Swanky noticed confident buying behaviour within a single order value, where a large percentage of users were happy spending up to $20 over the free next-day delivery threshold.

With these findings in mind, it was quickly agreed that there was potential to rapidly increase the brand’s AOV by slightly increasing the free next-day delivery threshold.

Test 1: How do we inform customers about the threshold?

Our first test was to determine how our client could better communicate its threshold to consumers.

With Google Optimise configured, Swanky proposed a site-wide A/B test where 50% of traffic on all devices were shown a banner with the following message: “Spend $X and receive free next-day delivery”.

Whilst we didn’t notice an increase in AOV, we did see a 3.11% uplift in conversion rate over a 14-day period. We attributed this to the increased emphasis on the free shipping threshold – making it more prominent and visible to a more significant percentage of the brand’s user base immediately upon landing.

This result validated the use of a promotional banner to directly inform users of an order value threshold. It also highlights that ecommerce testing isn’t just about “winning”. There’s lots to be learnt about user behaviour, even from tests that don’t drive an increase in your target metric (AOV, in this case).

Test 2: Communicating distance to threshold

Following this first test, we proposed a dynamic sitewide banner that would inform users how much more they would need to spend to reach the threshold (i.e. “Spend $10 more and receive free next day delivery”).

At the same time, we also increased the shipping threshold by $20 to better suit the highest value cohort who were spending $30-50.

We hypothesised that dynamically informing customers siteweide of their cart value and whether they have qualified for the free shipping threshold of $50 would increase the AOV of customers. This would also reduce the number of customers whose order value totalled under $30 (approx 29% of users in January 2022). Why? Because the banner messaging would be communicating the value of adding additional items to a user’s cart to save on freight costs.

After our 14-day test period concluded, we noted the following metrics on the theme that included the dynamic banner:

• Mobile (75% experimented sessions): 10.16% uplift in AOV.
• Desktop & tablet (25% experimented sessions): 9.64% uplift in AOV.

The AOV increase seen without a significant sale running is a positive indicator that customers are comfortable spending $50 or more in one transaction. On top of this, the increase in revenue when comparing the two periods (pre and post-test) supports our hypothesis.

An updated cohort analysis using the following month’s orders revealed that out of 4076 transactions, orders below $30 now accounted for 22.5% of total orders – a decrease of 22.41%.

We also noted that customers spending between $50 and $70 now stood at 30% of all orders, compared to January’s 15%, representing an uplift of 100% for this cohort.

Why stop there?

We’d also identified multiple other areas in which our client could improve their AOV – one being the use of dynamic post-purchase upsells using Zipify One-Click-Upsell.

Exciting product releases that were easy add-ons to current orders seemed like a “no-brainer”. So, having evaluated popular products via Google Analytics product sell-through reporting, we configured particular triggers to promote specific products in post-purchase upsell offerings, only appearing in the thank-you page after an order is already placed. Doing so mitigates the risk of a pre-purchase pop up causing customers to become disgruntled and bounce off the site, but still achieves a significant increase in AOV and therefore, per session value.

By A/B testing specific add-ons and copy in an attempt to increase the conversion rate of our upsell offers, Swanky achieved a 75% test success rate in proposed post-purchase upsell tests. Through experimenting with our offer and language we managed to take our upsell adoption rate from 3% (meaning 3 out of every 100 orders added an extra item) to 5.8%.

Overall impact: an increase in average order value

Since working with Swanky, this client’s AOV has increased by 32%. This uplift will aid in their expansion into international markets, bolstering their marketing budgets in what is a challenging time for ecommerce marketers.

Your Shopify Plus experts in Australia

Our Growth Accelerator team are passionate about combining creativity and innovation with data-driven insights and industry-leading technology to unlock our clients’ potential. We drive growth for ambitious, fast-growth brands through ecommerce testing and other commercially-led conversion rate optimisation services – all of which is done in-house.

Get in touch with our team of certified Shopify Plus Experts in Australia to find out more.