Getting A/B Testing Right: Lessons from Namogoo’s Work with Major Online Retailers
September 11, 2019
by Adam Segal
If you’ve ever worked in marketing, sales, or customer retention at an eCommerce company, then you know the importance of optimizing websites, emails, and just about every other part of the customer experience. You also know that this optimization is an ongoing process – and that the last thing you want this process to become is guesswork.
So there’s a good chance that you’re familiar with the idea of A/B testing, a versatile tool that can add a degree of certainty to a wide variety of decision-making processes. But while the basic idea of A/B testing may sound simple, ensuring its reliability requires careful planning and data analysis. Even successful eCommerce companies can be vulnerable to common pitfalls.
To help online retailers ensure that they are carrying out A/B tests as effectively and accurately as possible, we at Namogoo recently published a guide to the best testing practices. You can now download A/B Testing Best Practices Learned from 100s of Tests on Large eCommerce Websites for free right here.
In it, you will learn:
- What an A/A test is, when to run one, and why it’s an important step in order to test the reliability of your A/B tests.
- How a representative control group can help you to prevent both false positives and false negatives (and whether it’s even more effective to use two control groups).
- Effective ways to keep your data from getting contaminated by confounding variables (and how to know when you need to throw out your data and repeat a test).
- Why bots are a threat to your A/B tests and how to make sure you’re measuring the behaviors of real people.
Insights from a wide variety of eCommerce companies
The lessons in this guide have been fine-tuned over the course of our work with major online retailers.
That’s because A/B testing is the key method we use for measuring the impact of our Customer HIjacking Prevention (CHP) solution on these companies’ eCommerce KPIs. They turn to our solution in order to stop ad injections caused by malware and WiFi hijacking from stealing their customers away to competing online stores, hurting their brand reputations, and even charging them affiliate fees for their own web traffic. With injected ads appearing during 15-25% of all online shopping sessions, these retailers realize the risk such ads pose to their bottom line.
In order to ensure that our solution is successfully achieving our customers’ goals, we constantly run A/B tests. It is through this type of testing that we have been able to quantify the success of our CHP solution. While each of these companies has its own story to tell, we have also discovered some important figures across the board, including:
- Retailers that work with Namogoo see their conversion rates among “infected” users (those consumers using digital devices that are impacted by ad injections) rise an average of 12%.
- These companies consistently achieve an overall conversion rate increase of 2-5%.
- These companies see their revenue per visitor jump by 5-7%.
- After starting to work with Namogoo, these companies see a 90% reduction in the online revenue being lost to ad injections.
- To date, our CHP solution has enabled these companies to recover a total of $650 million that would otherwise be lost to ad injections.
Over the course of this testing, our Customer Solutions Consulting (CSC) team – the group that I lead in overseeing the tests – has learned a great deal from that experience. While some of these lessons are specific to testing for our CHP solution, many of them offer useful insights to a wide variety of eCommerce companies.
That realization is the reason I sat down to write this guide enabling other businesses to benefit from Namogoo’s A/B testing experience. Although we at Namogoo are focused on helping major retailers improve their eCommerce performance through Customer Hijacking Prevention, the reality is that we have also developed other areas of expertise that can likewise help these companies boost their online KPIs.
I sincerely hope that by sharing this information with your company, we can help you to get more out of your A/B testing.
To benefit from my team’s experience, download this free guide to A/B Testing Best Practices Learned from 100s of Tests on Large eCommerce Websites.