Split Test Like it Matters: Tips to Drive Revenues with CRO

Let’s examine how a scientific approach to split testing increased conversion rate by nearly 7% for one company, producing $650,000 incremental revenue per year.

While the concept of testing may seem easy, we all know that it takes a lot of preparation and strategy to create a test that produces valid results you can rely on to increase your revenue. We’ve gathered some tips to help you produce valid test results every time.

This post will focus on the first tip: isolating one variable at a time to show what actually moves the needle.

Tip 1: Test one element at a time. Tweet

If you change too many elements at once in a test, you won’t know whether, say, a bigger headline or a different button placement made the difference to your conversion rate. So it’s important to isolate each element you want to test. Let’s look at an actual experiment to better understand how this works.

We split-tested a Control (the original cart template) against 3 variations. Each variation had one hypothesis and one element that was different from the previous template.

The Original Cart Page: Our Control

The Control design featured a highlighted box for entering a coupon code, and a two-column layout for Billing and Payment.

Variation A: Layout of Billing & Payment, Coupon Field

Variation A decreased the emphasis on the coupon code, the hypothesis being that shoppers may be less inclined to abandon the cart to search for a code. Variation A also changed the Billing and Payment section to a one-column layout.

Variation B: Call-to-Action

Variation B was the same as Variation A, and changed the call-to-action to Continue instead of Place Order. The shopper was being directed to a Review page before placing their order, and the hypothesis was that this CTA would remove hesitation and increase clicks. This one change delivered a 1.4% lift, which equals $133,000 revenue per year.

Variation C: Checkout Button

Variation C was the same as Variation B, and it added a “Check Out” button to help users jump down the page to the Billing & Payment section. The hypothesis was that more shoppers would complete the fields if they did not have to scroll. This one change produced a 2.5% lift, which equals $237,500 in revenue per year.

Summary of Results 

 Variation C was the winning template, beating the Control by 6.8%. For this software vendor, it means an extra $646,000 in revenue per year.

The key takeaway – by testing only one element in each variation, we isolated the affect of the change to see what really moved the needle.  For example, Variation C proved that adding a simple Checkout button produces $237,500 in annual revenue! Tweet

To see more examples of how split-testing can increase your revenue, download our webinar on “Top Ten Myths of Conversion Optimization” and share your own insights with us on social media, using the hashtag #CROMyths.