Since its release in late 2007, MailChimp’s A/B testing tool has been a huge success with our customers. In fact, over 200,000 split tests were sent in 2013 alone. Using subscribers as lab rats is still both fun and educational, but we realize that some of you might not know how to get started with the tool. To help you out, we put together a quick refresher on getting the most out of A/B testing. We’ll also share some of our research on the best ways to test.
What is an A/B test?
A/B testing allows you to send two different versions of your campaign to part of your list, and then select the version that gets the most opens or clicks to send to the rest of your list. MailChimp can run the tests automatically for you, so all you have to do is tell us what to test and what percentage of your list you want to test it on. By changing aspects of the campaign between the two different groups, you can find out what your subscribers respond to. What can you test this way?
Subject line – Try different wordings to see what gets the most attention
From name – See if your subscribers prefer mail from a person or an organization
Delivery date/time – Figure out when your subscribers like to open/click
Content – Test different images, layouts, and messages
We recommend testing only one difference between the A and B groups. When there are several differences between test groups, it’s difficult to figure out which change impacted your engagement. If you make only one change, you’ll know exactly what caused the difference.
What kind of results can I expect?
We examined A/B tests sent in the past year to measure their impact. For subject lines and from names, we calculated a Levenshtein Distance …read more
Source: Mail Chimp