I’m often extolling the virtues of AB split testing in email marketing. This is especially true of one my favourite spilt tests, subject lines.
My love of split testing subject lines is twofold. Firstly, it’s fast. Very fast. Adding a split test is the last action I take before sending an email and it takes me about 5 seconds. (If this comes as a surprise, you’ll want to check out Campaignmaster’s AB split testing for yourself).
Secondly, it can result in some huge engagement boosts, and I’m talking really big. Using AB split testing I recently got a 113% uplift in clicks. Not bad for an extra 5 seconds work huh? Let’s see how it was done.
If you’re subscribed to our mailing list, you’ll have noticed our new Weekly Roundup email (If not, check it out!). In the most recent edition of the Weekly Roundup, we shared some of our blog posts including the Canadian Anti-Spam Legislation & Email marketing essentials.
I wanted to highlight both of these posts by adding them to the subject line, but wasn’t sure which one would work best. The perfect opportunity for an AB split test had arisen all by itself (once you get into it, split test subject lines tend to suggest themselves, but there are some good rules of thumb to be aware of)
These are the two subject lines I came up with, and the pros and cons I had of each:
A: Are you prepared for CASL?
Pro – In the form of a question, creates intrigue, short
Con – Acronym does not make sense to the those unaware of CASL
B: The email marketing essentials
Pro – Makes sense to anyone, short
Con – More generic, no enticing question mark
I split the subject lines over a test sample of 4,800 recipients (2,400 each) and waited an hour for the results. We usually see our highest levels of engagement within this window, so it’s the perfect amount of time to wait before declaring a winner. You may want to check your previous engagement levels over time when deciding how long to wait when performing a split test.
A: 17% clicks
B: 8% clicks
As you can see from the results above, subject line A had a huge uplift in clicks. Its short subject line, phrasing & relevant topic had elicited more engagement than it’s more generic counterpart.
What makes this more interesting is the fact both emails had identical open rates. This means that subject line A had ‘framed’ subscribers thought processes into locating and reading the article about CASL after having sub-consciously answered the question ‘Are you prepared for CASL?’.
As this test was sent out to a large sample size of 4,800 and achieved a high uplift, there’s a 99% probability of statistical relevance.
Hopefully after seeing the results above you’ll be raring to jump on the split testing bandwagon! It really is as simple as spending those few extra seconds adding your subject lines and watching those results roll in.
Then, if your results allow you declare a victor, you can send a markedly more successful email out to the remainder of your list.
If you’d like to see for yourself how quick and easy it is to create a split test using Campaignmaster, we’d love to show you around. Get a free tour with one of our consultants today.