Video 82: Email Marketing – Using A/B Testing to Improve Results

May 24, 2011 . 5 minute read | Posted by akassover EmailInsurance Agent WebsitesVideos

Hi, it’s Aaron Kassover from AgentMethods talking about insurance websites, and talking about email marketing.  And one of the coolest things about email marketing is that you can really test your variables, really kind of figure out what’s working, and you can use this to improve your results.  I talked a bit yesterday about kinds of things you can test.  I want to talk today about something A-B testing, and I’ll explain how it works and how this can help you really ratchet up the results on your email marketing campaigns and really turn a campaign that might not be working very well, might be working so-so, into a fairly phenomenal success just by fine tuning your variables and kind of feeling out exactly what your market responds to.

So the way that it works is that you take your list of recipients that your email is going out to.  And so if we say this is our list and you segment out a small portion of this list, so we’ll say maybe it’s 20% and you split this 20% into two groups, and, of course, one group is Group A—I’ll use different colors—Group A and the other is Group B.  And then for each of these groups, you create two versions of your emails, so Group B gets version B, Group A, version A.  And these emails will have different variables, and generally, you want to just test one variable at a time so you don’t have two totally different emails, but you can have two different subject lines or you could have two different messages or bodies or you could even have two different “from” emails or senders.  And you want to go and just sort of pick one of those variables and create two versions.

Now, don’t assuming anything.  In my experience, often the things that you think will work don’t, and so this is a chance for you to try what you’d use as your subject line normally and then maybe try a wildly different idea, try something that you think might be very different and see if that has a better result than the standard one.  And then what you do is you set the metric that you’re tracking.  And so this might be open rate, it might be click-through, but you sort of decide which one you are defining as the metric you’re trying to optimize toward.  And so you send the emails out to these two groups and you give it a period of time; it might be an hour, it might be a day, and you see which one performs the best, which one has the best open rate if that’s what you’re tracking or which one has the best click-through rate if you’re tracking.  And then once the test is done, you then send the winner to the rest of your list.

So what you’ve done is you’ve figured out which of the two variables performs better for what you’re focusing on and you use that one to send the rest of your list and what happens is you get the gain between these two by doing this test.

So a few things about the metrics—first off, open rate, if you’re focusing on that, you’re probably testing the subject line or you’re testing the sender address.  And just keep in mind that a high open rate doesn’t necessarily mean a high click-through rate.  So if you’re trying to just sort of get your brand out there, get your message read, get your email seen, open rate’s going to be important.  But as you can imagine, you could have some subject lines that can have a very high open rate but maybe they don’t really correspond to what your offer is or what you’re having click-throughs on, which they might have a lower click-through rate.  Whereas if you are really kind of consistent between your subject line and your click-through, then having a higher open rate will translate to a higher click-through rate, so keep that in mind.

And the second thing is to think about how big of a group you need to send this to, and you’ll find that sometimes you’ll have two variations that are sort of looking the same and then suddenly they’ll change very quickly as you get more people to test it.  And so you want to make sure you get a statistically significant enough difference to really be valid.  And so I find that testing, if you have less than about 20 respondents for each group, you’re not going to see very valid data because everything single recipient is too significant in the overall population.

So that’s A-B testing.  It’s usually built into most email marketing tools, so you should be able to just do this right away.  It’s included in email systems and so you can sort of automate the process.  If you don’t have a system that does it, you can actually even manually do it by just segmenting out the groups yourself, creating two different campaigns, checking the results, and then creating that third campaign that takes into consideration what you learned.  Try to keep note of what works so that over time you’re getting smarter, you’re learning more about your market, your audience, and so every test you become better at optimizing your open rate, your click-through rates, and ultimately generating more leads to your website.

That’s A-B testing, that’s what I’ve got today.  I’m going to have more on email marketing.  This is a big topic, there’s a lot to talk about, so I’ll have more tomorrow.  Thanks for watching.