Monday, September 14, 2009

Metrics, Schmetrics

Every once in awhile I find myself embroiled in some sort of vicious discussion about how to effectively measure the impact of email marketing. Rather than comment on the type of people (myself included) who would care enough about the subject to raise discussion to the level of argument, I thought I would throw it open for discussion here and see what people are doing to measure the success of their email efforts.

Here's our problem...people's actions are hard to measure. Yet our business is built on "High ROI."So we have a lot of people (mostly vendors) who tell us that measurement is now scientific as well as a lot of executives who ask for what the vendors have told them is readily available.

There are three distinct audiences that we need to measure. They go from easy to hard in three simple steps.

Group 1 - The Straight Liners - These are the folks that get an email, open it, click on it, then purchase something directly mentioned in the afore-mentioned email. We love these people, because they behave exactly like we want them to. Actually, they behave exactly how our measurement systems want them to behave. More often than not, this is the focus of reports we send to senior management...

Group 2 - The Crooked Liners - These are folks that behave in a non-linear fashion. They might open the email, they might not. They tend to click, but the trip to the shopping basket is either long or non-existent. They might do nasty things like clear their cache on a regular basis. They'll shop, then come back later and buy something that may or may not be what they looked at before. They may look online, then head into the nearest retail shop to actually purchase something. In essence, they're selfish because they do what they want, not what we want them to do.

Group 3 - The Flat Liners - These people rarely, if ever, interact with your email. They don't open, they don't click, they don't buy from the prescribed path we've determined they should walk down. Therefore, they must be dead. Any smart marketer would of course delete these people from their list because hey - if we can't measure them they must not be important. Better to save the $.0007 it costs to send them an email.

Almost ALL of our efforts as an industry are focused on Group #1. Which is all well and good. Problem is that it SIGNIFICANTLY undervalues your email program, especially if you have alternative methods of gathering customers (like your web site...)

When I was at another company who shall remain nameless, the best we could come up with for Group 1 was about $70MM in sales. Which wasn't bad, but really small when compared with overall sales.

What we did was look historically at all of our sales and divided them into "email supported" and "non email supported" (than goodness for non-cooperative marketers.) When looking at the total pool, the amount of sales attributable to email (I'd have to consult with you to tell you how we did it) rose to about $216 MM in sales - over three times the "straight line" method.

But what was really interesting is when we went back and looked at people who were on the database but hadn't ever responded to an email -yet who have never opted out to receiving email. Turns out they purchased over $70MM worth of goods during the measurement period - the same as The Straight Liners. Not bad for a bunch of "dead" people. We couldn't map that all back to email, but we figured why take the risk? So we happily spent the $.0007 to keep sending emails to them. Epsilon published a stat that about 1/3 of your recipients never read your email, but instead head straight for your web site. Looks like that number's about right!

But here's the real insight - each of these groups (at least for the company who shall remain nameless) is roughly the SAME size. That's pretty mind-boggling stuff. It may not be true for your organization, but I'm guessing it's way more true than it is false.

So how have you all handled this issue? Any interesting tales to tell? If so, comment away!

1 comment:

  1. Great post and interesting stats. e-Dialog references similar numbers indicating that offline sales generated by email marketing accounts for 3-4 times the straight line sales you describe above. Of course, attributing sales to any specific marketing medium is difficult to do, particularly if you're touching your customers through a number of different channels (i.e. print, email, online ads, broadcast, etc.).

    Notwithstanding, it's an exercise worth doing and it's amazing how far a little regression analysis and computer programming can really take you. In a previous life, I started and ran the MIT Blackjack Team, of the best seller "Bringing Down the House" and "21" movie fame. In addition to determining our optimal playing and betting strategies by analyzing the rules of the game of blackjack, we also simulated the eight-deck shuffles of each of the casinos so that we could track specific cards and packets of cards through freshly shuffled decks so that we would know when to place our maximum bets for upcoming rounds of play.

    Given the billions of dollars of sales at stake for marketers, tracking the impact of email marketing should be an imperative for all companies. Set some budget aside and do a proper analysis. Who knows? In addition to the potential significant boost in your ROI, there might even be a movie in it for you. :)