Comparing Apples to Pineapples…the email metrics dilemma
By Kelly Rusk, Communications Manager

If you've ever compared email marketing metrics or benchmark studies, surely you've noticed not all email reporting is created equal. In fact, email marketing is the only direct marketing industry that has absolutely no standardization for metrics. How's that for crazy?

While the email experience council is looking to standardize metrics, my guess is it'll be quite a while before we see standardization adopted by all Email Service Providers (ESP). In the meantime, the best thing you can do is understand the differences so you can make equal metrics comparisons.

Deliverability
Deliverability is probably the toughest to decipher. In part, this is thanks to Internet Service Providers (ISP) not providing detailed enough reports to ESPs. This leaves your email provider left to figure out what counts as a hard versus soft bounce. To add to the confusion, definitions of hard and soft bounces can also vary by provider.

It really gets tricky when you start to figure out how your delivery or bounce rate is determined. It may be hard and soft bounces subtracted from total sent. Or it may just include hard bounces. Sometimes your provider will give you reasons for each bounce, or break it down by categories, sometimes they won't.

Open Rates
It seems pretty straightforward - a small 1x1 pixel invisible image is hidden in your email. When that image is activated, the email is counted as opened. Yes, that makes sense, but when your ESP reports it, is it counting one open per recipient? Or every single time the email is opened, regardless of who it was? This is a reality given emails high rate of forwarding. Some ESPs even go as far as to calculate "Estimated Opens" or "Real Opens" factoring in people who may have viewed the immeasurable text opens, or those who may have viewed the email with images deactivated. Measured opens versus actual opens are never the same. You shouldn't you're your success solely on open rates.

Click Rates
As far as calculating goes, the biggest differences occur with the click rate. This is especially troubling since click rates are often regarded as the best measure of success. It's not uncommon to find clicks calculated as unique clicks divided by unique opens, or unique clicks divided by total delivered, or total clicks by total delivered, total or unique clicks divided by total sent. The list goes on… It's enough to make your brain spin!

If you switch vendors, and your click rate changes dramatically – chances are it's just a matter of mismatched calculations. Is there a right way and wrong way? Absolutely not. The more ways you calculate your click rate, the more valuable information you can pull from it. For example, calculating by "sent" can help determine the value of your list – i.e. a high bounce rate needs to be factored into expectations. Calculating by "delivered" provides an accurate portrait because it does not include bounced addresses – after all how can someone click to make a purchase if they haven't even received the email? Finally "dividing by opens" gives you a great snapshot of subscriber engagement: How many people are opening the email and interested enough to click through?

Using the totals instead of unique opens or clicks also helps you understand more about subscriber engagement, and can give you an idea about people forwarding your email from within their email program.

What do you need to know?
The bottom line is this. Any of the above metrics can be valuable. When developing (or revising) your strategy, it's wise to figure out what the best method is for your email program and to work with the raw data yourself. Pay attention to your own email trends. If you see open rates declining, take a closer look at subject lines. If you see increasing unsubscribes, take a look at content. And so on. It is worthwhile to be aware of industry stats, but keep those findings in perspective. Different industries get different results. It's also important to note how the author/company calculates metrics and make sure you are doing an equal comparison.

PS - Have you checked out the cardcommunications Quarterly Trends Reports?
Here's how we calculate our stats:
Delivered emails: the total sent minus all hard and soft bounces
The Opens: unique opens divided by total delivered
The Clicks: unique clicks divided by unique opens and total delivered .