Need to Keep Your KPIs REALLY Simple? The KPI Index – Part1

How a KPI Index saved my email newsletter… sort of

After shying away from sending out a regular newsletter for far too long, I decided to get to grips with my newsletter software a few weeks ago. If you have ever sent out newsletters by email you will know that a serious challenge is getting the email into your subscribers inbox and not blocked by filters. This is a particular challenge as there are SO many ways that your mail can be marked as unwanted, it can get rather overwhelming.

MailPoet have come up with a really nifty solution called Mail-Tester. They have worked out a way of distilling the complex (and really technical) challenge into

  1. A number
  2. A picture
  3. A simple description

Here’s the type of feedback you get when you send them a test email…

Poor mail tester score


What makes this so effective is that the overall message is crystal clear, it gives you a quantitive scale the relates to your likelihood of delivery. It also explains this in plain English and adds a picture to really hammer the point home. What makes it so good though, is that it shows you how the score is built up of sub-scores, which parts are causing you issues and what steps you need to take to IMPROVE your score. Here’s the type of feedback you get in the score breakdown below…

Broken Link Breakdown

  • It shows you how many ‘points’ you lose per issue, so you instinctively prioritise the ‘biggest win’.
  • It shows you details of the specific issue, sometimes linking to further advice
  • It often suggests ways to fix the problem

So what makes a good performance index?

A good performance index  should:

  1. Mix together elements that all genuinely contribute to the same measurable outcome (deliverability of email in this case).
  2. Offer ‘at a glance’ insight of a complex situation.
  3. Point the user in the right direction for more detailed analysis of the underlying problems.
  4. Be transparent and balanced. Indices often use ‘weightings’ on their component parts. These need to be visible and attempt to accurately reflect the true drivers behind performance.

When to use a performance index

Food Hygiene ScorePerformance indexes are great for situations where a snap assessment is important. You see them all the time, but may not even notice. Fitness trackers, food hygiene scores and even exam results. They are all indexes that summarise complex questions (‘Am I doing enough exercise?’,’ Will I get food poisoning?’ and ‘Does Bernie understand Mechanical Engineering?’) in very quick-to-assess ways.  They work particularly well when you ‘check in’ frequently on something, for instance Weight Watchers Points score, and can be powerful motivators for behaviour change.

When not to use a performance index

Performance Index’s are fairly blunt instruments. The exact scores varies depending on how they are designed. This means you cannot solely rely on them for ‘big’ decisions, especially where you are comparing small differences between index scores. They are good for ‘at a glance’ assessments, driving behaviour in the right direction and trend analysis. They can also be badly designed, incorrectly weighted and unrepresentative. If you have any doubts about how the index works, you need to pull it apart, understand how it works and do some deeper analysis of the constituent measures. Another risk is that you have more than one trend that cancel each other out. Again the only remedy is to look at the underlying trends, not just the index.

Do they work?

And my newsletters? Well thanks to I’ve gone from a rainy cloud, a score of 1.5, and a message saying ‘Your email will never see the light of anyone’s inbox’ through to this….

Mail Tester Score and Headlines - Good

That’s great, but a word of warning. Even with a ‘perfect 10’ my iCloud email STILL treats my emails as junk mail, so perhaps I shouldn’t have been quite so trusting of their Mail-Tester’s scoring index….

In Part 2 I’ll give you some practical tips on building your own Performance Index.


Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.