Too Many KPIs or Measures?

PostIts on a KPI Shortlisting matrix
  • Why is it dangerous to have too many KPIs?

Choosing the right number and type of KPIs is like walking a tightrope. Too many KPIs can lead to overload, confusion and frustration. Too few KPIs can mean you miss critical information necessary to keep your business, people or bottom-line safe.

So, how do you manage 'KPI overload', whilst staying confident that you are measuring the right things, are not skipping critical measures and will not overwhelm your organisation with KPI administration?

The method for achieving this balance is 'Shortlisting', an approach borrowed from Step 4 of the ROKS method, our unique structured method for developing KPIs safely, quickly and effectively.

A simple method for dealing with too many KPIs or measures

By the time you have created your KPI Tree(s) you will have a large number of candidate measures. Some teams are horrified when they realise that they have come up with 70, 80 or more measures. The good news is that you definitely won't use all of the measures that you have on your KPI Tree.

Reasons for not including the measures would include:

  • The measure isn't important enough.
  • It's just not physically possible to measure or report on it.
  • You will report on it, but it's going to require more work or time to do it.

How to cut the measures that don't really count

Your next task is to reduce that list. You will do this by getting the group to rate each measure on two criteria, Importance and Availability, and plot those ratings on a four box grid or matrix.

too many kpis - blank kpi triage matrix

Here is a bit more description on those two criteria:

Importance: Is the measure significant to your business? 

This is not normally a yes/no choice, but it's usually possible to get a group to rank importance on a 0-10 scale. Importance can be broken down into criticality and breadth-of-influence if you are having real problems agreeing, although I've never had to go this far in real life. 10 is the maximum importance rating here.

Availability: How easy is it to get hold of the data? 

10, the `most available' score, would be information that is automatically-generated, fully trusted and can be sliced and diced in whichever way you choose with little or no effort.

0 means that the data is effectively unavailable, requiring unrealistic levels of effort and disruption to collect it. In some circumstances it may not be physically possible to collect this information at all.

Measuring the unmeasurable

It is quite common in manufacturing process industries to be unable to measure a process parameter directly without `breaking' the process you are trying to measure. In that situation you will often measure the direct drivers of that parameter, knowing that if they are under control then the process variable you are interested in will be under control as well.

Umeasurable KPI example - paper thickness

For example, in paper making you have a very thin, very fragile sheet of paper forming on a machine which moves at high speed. Physically measuring the thickness of the sheet without stopping the process is virtually impossible, so you actually measure the absorption rate of gamma radiation, from a calibrated source, and calculate the thickness based on that absorption.

So the key with anything that is `unmeasurable' is to look at something that either affects or is affected by the thing you are interested in.

Another example of this is judging how interested toddlers are in a TV programme by seeing how frequently they are distracted by a pile of interesting toys. This is the technique used by the creators of Sesame Street to gauge how absorbing each element of their show was.

KPI shortlisting process checklist - Post It method

  • Grab a flip chart.
  • Label Importance on the vertical axis and Availability on the horizontal axis.
  • Next, number the bottom-level measures on your KPI Tree for reference. I haven't done this in the examples, but it is a good way of staying on top of things.
  • Then, add dividing lines at the midpoint of each axis so that you have a grid of four boxes or matrix.
  • Go through each candidate measure with the group rating it on a scale of 0-10 for:
    • Importance
    • Availability
  • As you go, note down any disagreements about the scores the group assigns - this can be useful if you are challenged later on the choices made, and it gives you an audit trail.
  • Plot each candidate measure on the chart, with a cross and the measure name (or number).

You should end up with something that looks like this..

too many kpis - sample kpi triage matrix

Use a Post-it notes for each measure as this gives you flexibility. If there is debate about the rating for each KPI you can easily separate them or move them individually. If you decide to use the Post It method, just make sure you take a picture of the flip before you take it down as you can lose a lot of work if your notes fall off!

How to use the matrix to tackle 'too many KPIs' challenge

too many kpis - descriptive kpi triage matrixTop right-hand box - The high-importance and easily available quadrant is the tranche of measures you put on your first dashboard/report. Easy to get and important, why wouldn't you?

Top left-hand box - The high-importance and difficult-to-collect quadrant. The Aspire list becomes the `to-do' list for your dashboard. This is where you focus your KPI development effort.

Bottom right-hand box - These are the trivial but easy to collect measures. It is worth double-checking any that border on important, but don't get sucked into putting something on the dashboard or report simply because you can.

Bottom left-hand box -The trivial and yet hard-to-collect measures. Unless these are close to the centre of the quadrant boundaries, forget these and move on. If they are close to the top right-hand box you may just want to double-check your assessment.

By the end of the session you should have divided your candidate measures into three:

Do-it-straight-away list of measures - the Use list

To-do list of measures to develop - the Aspire list

Rejected measures (with reasons) - the Discard list

Why the Discard list is important

It is easy to overlook the last point, as you are not going to implement these measures, but that would be a mistake. Recording which measures you chose to reject with the reasons why can be a really powerful way of defending the choices made. `We didn't go with that measure and here are the reasons why…' can be powerful in stopping report `bloat' and demonstrating the rigour of the process.

Finally, develop measures to fill the gaps - Your Aspire list measures

The measures that ended up in the Aspire list may just not be measured currently or they may be important but tricky to measure.

Although in some situations it may be easy to add the new measures into existing data collection systems, very often it isn't that straightforward. I would recommend setting up each measure as a `mini project'. This gives you some framework and structure and stops you potentially sleepwalking into something nasty.

Develop your missing measures and KPIs

  • Confirm that you do really want to measure this thing/these things.
  • Identify stakeholders (using the RACI matrix approach).
  • Define the measure (using the KPI definition checklist).
  • Review that definition with the key stakeholders.
  • Do a quick feasibility study - especially if there's IT involved.
  • Check to see what organisational project management methods and systems you need to fit in with.
  • Assign a full-time or part-time project manager.
  • Put together a one-page `charter'. A charter is a short document that outlines objectives, resources, timescales and anticipated issues (with mitigations).
  • Draft a simple project plan.
  • Review the timescale and cost - are you still sure you want to go ahead?
  • Create and maintain an action log.
  • Set up some simple project governance (e.g. rules, meeting schedule etc.) to make sure the project stays on track.

In some cases it may be more substantial than a `small' project but the principles remain the same and there's plenty of guidance out there for running projects.

Action Tracker - Screen Shot

Tip: Use action logs to keep your KPI implementation on track

Action logs are simple but useful tools to keep things on track. You can download a free action tracker template for one the I've built in Excel here.

Of course, you can easily create your own. If you do, make sure you have the following covered:

  • Track actions
  • A unique reference number for each action.
  • A grouping description, if it's a long list, like `IT' or `Training'.
  • A meaningful description of the action.
  • Who owns the action.
  • When the action is due by.
  • When the action is closed.
  • Whether the action is on hold.
  • Any relevant notes or comments.
  • Risks (optional).
  • Mitigations (optional).

Recap: How to shortlist your KPIs

If you have too many KPIs and are having heated debates about what you should, or should not, implement this is a structured method for coming up with a practical solution to the problem.

PostIts on a KPI Shortlisting matrix

Time Needed: 120 minutes

Estimated cost USD 100

Things Needed ?

- A list of KPIs or measure you are considering using
- A meeting room or video call
- A spreadsheet
- 4 to 8 people from your organisation involved in the things you want to measure

Required tools:

- A Computer.
- A spreadsheet (Excel, Google Sheets etc.)

Seven steps to shortlist your KPIs

1: Rate KPI on 'Importance'
  • Enter your KPI name on the spreadsheet.
  • Agree a score between 1 and 10 for the 'Importance' of that KPI. A 10 would be something that existentially crucial for the organisation. An example of a 10 for importance would be 'Cash in bank'.
  • Create a 'comments and notes' column and record the reasoning for your groups agreed score.
2: Rate KPI on 'Ease of Measurement'
  • As a group, agree a score for 'Ease of Measurement' between 1 and 10, for your KPI.
  • A score of 10 would be something that is calculated and reported with zero effort or input by a human.
  • Create a 'comments and notes' column and record the reasoning for your groups agreed score.
3: Repeat the rating process for the next KPI on the list
  • Move on to the next KPI candidate
  • Rate the KPI for Importance and Ease of Measurement
  • Keep repeating Steps 1 and 2 until the full KPI list has been reviewed and scored
4: Create a ranking score for each KPI

Set up a column that multiplies the Importance and Ease of Measurement scores for each KPI into a 'ranking score'.

This score will be over a range of 1-100.

5: Sort your KPIs by ranking score

Use the 'ranking score' column to sort your KPIs in descending order of size.

6: Review each KPI by score
  • Review each scored and ranked KPI.
  • Add a 'Status' column to the spreadsheet.
  • As a group, flag the 'status column' cell for each KPI as...
    • Primary, we will measure and report this (high ranking score)
    • Aspire. We would like to measure this in the future (typically high 'Importance' score, but low 'Ease of Measurement score)
    • Discard. Not important enough and/or possible to measure
7: Implement your 'Yes' KPIs
  • Go ahead and fully define the KPIs you decide to implement
  • Don't forget to review your 'Aspire' list as you progress your 'Primary' KPIs.

Leave a Comment