A simple method for dealing with too many KPIs or measures
By the time you have created your KPI Tree(s) you will have a large number of candidate measures. Some teams are horrified when they realise that they have come up with 70, 80 or more measures. The good news is that you definitely won’t use all of the measures that you have on your KPI Tree.
Reasons for not including the measures would include:
- The measure isn’t important enough.
- It’s just not physically possible to measure or report on it.
- You will report on it, but it’s going to require more work or time to do it.
How to cut the measures that don’t really count
Your next task is to reduce that list. You will do this by getting the group to rate each measure on two criteria, Importance and Availability, and plot those ratings on a four box grid or matrix.
Here is a bit more description on those two criteria:
Importance: Is the measure significant to your business?
This is not normally a yes/no choice, but it’s usually possible to get a group to rank importance on a 0-10 scale. Importance can be broken down into criticality and breadth-of-influence if you are having real problems agreeing, although I’ve never had to go this far in real life. 10 is the maximum importance rating here.
Availability: How easy is it to get hold of the data?
10, the `most available’ score, would be information that is automatically-generated, fully trusted and can be sliced and diced in whichever way you choose with little or no effort.
0 means that the data is effectively unavailable, requiring unrealistic levels of effort and disruption to collect it. In some circumstances it may not be physically possible to collect this information at all.
Measuring the unmeasurable
It is quite common in manufacturing process industries to be unable to measure a process parameter directly without `breaking’ the process you are trying to measure. In that situation you will often measure the direct drivers of that parameter, knowing that if they are under control then the process variable you are interested in will be under control as well.
For example, in paper making you have a very thin, very fragile sheet of paper forming on a machine which moves at high speed. Physically measuring the thickness of the sheet without stopping the process is virtually impossible, so you actually measure the absorption rate of gamma radiation, from a calibrated source, and calculate the thickness based on that absorption.
So the key with anything that is `unmeasurable’ is to look at something that either affects or is affected by the thing you are interested in.
Another example of this is judging how interested toddlers are in a TV programme by seeing how frequently they are distracted by a pile of interesting toys. This is the technique used by the creators of Sesame Street to gauge how absorbing each element of their show was.
KPI shortlisting process checklist
- Firstly, make sure you are dealing with actual measures (the lowest-level boxes on the KPI Tree), not the higher-level headings.
- Get a flip chart.
- Label Importance on the vertical axis and Availability on the horizontal axis.
- Number the bottom-level measures on your KPI Tree for reference. I haven’t done this in the examples, but it is a good way of staying on top of things.
- Add dividing lines at the midpoint of each axis so that you have a grid of four boxes or matrix.
- Go through each candidate measure with the group rating it on a scale of 0-10 for:
- Note down any disagreements about the scores the group assigns – this can be useful if you are challenged later on the choices made, and it gives you an audit trail.
- Plot each candidate measure on the chart, with a cross and the measure name (or number).
You should end up with something that looks like this..
Using Post-it notes for each measure can give you flexibility if there is debate about the rating for each KPI. Just make sure you take a picture of the flip before you take it down as you can lose a lot of work if your notes fall off!
How to use the matrix
Top right-hand box – The high-importance and easily available quadrant is the tranche of measures you put on your first dashboard/report. Easy to get and important, why wouldn’t you?
Top left-hand box – The high-importance and difficult-to-collect quadrant. The Aspire list becomes the `to-do’ list for your dashboard. This is where you focus your KPI development effort.
Bottom right-hand box – These are the trivial but easy to collect measures. It is worth double-checking any that border on important, but don’t get sucked into putting something on the dashboard or report simply because you can.
Bottom left-hand box –The trivial and yet hard-to-collect measures. Unless these are close to the centre of the quadrant boundaries, forget these and move on. If they are close to the top right-hand box you may just want to double-check your assessment.
By the end of the session you should have divided your candidate measures into three:
Do-it-straight-away list of measures – the Use list
To-do list of measures to develop – the Aspire list
Rejected measures (with reasons) – the Discard list
Why the Discard list is important
It is easy to overlook the last point, as you are not going to implement these measures, but that would be a mistake. Recording which measures you chose to reject with the reasons why can be a really powerful way of defending the choices made. `We didn’t go with that measure and here are the reasons why…’ can be powerful in stopping report `bloat’ and demonstrating the rigour of the process.
Develop measures to fill the gaps – Your Aspire list measures
The measures that ended up in the Aspire list may just not be measured currently or they may be important but tricky to measure.
Although in some situations it may be easy to add the new measures into existing data collection systems, very often it isn’t that straightforward. I would recommend setting up each measure as a `mini project’. This gives you some framework and structure and stops you potentially sleepwalking into something nasty.
Develop your missing measures and KPIs
- Confirm that you do really want to measure this thing/these things.
- Identify stakeholders (using the RACI matrix approach).
- Define the measure (using the KPI definition checklist).
- Review that definition with the key stakeholders.
- Do a quick feasibility study – especially if there’s IT involved.
- Check to see what organisational project management methods and systems you need to fit in with.
- Assign a full-time or part-time project manager.
- Put together a one-page `charter’. A charter is a short document that outlines objectives, resources, timescales and anticipated issues (with mitigations).
- Draft a simple project plan.
- Review the timescale and cost – are you still sure you want to go ahead?
- Create and maintain an action log.
- Set up some simple project governance (e.g. rules, meeting schedule etc.) to make sure the project stays on track.
In some cases it may be more substantial than a `small’ project but the principles remain the same and there’s plenty of guidance out there for running projects.
Action logs are simple but useful tools to keep things on track. You can download a free template for one the I’ve built in Excel by following this shortlink: http://wp.me/p1H6XP-zD
Of course, you can easily create your own. If you do, make sure you have the following covered:
- Track actions
- A unique reference number for each action.
- A grouping description, if it’s a long list, like `IT’ or `Training’.
- A meaningful description of the action.
- Who owns the action.
- When the action is due by.
- When the action is closed.
- Whether the action is on hold.
- Any relevant notes or comments.
- Risks (optional).
- Mitigations (optional).