Here’s how to fix it: A step-by-step guide to creating ‘Composite KPIs’
Time to read: 10 minutes
It’s a pretty common situation. You decide to try and fit all your measures on one A4/foolscap side. After lots of haggling, and increasing use of small font sizes, you can just about squeeze everything on one page (if you have good eyesight). You show it to the executive and they throw half an dozen extra measures and charts into the mix, blowing any chance of creating a one page dashboard.
At this point you have two options:
1) Leave something out. This is the easiest option, but also risky. In some businesses it really is impossible to fit all the relevant measures on one page. There is often fierce debate amongst individual ‘interest’ groups about which should be included. The surviving measures are often more an indicator of the political power of the stakeholder than of business relevance.
2) Dip your toes into the wonderful weird world of “composite” KPIs/measures.
Composite KPI is a term I coined to describe a measure that isn’t something that you can see/hear/touch/count. They are all over the place, but rarely identified as something “artificial”
Here are some examples of composite KPIs you are very likely to have used or seen…
- BMI – Body mass index. It gives an indication of “lean-ness” in a single number
- FTSE 100 – A stock exchange index showing how the UK stock market is performing
- Weight Watchers Points – using a measure that mixes calorific content, fat and fibre content to give a “food healthiness score”
- RPI – The Retail Price Index – A measure used to show how a selected “basket” of prices have increased
- OEE – Overall Equipment Effectiveness – A measure of efficiency that factors in quality, up-time and speed in production operations.
- Fitness “scores” used by fitness monitoring gadgets like Fitbit
- Eco Score on a Toyota Prius dashboard
The more you look the more you find them all over the place. It’s not often that people have the confidence to create new ones. This is a pity as they can bring some serious benefits:
- Summarise complex situations with a single number
- Free up much-needed dashboard space without compromising on the mix of input data
- Drive you to think about the way measurement entities interact (e.g. downtime, quality and rate in the OEE – Overall Equipment Effectiveness – measure)
Step-by-step guide to making your very own composite measure
1) Engage the users of the new measure
Get those who will live with the measure into a room and get them bought into the composite measure. Best to do this right from the start as objections (or worse, being ignored) are the biggest challenges you are likely to face with your new composite KPI.
- Get the right people in the room for the discussion
- Explain the problem you are trying to solve
- Discuss all the options for tackling this problem
- Develop a shortlist of “ingredient measures” that should go in the composite measures – these are the measures that will be used to calculated the composite measure.
- Genuinely listen to, and think about, any objections and concerns raised at this point
For some more tips on how to create KPI engagement, read this blog.
2) Map the relationship between the measures using this approach
For a composite KPI to work there needs to be a relationship. Sometimes this will be obvious, where there is a physical relationship between the behaviours and the top level measure – such as fuel efficiency and Eco Score on Toyota cars. In that situation the ‘Eco Score’ is a leading indicator of fuel economy.If you get that algorithm wrong it will become clear when you get a ‘good’ Eco Score and poor fuel economy.
In other situations it may not be so clear cut, for instance where you are trying to ‘measure staff engagement’ or ‘customer service’. There will be lagging indicators of the effectiveness of both of these examples but those outcomes are partially determined by outside factors, so it’s not completely clear-cut.
3) Discover and describe the logical relationship between the ingredient KPIs
Let’s use Body Mass Index (originally called the Quetelet index) as an example here. The measure was developed by Adolphe Quetelet. Put very simply, this allows for the fact that healthy tall people, on average, will have a higher body mass than shorter healthy people. He showed that if you divided the height of someone (in metres) by the square of their mass (in kg) you end up with a series of BMI “bands” that allow you assess how healthy someone’s weight is without having to look them up on a 2 axis chart.
Once you understand the figure and the range of “healthy” BMI figures it becomes really quick and easy to broadly judge the level of someone’s obesity. The key thing here is that he spotted that as someone’s weight goes us a function of the square of their height, so there’s a meaningful insight about the underlying physical relationship here. If you want to get all geeky here and find out about an improved version of BMI have a read of this article http://people.maths.ox.ac.uk/trefethen/bmi.html. I particularly like this improved method as it moves my figure down (I’m a tall guy). If you are short you may not like the answer so much!
If there isn’t a clear-cut logical relationship then you will need to build one from scratch. Think about whether the ingredient KPIs are ‘additive’, i.e. you total them up, or are factors that multiply together. Be careful with factors as extreme values can have a massive impact on other measures – as you will see when you get to Step 5.
4) Build your first algorithm
Algorithm sounds a bit fancy, but all we are talking about here is creating a step-by-step formula that represents the relationship between the “ingredients” in our composite measure. Understanding the physical or logical relationship behind our “ingredient” measures can help hugely. If you are creating something a bit more abstract (like Weight Watcher Points) then you will need to put a bit more thought into how much weighting you put behind certain inputs.
The key ways to tweak and control you algorithm are:
A) The big guns in descending order of “amplification”
- Indices (raising something to a power of x)
- Division or multiplication to drive the algorithm
- Good ol’ addition and subtraction
B) The small guns
- Coefficients – fixed multipliers. The BMI figure is simply the coefficient that allows us to equate kg to height squared, all Adolphe did was to rearrange the equation to put the coefficient in the hot seat.
- Correction values – These are offsets to enable us to make things work. If you think back to school, linear charts could be described as y=Mx+c. We are talking about c here.
You also need to consider if it is open ended (like the FTSE 100) or a fixed scale (like OEE, where 100% is your upper limit)?
5) Try and break your new algorithm
Think about as many extreme situations as you can: Test your algorithm against the extreme situations and target state
Once you have your basic algorithm, like all new measures, you should really try your best to break it. Things that usually cause the most problems are the “big gun” ways of expressing relationships. You really need to think about what happens if a value goes high or drops to zero.
Typically if you are dividing by something that goes to zero you are in real trouble. Raising a figure to the power of a large figure can also cause some surprising outputs. The best way to do this is to create a spreadsheet with your algorithm in and a good range of extreme inputs.
You may also want to plot a graph (chart) of the outputs. Rethink you algorithm if you get any strange results you think don’t reflect the reality of situation you are trying represent.
One particularly poor measure I have seen affected by this is the Bradford Factor. It’s a dreadful HR measure that supposedly measures workplace disruption caused by sickness (read more about it here – note that no-one has ever found the supposed research that this measure is based on and Bradford Business School have pointedly distanced themselves from any involvement in its creation).
The Bradford Score (a composite KPI) is determined as B=S² x D, where B is the Bradford score, S is the number of absences and D is the total number of days missed. Let’s look at two situations.
a) Tim has 20 working days off for a back operation. Bradford score = 20
b) Sarah has chronic asthma and has had to take 20 working days off, as half days. Bradford score = 32000
Unfortunately this wrong-headed measure is being used to make decisions about real people even today
6) Fine Tune things
Tune and re-test the algorithm until you get the right response curve
The next question is “Does the response curve of the algorithm give the correct ‘real world’ response curve?”.
Look through the input cases you developed in the previous step and ask yourself if the score for each situation is proportionate to that situation. If it isn’t then you may need to tweak things until is does make sense. It’s easier in situations where there is a common baseline you can equate everything back to (e.g. line hours when calculating OEE.)
7) Double check with your audience
Take the new measure on a ‘road-show’ with those who will be creating the measure or using it to drive the business (though it’s best to involve them from Step 1, if you can)
An open discussion, with no ‘right answer’, will be the most effective way of developing acceptance and engagement. Questions that normally get some interesting answers include:
- Do you feel the measure is fair?
- Which situations do you think might give a misleading output?
- How could the measure be better and/or fairer?
Depending on the responses, you may have to go back to Step 4 at this point or you may be lucky and be ready to launch your new shiny composite KPI.
Composite KPIs work well when…
- Existing management information is fairly stable, reliable and mature
- Sets of ingredient measures that are fundamentally, causally, related (height and weight, price, production line performance)
- A single “at a glance” summary is really helpful
Composite KPIs don’t work so well when…
- The audience is a casual or not very sophisticated one, so they don’t have the time, inclination or ability to understand how the KPI works
- There is little meaningful causal relationship between the different elements of the composite KPI/measures
- The measure has not been explained to the target audience, or they don’t trust the measure
A final word of warning, it can take time and effort to create a composite measures, so it’s not something you would want to do on a whim. If you do end up creating a good one you could find that it rapidly becomes a crucial business tool, giving your organisation a serious competitive advantage.
Good luck and drop me a line with any that you develop that you are particularly proud of.