Designing Performance Indexes
I talked about when you should, or should not, use a Performance Index in Part 1. If you have decided to go ahead this article gives you some practical advice on designing performance indexes.
Working out what to include
Less is definitely more with Indices. Some, like the FTSE 100, can accommodate large numbers of inputs as it’s a very simple index (It’s the value of all the shares added together) but as soon as you start weighting things or doing more exotic arithmetic you will save yourself a lot of work by keeping it as simple as possible. Use this approach to help you cull your KPIs before you attempt to roll them into an index. Remember, the more ingredients you have when designing performance indexes, the more opportunities there are for it to go wrong!
What to leave out of your index
- ‘Transient measures’ – things that may be relevant in the short term but will disappear
- Things that don’t have a logically coherent observable output. OEE works as a measure because all the elements result in a quantity of saleable product. Weight Watchers points work because (when you hit your target) you have taken on board a certain number of calories and nutrients – it’s all about food in this case. It might be tempting to stick ‘tidiness score’ into your efficiency measure, but tidiness doesn’t have an observable effect on the output of a factory (feel free to flame me 5S fanboys…) so they shouldn’t be mixed.
Building your Performance Index algorithm
There are a few occasions where the index has to represent a physical relationship and the maths gets very slightly more complicated – for example Body Mass Index (BMI). From experience, it’s unusual to have to go beyond basic arithmetic though.
Go through each of your performance Index elements and ask these questions:
Is bigger better?
Does a high score indicate better or worse performance? It’s a simple question, but one you need to answer at the start to make sure that your Index arithmetic is logical and consistent. Once you have that sorted, make sure that each element of the Index algorithm pushes the overall score in the ‘right’ directions when each element varies (use the ‘Does that make sense?’ test).
Is this a critical element to performance?
When it ‘fails’, does it render all the other elements meaningless? If so, then often you will multiply the whole expression by this value, so that when it falls to zero the index falls to zero.
Not a ‘do or die’ element? Then how important is it?
Different elements have different importance. You can weight each element to contribute to the final score, like this
The best way to decide the weightings, in order of importance are…
- Fundamental physical relationships: We KNOW that density=mass/volume
- Empirical relationships: We don’t understand the science, but when the weather goes up 10C we sell 20% more soft drinks
- Opinion: As a group we have agreed that customer service representative hairstyle contributes 20% to customer satisfaction
- Solo decision. I have unilaterally decided that number of web site hits comprises up to 10% of our ‘inbound
Is my Index ranged or open-ended?
- An open-ended index is one that can just carry on getting bigger – there’s no upper limit. Stock market indicators, Body Mass Index and the Retail Price Index.
- Ranged index’s are measures where there’s a maximum and minimum. Examples would include credit scoring, food hygiene scores or exam grading.
Both have their uses, just be very cautious about mixing both types together in one index.
Odd situations and the maths to watch out for
Answers heading to infinity
If A is 5 and B is 10, then C= 0.5
If B heads down to practically zero, let’s say 0.000001, our result shoots up to 5000,000.
As B tends towards zero C tends towards infinity.
Now it may be that answer is a fair reflection of what you are trying to show. More often though, it’s the unexpected side effect of the way the index is designed.
We create a measure for ‘cooking accuracy’ for baking a cake. That measure is for the difference between the weight of each ingredient we put in the cake, compared with the target weights in the recipe.
If you bake a cake and put 5 grammes too little sugar in it, you don’t fix that problem by putting 5 grammes of extra salt in it! A common mistake when designing performance indexes is to encourage the user to do just this. Often, the simplest solution is to use the ‘absolute’ variance – i.e. make the difference positive, whether it’s a ‘plus’ or ‘minus’ deviation.
When people are designing performance indexes, they will usually test them with ‘realistic’ values – and these are normally the values they are getting today. Whatever is ‘realistic’ during design, may prove to be a very bad picture of future values. To avoid this, make up a table of extreme values for your input measures and create a series of extreme ‘what if’ situations that really tests your KPI to the limit.
For each extreme situation ask the question ‘Is the Index value being presented an appropriate and realistic reflection of the impact of that situation?‘
Warming your audience up
Index’s can be a bit strange and new when people are first exposed to them. Clear explanation is one of the best ways to help people become comfortable with them, along with regular exposure to the output values. A ‘KPI Cheatsheet’ can be a really effective way of communicating how an index is calculated and can act as a handy reference tool.
Most critically, performance indexes will be used if they are seen as useful.
Good luck with designing performance indexes and let me know if you feel like sharing your shiny new Performance Index with the our KPI community.