Designing Performance Indexes. The ‘Nuts and Bolts’ Part 2

a Boy looking at math problem and think in itDesigning Performance Indexes

I talked about when you should, or should not, use a Performance Index in Part 1. If you have decided to go ahead this article gives you some practical advice on designing performance indexes.

Working out what to include

Less is definitely more with Indices. Some, like the FTSE 100, can accommodate large numbers of inputs as it’s a very simple index (It’s the value of all the shares added together) but as soon as you start weighting things or doing more exotic arithmetic you will save yourself a lot of work by keeping it as simple as possible. Use this approach to help you cull your KPIs before you attempt to roll them into an index. Remember, the more ingredients you have when designing performance indexes, the more opportunities there are for it to go wrong!

increase and decrease arrows talking about enonomy

What to leave out of your index

Don’t include:

  • ‘Transient measures’ – things that may be relevant in the short term but will disappear
  • Things that don’t have a logically coherent observable output. OEE works as a measure because all the elements result in a quantity of saleable product. Weight Watchers points work because (when you hit your target) you have taken on board a certain number of calories and nutrients – it’s all about food in this case. It might be tempting to stick ‘tidiness score’ into your efficiency measure, but tidiness doesn’t have an observable effect on the output of a factory (feel free to flame me 5S fanboys…) so they shouldn’t be mixed.

Building your Performance Index algorithm

businessman with multiple ideas swirling around himOur index will generally just involve basic arithmetic – addition, subtraction, division and multiplication

There are a few occasions where the index has to represent a physical relationship and the maths gets very slightly more complicated – for example Body Mass Index (BMI). From experience, it’s unusual to have to go beyond basic arithmetic though.

Go through each of your performance Index elements and ask these questions:

Is bigger better?

Does a high score indicate better or worse performance? It’s a simple question, but one you need to answer at the start to make sure that your Index arithmetic is logical and consistent. Once you have that sorted, make sure that each element of the Index algorithm pushes the overall score in the ‘right’ directions when each element varies (use the ‘Does that make sense?’ test).

Is this a critical element to performance?

When it ‘fails’, does it render all the other elements meaningless? If so, then often you will multiply the whole expression by this value, so that when it falls to zero the index falls to zero.

Not a ‘do or die’ element? Then how important is it?

Different elements have different importance. You can weight each element to contribute to the final score, like this

Simple weighted index for regulated complaints

Simple weighted index for regulated complaints

The best way to decide the weightings, in order of importance are…

  1. Fundamental physical relationships: We KNOW that density=mass/volume
  2. Empirical relationships: We don’t understand the science, but when the weather goes up 10C we sell 20% more soft drinks
  3. Opinion: As a group we have agreed that customer service representative hairstyle contributes 20% to customer satisfaction
  4. Solo decision. I have unilaterally decided that number of web site hits comprises up to 10% of our ‘inbound

Is my Index ranged or open-ended?

  • An open-ended index is one that can just carry on getting bigger – there’s no upper limit. Stock market indicators, Body Mass Index and the Retail Price Index.
  • Ranged index’s are measures where there’s a maximum and minimum. Examples would include credit scoring, food hygiene scores or exam grading.

Both have their uses, just be very cautious about mixing both types together in one index.

Odd situations and the maths to watch out for

Answers heading to infinity

businessman using a telescope on a bar chartAnything that is multiplied or divided by anything else can go very screwy at the extreme. Let’s say we diving A by B to give answer C.

If A is 5 and B is 10, then C= 0.5

If B heads down to practically zero, let’s say 0.000001, our result shoots up to 5000,000.

As B tends towards zero C tends towards infinity.

Now it may be that answer is a fair reflection of what you are trying to show. More often though, it’s the unexpected side effect of the way the index is designed.

baking a cake can have the same pitfalls as designing performance indexesTwo wrongs making a right

We create a measure for ‘cooking accuracy’ for baking a cake. That measure is for the difference between the weight of each ingredient we put in the cake, compared with the target weights in the recipe.

Ingredient Target Actual Difference
Flour 300g 300g 0
Sugar 75g 70g -5g
Salt 3g 8g +5g
Egg 50g 50g 0
Net error0

If you bake a cake and put 5 grammes too little sugar in it, you don’t fix that problem by putting 5 grammes of extra salt in it! A common mistake when designing performance indexes is to encourage the user to do just this. Often, the simplest solution is to use the ‘absolute’ variance – i.e. make the difference positive, whether it’s a ‘plus’ or ‘minus’ deviation.

businessman looking for successTesting your shiny new index

When people are designing performance indexes, they will usually test them with ‘realistic’ values – and these are normally the values they are getting today. Whatever is ‘realistic’ during design, may prove to be a very bad picture of future values. To avoid this, make up a table of extreme values for your input measures and create a series of extreme ‘what if’ situations that really tests your KPI to the limit.

For each extreme situation ask the question ‘Is the Index value being presented an appropriate and realistic reflection of the impact of that situation?

Warming your audience up

Group Of Cheerleaders In Action With Male Coach - Concept Of Unity And Team Sport - Training At Coll

Index’s can be a bit strange and new when people are first exposed to them. Clear explanation is one of the best ways to help people become comfortable with them, along with regular exposure to the output values. A ‘KPI Cheatsheet’ can be a really effective way of communicating how an index is calculated and can act as a handy reference tool.

Most critically, performance indexes will be used if they are seen as useful.

Good luck with designing performance indexes and let me know if you feel like sharing your shiny new Performance Index with the our KPI community.


Speak Your Mind