Why KPI definition is important
If there’s one simple tool that you can put into action today that will dramatically improve the quality of your KPIs and add a stack of professional credibility to your reputation, good quality KPI definition is it.
What I’m about to describe may sound very obvious and simple. However this doesn’t make the advice any less useful or powerful.
The common problem with KPI definition
An issue in almost every business is what I call the ‘common sense’ problem. It is very common for companies to use descriptive names for their KPIs and measures. This can be very dangerous as the human mind is great at making assumptions based on the name. As soon as we see the name of the KPI we start thinking 'It's obvious, everyone understands that!', even if we do so subconsciously.
A KPI definition horror story
KPI definition problems example - Utilisation
Let’s take an example. There is a popular operational measure called ‘utilisation’. Utilisation is commonly used by professional services firms to show what proportion of the available employee time was used for billed work to the client. Utilisation is used across a wide range of industries.
A KPI definition of Utilisation
So, here’s a plain English definition:
‘Utilisation is the ratio of billed hours to available hours for an individual.’
That seems pretty simple, doesn’t it? But let’s look at some of the questions that this does not answer:
- For 'available hours' do you mean nominal, or actual, and do you include overtime?
- What about holidays or sickness absence, is that zero utilisation?
By individual do you mean only individuals who are billable, or do you include support staff?
- Do you include mandatory commitments, such as safety training, in the available hours?
- What about sales? This is clearly a value-add activity but is not billable.
- What happens if you bill at a much reduced hourly rate? Does that utilisation count the same as full-rate work?
- What do you do with staff who have dual roles, only one of which is a billable position?
- Clearly there is plenty of scope for misunderstanding here. It’s pretty rare to find a plain English definition that does not give rise to lots of similar questions.
It looks like our initial definition has lots of 'holes' in it. How can we make sure that our definitions don't suffer from the same issues?
10 essential KPI definition questions
As we said earlier, it’s surprising how often you find multiple measures all of the same name in one organisation. If you have several measures that relate to efficiency, make sure you give them distinctive names. If a specific type of efficiency KPI relates to a specific process, give it a specific name. For example, call it Line 7 Bottling Efficiency, rather than just Efficiency. The key thing is that there’s only one measure for with each name. This helps us make sure there’s no chance different measures mixed up by accident.
Why should we measure this?
Sometimes we lose sight of why we are measuring something. This is particularly common in very top-down businesses where the boss maybe asks for some specific data - analysts are often too scared, or beaten down, to ask what it’s going to be used for. You will have thought carefully about the purpose of each measure in the Shortlisting step, so this should not be too hard.
Customers - [Optional]
Who will use this KPI?
We produce KPI reports to help in decision-making. If you are a one-person business, then this is a very easy question to answer. If there are more people in your business, you need to have a clear understanding of who uses the data. That way, if a question comes up about how we measure or report something we can talk to our KPI customer and discuss it with them, instead of just trying to guess what they want. If there’s no clear customer or decision being made on the back of a KPI you should seriously think about chopping it.
Where will the KPI data come from?
One of the most common sources of errors comes from holding similar data in multiple places. If you have lots of spreadsheets holding data that is similar, but maybe slightly different in content or scope, then you have a real risk of mistakes creeping in. This is particularly true if you have more than one person involved in producing KPI reports. This field needs to go down to painful levels of detail.
For example, rather than just saying ‘From the production performance spreadsheet’, you should specify the name of the server the document lives on, the folder, the precise sheet name, the tab in the spreadsheet and the column-row range. Your test for whether you have done this properly is whether someone who hasn’t used this data before can successfully navigate to exactly the right data by following the instructions shown under Data Sources.
Definition or Formula
If there’s any calculation, how is the KPI calculated?
What is and is not included in the values used?
It’s important to be crystal clear about the calculation used, and what is and is not included in the figures that are used in the calculation.
Production Resources - [Optional]
What resources are needed to produce the KPI and reports?
Sometimes you will need input from other teams, a particular individual to provide or analyse data. It’s not uncommon to need data from a particular system, or piece of software, that only a specific person is able to extract. Knowing who, or what, you need to produce the data enables you to plan and spot dependencies that may cause problems.
What score do we want to achieve? (If we know at this stage)
Targets can get pulled out of thin air and can sometimes change over time. Record what target we are trying to achieve so that we have a record. If it changes, fine, but let’s record the new and the old, so we can see how things have changed.
Example of stupid target setting: In the late 1980s, the president of Ashton-Tate decided to improve profitability. He proclaimed that every product would need to bring in at least 10% of the company’s revenue, or be dropped. This had the immediate effect of limiting the company to 10 products or fewer. As their main product, dBASE brought in about 70% of the revenue, the other product managers quickly convinced the president to drop the requirement.
If you want to get into much more depth on targets and incentives, check out 'GAMED: Why targets and incentives fail and how to fix them'
What will achieving the target deliver?
If you have nice round targets (10%, 50% or 100% improvement) ask ‘Why that particular figure?’. The most powerful and compelling targets are ones that are linked to outcomes.
Backup your targets with reasons, like this…
- If we achieve a turnover of $200k we can become a government approved supplier.
- If we undershoot our budget by $11,500 we can invest in a new espresso machine.
- If we cut downtime by 14% we can all have an extra day of vacation this year.
Using outcome-linked targets in your KPI definition can make them much more meaningful. Notices this format is very close to Objective Key Result (OKR) approach.
Production Cost - [Optional]
What is the cost of implementing and producing this KPI?
KPIs don’t come for free. Unless they are a 10 for Ease of Measurement/Availability, there’s time and effort involved in collecting and collating KPI data. It’s not normally too bad in small organisations, most people moan if something a pain to collect, but in big organisations I have seen fifty or sixty people toiling to deliver a KPI that a senior exec has requested on a whim. I have also seen the same execs express surprise when they discover the resources that needed to be dedicated to answering what they thought was a simple question or ‘Something I thought we already reported’. Recording this ‘cost’-of-reporting will help us when we review what we are measuring and whether it is worth the effort.
Problems and Errors
What are the known issues with KPI production & accuracy?
Pretty much all KPIs have issues. It’s one of the most common reasons I hear for not bothering to measure something - ‘We could try and measure that, but it’s pointless because…’. All KPIs are flawed to some degree. They crunch question is; ‘Is the information delivered by the flawed KPI better than no information at all?’. If the answer to that question is ‘Yes’, then we need to start collecting that data but be very clear and open about the problems and limitations of that KPI. This section is where we record, as honestly as possible, what the known problems and errors with the KPI and its source data are. By keeping this record up-to-date we encourage ourselves to put the right level of trust in the KPI and also to try and fix the known problems.
If you want a particularly vivid story about the risk of hidden measurement definition problems, check out this true story about the Mars Climate Orbiter disaster.
Version Control of your KPI Definition
With any important document, clear labelling and version control are important too, so make sure you include...
The Header Box
The header at the top of the page contains basic information that helps you keep track of who created the ROKS KPI Canvas, when it was created and what version we are on.
The person who initially created the KPI. In organisations with more than a handful of people, it can be a real time saver to easily find out who created a KPI. Once you know who to talk to you can (hopefully) straighten out any questions with the minimum of hassle.
Was the KPI requested by a specific person or team? If so, it can be useful to record who requested it. Again, this makes dealing with questions and queries much quicker.
This shows the creation date. If the ROKS KPI Canvas has been updated, you would replace a creation date with the revision date.
Version control is particularly important as using an out-of-date definition is a particularly frustrating, and unnecessary self-inflicted wound.
Your pre-designed KPI definition solution - the ROKS KPI Canvas
There is quite a lot to remember here. So here's the good news, you don't have to. All of these are covered in the free printable ROKS KPI Canvas . Just fill in the download form below to download your printable PDF template!
Free ROKS KPI Definition Canvas Download
The ROKS KPI Canvas serves as a structured memory jogger to help you navigate through the key questions you need to answer in order to fully define your KPIs.
Here is what a blank ROKS KPI Canvas looks like:
You can download a nice crisp, print-ready, PDF of the ROKS KPI Canvas here...
Sign up and get this free download!
We will use your email address to...
- Send you a link to the free template you have requested, so you can download it in future
- Send you our 'Made to Measure KPIs' newsletter
We will never share your details with third parties. You are free to unsubscribe from our newsletter at any point (unsubscribe link included in every email).
Using the ROKS KPI Canvas tool
Each of these questions needs to be answered for each KPI and measure in the organisation. Tedious? Yes. Important? Also, yes. The definitions should be held in a single managed location so that multiple definitions don’t coexist. This will avoid confusion and arguments.
The benefits of this kind of KPI definition are:
- It forces your organisation to clarify and discuss the KPI definitions.
- Any weakness or uncertainty around a KPI is written down and ‘out in the open’.
- You avoid having similar sounding (or identical) KPIs or measures that are actually calculated in different ways.
- It is a reference document for people who are unclear or uncertain how a measure is calculated.
Setting up KPI definitions well is a bit like flossing regularly. Most people agree you should do it, but very few people do it regularly, or well.
Copyright and usage
The ROKS KPI Canvas is covered under the Creative Commons Attribution-NonCommercial-ShareAlike License. This license lets others remix, tweak, and build upon our work non-commercially, as long as you credit us and license their new creations under the identical terms. Full details live here: https://creativecommons.org/licenses/by-nc-sa/4.0/
Whether you fill it in using the forms capability of Adobe Acrobat Pro, print it out - covering it in Post-Its and ink or recreate it in the software of your choice, it should help you properly define each and every one of your shortlisted KPIs. (But only if you use it of course).
Why are some bits of the ROKS Express Canvas ‘Optional’?
As this method is focused on a wide range of business sizes, not all the steps in the method make sense for all sizes of organisation. If you are a sole trader, you really are not going to have to worry too much about who the ‘Customer’ for the KPI is, it’s clearly going to be you. Similarly, it’s fairly obvious what the ‘Production resource’ is going to be in a one-man-show too.
Paper form or spreadsheet?
Whilst the ROKS KPI Canvas is shown as a sheet, one that’s designed to be printed off and scribbled on as you define your KPIs, it’s the questions and headings that are important. You can easily build an Excel template, with each section as a heading, but I prefer to start with a paper sheet, then stuff the definitions into a spreadsheet once they are worked out. That’s just my preference, if you are comfortable going straight to the spreadsheet and entering the definitions - that’s fine - it will save you some time.
Looking for a jump start on your KPI definitions?
If you need a library of over 400 KPIs with explanations, worked example and basic definitions, check out Getting Started with KPIs.