I learned something new yesterday. I heard about the "Cobra Effect". When Britain ruled India there was a problem identified with the number of [venomous] cobra snakes in the country. The government decided to offer a reward for every dead cobra brought to the authorities. Initially it had the desired effect, reducing the number of wild cobras. After a while people got wise. They started breeding cobras in captivity, giving them an income without having to tap into the dwindling supply of wild cobras. The authorities got wise to this and stopped paying the bounty. The newly created "cobra farms" released their, now worthless, cobras into the wild. So by the end of the exercise government money had been wasted and the cobra population boosted by deliberate farming.
This sounds almost comic in its unintended effect, but it's remarkably similar to some of the ill-thought-out measures I see in use today.
How do you avoid this? The best approach is to ask the teams that you will be measuring. Share the measure(s) and ask "How could you make the measure go the 'right' way whilst missing the point of the measure completely?". As long as there's some trust in the group you will normally get some really good (and surprising) answers. If you dig a little deeper the group will normally come up with a much better set of measures to achieve your goals.
If you are serious about creating a balanced set of measures that can be instantly understood it might be time to look at creating an Index KPI. Here's a detailed article talking about the pros and cons of Index KPIs and how to create one.