How focusing on the wrong numbers can kill your sustainability

14 April 2016

Man with abacus

Real life experience, and some of the best research, confirms one basic truth about us. People follow incentives. And the simpler and clearer the incentive is, the more this is true.

The most powerful incentives are those that can be recorded in numbers, especially the sort of numbers that have a currency symbol attached. But almost any number will do.

For instance, I’m a member of a tennis club that uses the ‘tencap’ system. Players get their performance recorded, and their relative rating compared to their peers moves up and down accordingly. It’s a real measure of progress, and a number of the members become heavily focused on improving their tencap rating.

It’s actually quite a dodgy number. The methodology for producing it is overcomplicated. Sometimes you get bizarre outcomes. Any system that tries to factor in lots of different factors and turn it into a single number is prone to creating some counter-intuitive results.

The players see the occasional anomalies. It doesn’t matter. The existence of the number still focuses the attention like nothing else.

Some of the sustainability measures that actually represent real progress are really difficult to measure

When people see performance measured in numbers, they focus on the numbers, not the methodology that produced them.

This applies in corporate social responsibility as much as anywhere else. This can be a positive thing, but the danger for the opposite is obvious as well.

I remember attending the event to launch the results of the very first Business in the Community environment index – the predecessor of what is now the CR Index. I was accosted by one particularly angry individual who showed me the league table published in the Financial Times and insisted that it could not be the case that his company was towards the bottom while his competitor was rated so highly.

That index had one very positive impact. It pushed the issues onto a number of boardroom agendas for the very first time. That simple equation – “we’re down here, and they’re up there” – captured the attention of a number of competitive-minded CEOs.

That was a good thing, obviously. But only so long as the order of companies in the league table represented a broadly correct outcome. For that, it was important that the methodology behind the index was right. So if I company changed its operations to fit the requirements of the index, it would be performing better in the real world as a result.

Because it really mattered. Some heads of environment or CSR were given the target of achieving a higher score in the index as one of the key performance criteria that dictated the size of their pay packet. You can believe that those individuals followed the incentives energetically. Some of them noted that the process didn’t always result in the best environmental outcome given the complex nature of the choices their particular industry faced. That being the case, they might moan about it to people like me, but they still followed the incentives.

We can agree amongst ourselves that the CEOs who set those targets were wrong to do so. They were chasing the wrong measures because they didn’t understand what real performance measures might look like. We hope that things have moved on considerably in the decades that have passed since then.

But the perverse incentives may now be different, but they do remain. Why? Because some of the sustainability measures that actually represent real progress are really difficult to measure. So we come up with proxies. Can’t measure the impact of a programme to help homeless people? Well, measure the numbers of people that go through the programme. It’ll give you the broad measure.

And indeed it will. Until the point where you make it a target. When you measure a thing and put a target on it, you change the nature of the thing.

Suddenly people are motivated not to help the homeless, but to get people onto the programme. You have a perverse incentive to shove people onto the programme even if they’re not ready for it. And you don’t have much motivation to make sure they stay engaged once they’re on the programme. Suddenly, the mere fact of measuring the wrong thing begins to make the programme less and less effective.

And then when it comes to the broader sustainability agenda, there can be other problems to do with the scope of where you look.

It may make intuitive sense to you that, for instance, a motor company should focus on reducing the emissions from its vehicles. And therefore, setting targets for a year on year improvement in performance in this area is the right thing to do. But that sort of approach encourages, and really depends upon, incrementalism. It doesn’t set the scene for the disruptive market innovation represented by, for instance, the emergence of Tesla.

The trick is to redefine the challenge. It isn’t to reduce emissions. Or to get people through a community programme. Or to get a percentage of women or ethnic minorities onto the board. It’s to create a system for organising the company that builds the right culture, and incentivises people to care about outcomes. The financial success of the business. The sustainability of the company and its impact on communities. It’s about systems design.

For all that some companies may claim that sustainability “is part of our DNA” the truth is that only a few have actually understood the breadth of ambition behind that particular claim, and the counter-intuitive way that focusing too much on the obvious numbers can actually undermine it.