The Tyranny of KPIs

It always daunts me to see major events unfolding when I’m traveling. The Berlin wall fell on 11/9/89 when I was on a school-trip in Texel (north of the Netherlands). This year, on September 14th, while I was flying over the pond, the French finally came out with a long awaited report about how GDP is a flawed measure of “happiness”.

Yes, yes, many things happen in between but honestly, this is a major one for me and I’ll try to explain why.

French President Nicolas Sarkozy, while leading the EU’s French presidency for it’s 6 months period, announced in 2008, he wanted to appoint a commission to determine alternative measures of well-being for society.
This comment made me think a lot about the only country I know off that actually does try to measure “happiness” of it’s people and that is Bhutan, through the notion of Gross National Happiness (GNH), a term coined first by Bhutan’s former King Jigme Singye Wangchuck back in 1972.
I had forgotten Sarkozy’s call to revisit GDP when The Economist reminded me of the completion of the report and the presentation of it’s findings: Measuring what matters.
Head by Joseph Stiglitz, surrounded by 25 prominent social scientists (amongst which one of my teachers in Macro economics) and five with Nobel prizes in economics, the 292-page report is a call to abandon “GDP Fetichism”.

Why is this important?
With the increase in environmental accidents, economists already raised the issue of externalities and how these could affect GDP positively while actually harming society overall. The Erika disaster back in December 1999 off the coast of Brittany made the French heavily aware of GDP measurement flaws, just like the Americans found out 10 years before that, on the shores of Alaska following the Exxon Valdez oil spill.
Accountability for such externalities have also been the basis for the Carbon trading initiatives, that I had discussed back in 2006 (Setting Pollution benchmarks), but unfortunately, it seems that as the price for carbon trading is still so low, it’s still cheaper to pollute than to invest in green energy: A collapsing carbon market makes mega-pollution cheap!
My hope for a better world had withered away with time as I didn’t see the price of polluting actually going up – February 2009 was an all time low with 8,20 €/tons, down from an all time high at 32,80 € in April 2006. Let’s hope the Copenhagen discussions about climate change will bring fresh winds of change.

Of course, externalities is just one axis of why GDP is a flawed measure. It always daunts me when I read about the industrial age, the mills in the North of the UK, the robber barons in the U.S. and all that preceded to come to indeed a wealthier society but at which expense? so many lives were spilled…
Within the current debate about climate change and how this is affecting our lives and will certainly affect more dramatically the lives of the generations to come, it’s getting more and more important to rediscuss how data is being collected, which KPIs are being defined to make decisions that are based upon them and this on a global scale.

And the French being the French, they also see the evolution of society in a different light than their American counterparts.
Wasn’t it Alexis de Tocqueville in Democracy in America who brilliantly noted that while the American society is based upon personal vertical evolution within society, aka the American Dream, the French revolution had brought along a more horizontal view of distribution of wealth amongst the population at a given time.
Note that the initial draft of the United State’s Declaration of Independence stated “Life, liberty, and the pursuit of happiness” as three aspects listed under the undeniable rights of man. Interestingly enough, this is often mistaken by an alternative version, which would be “Life, Liberty and Property”.
The initial tripartite motto is comparable to the French “Liberté, Egalité, Fraternité” but where Fraterny might oppose the US translated premise of Property.

The backbones of our economic thinking and values are thus, I would argue, radically opposed in this sense and the way social security is set-up in Europe for example, compared to the U.S., is a partial result of diversity of opinion about how a society should view evolution and well being of it’s citizens.

The metric of GDP was invented in 1942 in the U.S., during the Second World War, following the data available at the time and highly influenced by it’s surrounding context.
Those of us who are in the business of doing Web Analytics – have walked the road as a wise entrepreneur would say – know how highly influenced the choice of a metric or KPI can be (read politically if you want). And those of us who think of themselves as Web Analytics ninjas, listening to the wise voice of Indian humility, also know that no metric or KPI should be set in stone.
So this idea of revisiting the very foundation of decision making in our globalized world might not be such a bad thing after all if we were decided to continuously strive towards the pursuit of truth and happiness. For all involved.

Or as the Commission puts it:
“What we measure affects what we do; and if our measurements are flawed, decisions may be distorted. Choices between promoting GDP and protecting the environment may be false choices, once environmental degradation is appropriately included in our measurement of economic performance. So too, we often draw inferences about what are good policies by looking at what policies have promoted economic growth; but if our metrics of performance was flawed, so too may be the inferences that we draw.”
Section 3. of the Executive summary

Christine Lagarde calls this the happiness Commission, la Commission du Bonheur in French.

Here you’ll find the synthesis of the report in English (14 pages, a good night’s read!)

The Report by the Commission on the Measurement of Economic Performance and Social Progress itself in English.

Some recommendations from the report:

1. When evaluating material well-being, look at income and consumption rather than production;
2. Emphasize the household perspective;
3. Consider income and consumption jointly with wealth;
4. Give more prominence to the distribution of income, consumption and wealth;
5. Broaden income measures to non-market activities;
6. Quality of life depends on people’s objective conditions and capabilities. Steps should be taken to improve measures of people’s health, education, personal activities and environmental conditions. In particular, substantial effort should be devoted to developing and implementing robust, reliable measures of social connections, political voice, and insecurity that can be shown to predict life satisfaction;
7. Quality-of-life indicators in all the dimensions covered should assess inequalities in a comprehensive way
8. Surveys should be designed to assess the links between various quality- of-life domains for each person, and this information should be used when designing policies in various fields;
9. Statistical offices should provide the information needed to aggregate across quality-of-life dimensions, allowing the construction of different indexes;
10. Measures of both objective and subjective well-being provide key information about people’s quality of life. Statistical offices should incorporate questions to capture people’s life evaluations, hedonic experiences and priorities in their own survey;
11. Sustainability assessment requires a well-identified dashboard of indicators. The distinctive feature of the components of this dashboard should be that they are interpretable as variations of some underlying “stocks”. A monetary index of sustainability has its place in such a dashboard but, under the current state of the art, it should remain essentially focused on economic aspects of sustainability;
12. The environmental aspects of sustainability deserve a separate follow- up based on a well-chosen set of physical indicators. In particular there is a need for a clear indicator of our proximity to dangerous levels of environmental damage (such as associated with climate change or the depletion of fishing stocks.).

The Commission hopes that this Report will provide the impetus not only for this broader discussion, but for on-going research into the development of better metrics that will enable us to assess better economic performance and social progress.

Something hopefully to ponder about, independent of any acquisitions or changing in life expectancy of the page tag.
So, would do you think? Can KPIs induce tyranny and what are the factors that prevent them from evolving?

Posted Tuesday, September 29th, 2009 | 5 responses | Add a Comment | Share, Save or Email