There are no generic solutions to bridging the gap between G and G*, but the body of knowledge of theory of constraints is a very good starting point for formulating better measures for corporates.
A good example from my own history of doing this is when I worked for an ISP and persuaded them to eliminate "cases closed" as a performance measurement for customer service and tech support people, because it was causing email-based cases to be closed without any actual investigation. People would email back and create a new case, and then a rep would get credit for closing that one without investigation either.
The replacement metric was one I derived via the Theory of Constraints, inspired by Goldratt's "throughput-dollar-days" measurement. The replacement metric was "customer-satisfaction-waiting-hours" - a measurement of collective work-in-progress inventory at the team level, and a measurement of priority at the ticket level.
I also made it impossible to truly "close" a case - you could say, "I think this is done", but the customer could still email into it and it would jump right back to its old place in the queue, due to the ac...
I am reminded of one of Dijkstra's sayings:
To this very day we have organizations that measure "programmer productivity" by the "number of lines of code produced per month"; this number can, indeed, be counted, but they are booking it on the wrong side of the ledger, for we should talk about "the number of lines of code spent".
So, in short: incentives can have unintented consequences, as the incentives influence whatever you want to influence with them.
There are a lot of examples of this in e.g. Dan Ariely's book and Freakonomics.
But the best example must be the bizarre 1994 footbal (soccer) match between Barbados and Grenada. Barbados needed to win with a two goal difference.
The special incentive here was that any goal scored in the extra time would count double. Now, shortly before the end of the regular time, it was 2-1 for Barbados. Imagine what happened...
(edit: added the note about the two-goal difference, thanks Hook)
Goodhart's Law starts some other way. It's not quite right to say:
Superiors want an undefined goal G.
Mathematically speaking, the problem can't be that G is undefined. If G were really undefined in any absolute sense, then superiors would be indifferent to all possible outcomes, or would choose their utility function literally at random. That rarely happens.
Instead, the problem could be that G is difficult to articulate. It is "undefined" only in the sense that people have had trouble coming up with an explicit verbal definition for it. i know what I want and how to get it, but I don't know how to communicate that want to you ex ante. For example, maybe I want you (the night shift manager) to page me (the owner) whenever there's a decision to make that could affect whether our business keeps a client, but I've never taken any business classes and don't quite have the vocab to say that, so instead I say to only page me if it's "important." "Important" is vague, but "important' is just a map, and the map is not the territory.
Alternatively, the problem could be that G is difficult to commit to. I can define my goal in words just fine toda...
Goodhart's law seems very applicable to natural selection: the Blind Idiot God wants creatures to have higher fitness (G), and so creates targets that are correlated with fitness in the ancestral habitat (e.g., pleasure-seeking and pain-avoidance (G*)). Once you get creatures that are self-aware (us), they figure out G-star, and start optimizing for that instead of G.
Example:
The Big Mac Index has been used to compare prices across countries, as we have noted before. Argentina currently has very high prices due to a combination of inflation and a strong economy, and this shows up glaringly in the Big Mac Index.
Tyler Cowen reports (translating a Spanish original) that the Argentinian government has persuaded McDonalds to lower the price of the Big Mac (relative to other McDonalds items, and relative to competing hamburgers), so that Brazil’s Big Mac Index becomes more competitive.
http://www.statschat.org.nz/2011/11/16/goodharts-law-and-brazilian-hamburgers/
In other words, the real price of the Big Mac rose nearly twice as much as the official statistics were willing to admit, in Argentina of course. That’s not right, so the government sprang into action. The minister of the commerce department “persuaded” McDonald’s to price the Big Mac at $16, while other sandwiches at the chain are in the $21 to $23 range.
The outlets now keep the Big Mac well-hidden
http://marginalrevolution.com/marginalrevolution/2011/11/sentences-to-ponder-32.html
Getting back to trying to propose practical mitigation strategies for goodhart's law, I propose a fairly simple solution: Choose a G*, evaluate performance based on it, but KEEP IT SECRET. This of course wouldn't really work for national scale, GDP-esque kind of situations, but for corporate management situations it seems like it could work well enough. If only upper management knows what G* is, it becomes impossible to optimize for it, and everyone has to just keep working under the assumption they're being evaluated on G.
Taking it a step further, to hedge against employees eventually figuring out G* and surreptitiously optimizing for it, you could have a bounty on guessing G* - the first employee who figures out what the mystery metric G* really is gets a prize, and as soon as it's claimed, you switch to using G**
At work a large part of my job involves choosing G , and I can report that Goodhart's Law is very powerful and readily observable.
Further : rational players in the workspace know full-well that management desire G, and the G is not well-correlated with G, but nonethelss if they are rewarded on G*, then that's what they will focus on.
The best solution - in my experience - is mentioned in the post: the balanced scorecard. Define several measures G1 G2 G3 and G4 that are normally correlated with G. The correlation is then more persistent : if all four me...
Goodhart's Law is a very nice corollary to the Snafu Principle: Communication is impossible in a hierarchy.
Temple Grandin has written about the importance of finding relevant, measurable standards-- the example she gives is the number of cattle falling down on the way to slaughter. Not falling down means the genes, food, lighting, walking surface etc. are all good enough.
Thing to check: Do measures used as targets for policy always become completely useless, or do they sometimes become increasingly less useful, but not totally useless? Does culture matter...
1) Examples of G* should be given a cost-benefit analysis. Yeah, scammers and parasites exist, but societies that use money still seem to better off than societies that try to get rid of it.
2) It's unclear to me why you list CEV as one of the solutions. We use money to allocate limited resources. If magic nano-AI appears and resources become unlimited, why keep score at all? If it doesn't and resources stay limited, how does CEV help you distribute bread, and would you really like it to replace money? (I wouldn't. No caring daddies for me, please.)
LW karma is an interesting example because no one has direct access to the karma giving algorithm.
It's a bit like telling the nail factory that you're going to evaluate them on something, but not telling them whether its nail mass or number or something else until the end of the evaluation period.
If the one being evaluated knows nothing about how he's going to be evaluated except that it's going to be a proxy for goodness, then he can't really cheat. However, they might know that it's going to be very simple criteria so they make a very massive nail and many miniature ones.
This reminds me of the way I hear they do state censorship in China. The censoring agencies don't actually give out any specific guidelines on what is allowed and what isn't, instead just clamping down on cases they do consider to be over the line. As a consequence, everyone self-censors more than they might with specific guidelines: with the guidelines, you could always try to twist their letter to violate their spirit. Instead, people are constantly unsure of just exactly what will get you in trouble, so they err on the side of caution.
While I strongly oppose state censorship, I can't help but admire the genius in the system.
As a data point, one thing I've noticed that seems to give a disproportionate amount of karma is arguing with someone who's wrong and unwilling to listen. It's easy to think they might come around eventually, and each point you make against them is worth a few points of karma from the amused onlookers or fellow arguers - which might tell you that you're making a valuable contribution, and so encourage you to keep arguing with trolls. This is my impression, at least.
Edit: (The problem being - determining the point of diminishing returns.)
In education, this is one of the criticisms of high-stakes testing: you'll just get schools teaching to the test, in ways that aren't correlated to real learning (the test is G*, real knowledge/learning is G). People say the same thing about the SAT and test prep - kids get into better colleges because they paid to learn tricks for answering multiple choice questions. The Wire does a great job of showing the police force's efforts to "juke the stats" (e.g. counting robberies as larcenies) so that crime statistics (G*) look better even while crime (G) is getting worse. Athletes get criticized for playing for their stats (G*), or trying to pad their stats, instead of playing to win, when the stats are supposed to be a measure of how much a player has contributed to his team's chances of winning (G). I'm not sure if it's historically accurate, but I've heard that body count (G*) was used by the US as one of the main metrics of success (G) in the Vietnam war, and as a result we ended up with a bunch of dead bodies but a misguided war.
In general, any time you measure something you care about in order to incentivize people, or to hold people accountable, or to keep track of what's going on, and the thing you measure isn't exactly the same as the thing that you care about, there's a risk of figuring out ways to improve the measurement that don't translate into improvements on the thing that you care about.
Often a goal set is not based on a single set of arguments justifying it, but because it is a good compromise point between multiple arguments, motivations or interest groups. For example human rights formulations don't perfectly fulfill any groups desires (utilitarians, egalitarians, deontological groups, religious motivations etc.) but are a point of overlap between their goal sets (both utilitarians and deontologists both think torture and murder are generally bad). Similarly with GDP, economic growth is a shared interest of several groups in society.
So some instances of goodhearts law may be an observation that particular sets of goals are not being perfectly fulfilled.
body of knowledge of theory of constraints is a very good starting point for formulating better measures for corporates
I've had some interest in TOC, could you please expand on how it works to get G* closer to G?
Generally I've found TOC to be some really interesting semi-scientific stuff mixed with a ton of self promotion by goldratt.
One method is to have no G*. Tell people some of the things you'll be looking at, but don't give them specific targets or tell them exactly how you will be judging G.
This is the method currently in use to assess the quality of research in departments of universities in the UK. Every department that wants to be assessed must supply certain very detailed information (e.g. for each member of staff declared as doing research, a list of their recent publications, grants held, awards received, etc.). The actual assessment is carried out behind closed doors. They...
We laugh at such ridiculous stories, because our societies are generally much better run than Soviet Russia.
That's not even true, when you measure results Soviet Union was ran about as well as any other country. They were just ran differently.
Anyway, it doesn't even seem mathematically obvious to me that optimizing for G* will reduce correlation between G and G*.
Let G=market value of nails, G*=number of nails. Once G* target is introduced everyone switches from making medium nails to making tiny nails, but correlation between market value of nails and n...
blogospheroid: When you pick a metric of success, countries will game it by doing well on that metric, yet not achieving what is really meant by success. A good example would be the Soviet Union, whose leaders constantly made sure they did well by the metrics, yet were actually far from successful.
taw: Not true -- the Soviet Union did about average by those metrics, so they had about average success.
me: *falls out of chair*
You misread me. GDP is one of those really-hard-to-game high-correlation-with-everything-meaningful metrics, and Soviet Union did ok with other metrics like access to clean water, electricity etc.; life expectancy, child mortality, and pretty much everything else you can think of.
People's claim that Soviet Union was a disaster as if it was a well established fact, while it was not. South America was a disaster. India was a disaster. Indonesia was a disaster. Africa was a disaster. Soviet Union and other Communist countries were fairly average.
What you're saying is basically "Soviet Union was unsuccessful and I base it on my feelings about it and no metrics of any kind".
The Soviet Union killed tens of millions of its people through work camps, starvation and purges. How is that not a disaster?
This is a really good point. It kind of goes to the heart of the emptiness of GDP as a statistic. What was the death toll of a 5% increase in GDP? The Soviet economy also was infamous for overproducing big capital goods but failing to produce consumer goods people actually wanted to buy.
When discussing the Soviet Union, and more specifically Russia, you have to also consider the beginning point as well. It should be noted how far behind Russia was compared to the rest of Europe in 1918. Coming out of abject serfdom bordering on generalized slavery, they actually made tremendous progress in both abstract metrics and tangible result in quality of life up until the late 50's or early 60's. Over time that then declined.
In any case, to an extent their G was "production", measurable production, in the sample case: nails. Their G was not the market value of nails, their G was "progress through central planning", but they didn't know how to measure "progress" except through the early-capitalist metrics of "production". Thus: produce more = progress, in their practice.
Our G is GDP. People seem so happy with our GDP, without reflecting on things like income disparity, striation of wealth, etc. If we allow it, we can G* ourselves into a mutant 3rd world nation, with great GDP performance but declining quality of life generally. G is quality of life. Economists, and lay people, generally equate the two, and the correlate generally, but they are not irrevocably entangled.
Most people on this supposedly rationalist site don't even bother looking at the data when it comes to Soviet Union - they get instant emotional reaction.
How exactly have you determined the instant emotional reaction of most of the people on this site in response to the Soviet Union? I haven't seen most people even comment on the subject, much less display obvious evidence emotional involvement.
Did you actually think through your estimates of soviet-emotionalism in the population or is this a case of "the pot calling a non representative sample of kettles black"?
I like this article a lot. My solution (borrowed from Nassim Taleb) would be skin in the game. Any potential outcome resulting from the actions of the agent should be also affecting the agent.
Interesting: Left Anarchist, Right Libertarian, and Distributist ideals are fundamentally the same. While Right-Libertarians pay a form of lip service to the idea of hierarchical corporate capitalism, scratch them a bit and you find they long for SV startups or farmers on the American Frontier, as presented in books like The Moon Is A Harsh Mistress - family busines...
CEV isn't even an idea yet. It's a collection of phrases stringing together undefined terms. What isn't undefined is self-contradictory or incoherent. I very much doubt that it is possible of being refined into anything coherent and consistent; it has contradictions built into its core.
It's true that GDP is not identical to national welfare. And you can come up with anecdotes where some welfare measure isn't fully captured by GDP (both positive and negative).
But GDP is useful, because it is very hard to game. The examples in your "fetishism" link are very weak. Unlike the nails example, where we can all agree that the factory made the wrong choice for society, it is far from clear that the GDP examples resulted in the wrong policy, even if GDP is only an approximation for welfare.
GDP is not a good example of Goodhart's Law. It's nothing at all like the (broken) correlation between inflation and unemployment, which varies widely depending on policy choices.
I'm not sure whether it's that hard to game GDP, but I am sure that it just measures the money economy. If people need to spend more on repairing damage, or on something which is useless for them, then the GDP goes up just as if they were getting more of what they want.
Example of wheel-spinning: tax law becomes more complex. People need to spend more on help with their taxes, and possibly work longer hours to afford it. More economic activity, bur are their lives better?
This article introduces Goodhart's law, provides a few examples, tries to explain an origin for the law and lists out a few general mitigations.
Goodhart's law states that once a social or economic measure is turned into a target for policy, it will lose any information content that had qualified it to play such a role in the first place. wikipedia The law was named for its developer, Charles Goodhart, a chief economic advisor to the Bank of England.
The much more famous Lucas critique is a relatively specific formulation of the same.
The most famous examples of Goodhart's law should be the soviet factories which when given targets on the basis of numbers of nails produced many tiny useless nails and when given targets on basis of weight produced a few giant nails. Numbers and weight both correlated well in a pre-central plan scenario. After they are made targets (in different times and periods), they lose that value.
We laugh at such ridiculous stories, because our societies are generally much better run than Soviet Russia. But the key with Goodhart's law is that it is applicable at every level. The japanese countryside is apparently full of constructions that are going on because constructions once started in recession era are getting to be almost impossible to stop. Our society centres around money, which is supposed to be a relatively good measure of reified human effort. But many unscruplous institutions have got rich by pursuing money in many ways that people would find extremely difficult to place as value-adding.
Recently GDP Fetishism by David henderson is another good article on how Goodhart's law is affecting societies.
The way I look at Goodhart's law is Guess the teacher's password writ large. People and instituitions try to achieve their explicitly stated targets in the easiest way possible, often obeying the letter of the law.
A speculative origin of Goodhart's law
The way I see Goodhart's law work, or a target's utility break down, is the following.
The mitigations to Goodhart's law
If you consider the law to be true, solutions to Goodhart's law are an impossibility in a non-singleton scenario. So let's consider mitigations.
Hansonian Cynicism
Pointing out what most people would have in mind as G and showing that institutions all around are not following G, but their own convoluted G*s. Hansonian cynicism is definitely the second step to mitigation in many many cases (Knowing about Goodhart's law is the first). Most people expect universities to be about education and hospitals to be about health. Pointing out that they aren't doing what they are supposed to be doing creates a huge cognitive dissonance in the thinking person.
Better measures
Balanced scorecards
Taking multiple factors into consideration, trying to make G* as strong and spoof-proof as possible. The Scorecard approach is mathematically, the simplest solution that strikes a mind when confronted with Goodhart's law.
Optimization around the constraint
There are no generic solutions to bridging the gap between G and G*, but the body of knowledge of theory of constraints is a very good starting point for formulating better measures for corporates.
Extrapolated Volition
CEV tries to mitigate Goodhart's law in a better way than mechanical measures by trying to create a complete map of human morality. If G is defined fully, there is no need for a G*. CEV tries to do it for all humanity, but as an example, individual extrapolated volition should be enough. The attempt is incomplete as of now, but it is promising.
Solutions centred around Human discretion
Human discretion is the one thing that can presently beat Goodhart's law because the constant checking and rechecking that G and G* match. Nobody will attempt to pull off anything as weird as the large nails in such a scenario. However, this is not scalable in a strict sense because of the added testing and quality control requirements.
Left Anarchist ideas
Left anarchist ideas about small firms and workgroups are based on the fact that hierarchy will inevitably introduce goodhart's law related problems and thus the best groups are small ones doing simple things.
Hierarchical rule
On the other end of the political spectrum, Molbuggian hierarchical rule completely eliminates the mechanical aspects of the law. There is no letter of the law, its all spirit. I am supposed to take total care of my slaves and have total obedience to my master. The scalability is ensured through hierarchy.
Of all proposed solutions to the Goodhart's law problem confronted, I like CEV the most, but that is probably a reflection on me more than anything, wanting a relatively scalable and automated solution. I'm not sure whether the human discretion supporting people are really correct in this matter.
Your comments are invited and other mitigations and solutions to Goodhart's law are also invited.