Measuring progress and performance is fundamental to almost any organized human activity. But there's an interesting paradox when we try to use those measurements to improve things: the moment we turn a metric into a target, it tends to lose its value as an indicator. This phenomenon, known as Goodhart's law, is an economic adage that goes something like this:
When a measure becomes a target, it ceases to be a good measure. 1
It became popular during Margaret Thatcher's government, thanks to Charles Goodhart's contribution to a critique of the monetary policy of the time. In its original formulation:
Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes. 2
Even though it doesn't have the status of a natural law, it has been referenced and re-stated many times across fields like sociology, education, and risk analysis... and once you understand it, it seems like common sense — yet it's often ignored when setting goals and measurements in many domains.
What does it mean?
Goodhart's law isn't a natural law because it doesn't happen spontaneously. It's not part of physics or mathematics; it's a phenomenon that only appears in the presence of a certain degree of intelligence. If you give a thermostat a target temperature, it makes sense that the thermostat measures the temperature every X seconds and adjusts its power based on the difference between the current temperature and the target...
But what if, instead of a thermostat, you have a person in charge of regulating the temperature, and you pay them based on how long they spend regulating it? If you tell that person the goal is to reach temperature X, they might decide to take longer on purpose so you get paid more... So you change the target: you pay them the same regardless of how long it takes, as long as they reach the target temperature. Anyone in that position would crank the heating to maximum — but that's probably too much unless you stop before reaching the target temperature...
The person, as an intelligent individual with their own goals, won't just optimize the goal you gave them. In most cases, the goal being measured is an intermediate goal or a subgoal toward a larger end (getting paid), so if there are ways to reach it that optimize the final goal at the expense of the intermediate one, they'll take them.
This example seems silly — the task is too simple and we could specify it better. But it's what happens every time you set a goal and a way to measure progress for an intelligent system. It happens in education with assignments and exams, where students study to get a better grade rather than to learn more. It happens in politics, where candidates optimize for staying in office instead of serving citizens or thinking long-term. On the web, clickbait and misinformation only chase higher impression counts, instead of informing or entertaining. It happens with company KPIs, or code quality metrics: ask for 100% test coverage and you'll get tests that don't really verify the code does what it should — they just chase running every line.
History is full of documented cases where metrics produce unexpected, counterproductive effects. One of the most famous happened in colonial India, when the British government, worried about the cobra population, offered a bounty for every dead cobra. The result: people started breeding cobras to claim the reward. When the government realized and cancelled the program, the breeders released all the cobras, making the original problem significantly worse.
In the modern corporate world, there's the case of Wells Fargo in 2016. The bank set aggressive targets for the number of accounts opened per employee. The result: employees opened millions of fake accounts without customers' knowledge, leading to massive fines and severe reputational damage.
In education, the No Child Left Behind Act in the US made standardized tests the main metric for evaluating schools and determining their funding. The result: many schools started "teaching to the test", cutting time from unexamined subjects like art or music, and in some cases there were even documented cases of score manipulation.
You don't need much intelligence or complex systems for Goodhart's law to kick in. It also happens with children, with animals... There are hundreds of stories and anecdotes in the same vein — you can probably think of one. For example, there's a story about dolphin trainers at an aquarium who decided to teach the dolphins to pick up trash that fell into the tank. In exchange for bringing a cup, a bag, or a bottle that had fallen in, they gave the dolphin a fish. To get more fish, the dolphins started tearing the trash into smaller pieces so they could bring it in more trips.
It's also one of the main arguments that make AI alignment difficult. In that field it goes by other names: specification gaming or reward hacking, and there are hundreds of examples too. Anyone who has trained an ML system will tell you that if you're not careful about cleaning your data and designing the training properly, the model will pick up "heuristics" or "shortcuts" to get the expected results.
The underlying problem is always the same: a metric can be optimized in many ways, and that metric is usually a proxy — a goal close to the real one, but not the real goal:
In most cases we don't know the final goal, or we don't know how to reach it, or it's very hard to measure.
In the students-and-exams example, it would be better to personally interview each student to understand how far they've really internalized the material — but that's very expensive and prone to bias, which brings its own auditing problems... So most teachers fall back on assignments and exams, and we all know studying for an exam is not the same as mastering the subject.
Another typical example is customer support: if you reward the operator who handles the most customers, workers will optimize for short calls, not for solving the customer's problem. If you reward the one who stays on the phone longest, they might just chat the customer up instead of solving their issue. How do you measure that calls are professional, solve the problem, and don't go off the rails? Many companies fall back on customer satisfaction scores — and many workers will just ask the customer to please give them a good rating...
Whenever you take a metric and make it a target for an agent, that agent will tend to hit the target, regardless of your original intentions. It's the story of King Midas and the genie. The genie will always give you what you asked for, not what you meant.
But metrics are useful — how else would we know where we stand? If we want to track our progress toward a goal, we have to measure it somehow. The problem isn't measuring; it's turning the metric into a target, rewarding the agent in some way for improving that metric. With humans, the reward can even be just seeing the metric go up — most people feel satisfaction from knowing they're doing a good job. So how can we fight Goodhart's law?
How to avoid Goodhart's law
There are several strategies that help prevent Goodhart's law from hacking our metrics. I'll explain them from least to most useful, based on my experience:
Make it clear the goal is not to optimize the metric
For me, this is the least effective strategy with people in a work environment. The moment you create a metric, the people affected will try to improve it — whether because they think it's the right thing to do, because they think they'll be rewarded, or just to feel good about themselves. Even if we explain that the metric is just a way to take a measurement, not the final goal, everyone reads it as: "If I make this metric go up, I'm working in the right direction" — even when that's sometimes not true.
Back to the code coverage example: if our repo shows the current coverage, raising it is seen as improving the quality of the test suite, and lowering it as making it worse — even though we all know the relationship isn't direct, and you can perfectly well raise coverage with tests that don't actually test what we want and therefore don't improve code quality.
Keep the metric secret
If agents don't know the metric exists, they can't optimize it. This can be harder than it sounds with adult humans — we're very good at detecting this kind of thing, and even if we say nothing but reward the people who improve the chosen metric, the rest of the team will watch that person's behavior and interests to try to get the same reward.
This can be even more harmful than the first case, because now you have a proxy of a proxy. Imagine a teacher assigns a written essay and doesn't specify the grading criteria. The teacher intends for students not to focus on presentation or formatting details, but on the substance of the subject. Now suppose the student with the best essay — in content, readability, understanding of the material — also happens to hand it in bound with an illustrated cover. When grades come out, all the students see that the one who handed in the booklet with a cover got the best grade, so they assume binding and covers matter. Next assignment, many more students will make an illustrated cover and bind their work, even if they put the same or less effort into the content.
Change/review the metric often
Changing the metric forces the dynamics built around it to change. Reviewing it also lets us find the one that gets us closest to the goal at any given moment. This is exactly where AI companies trying to reach Artificial General Intelligence (AGI) find themselves. The proxies here are the benchmarks these companies use. Since LLMs can hold human-level conversations, the Turing test seems passed; on top of it, other benchmarks appeared years ago like Winograd, MMLU, or GSM8k. When those benchmarks became "saturated" in turn (models score at human level), they kept looking and building other benchmarks, like the ARC AGI Challenge or SWE-bench.
None of these benchmarks — or the many others used so far — truly measures every aspect of human intelligence, but adding new benchmarks keeps the measurement going and stops you from getting stuck at a local optimum, or from not knowing whether the models are really improving.
Use several metrics for the same goal
This strategy is similar to the previous one; in fact it's also what AI companies use to measure their new models. They don't use just one benchmark, but many. And usually a new model will be the best on some of them, but not all. Or maybe it's very good at one or two, but below the latest models on all the others.
When metrics don't saturate — that is, when we can keep improving them — we can have several at once. This stops us from falling into heuristics that optimize just one metric, and it will probably make the solution generalize better and keep us working closer to the final goal.
This is also what some teachers do when they measure students not just by a final exam, but through several exams and assignments of different kinds, giving a more complete picture of the student's learning.
Conclusion
Goodhart's law reminds us of something fundamental about the nature of measurement and goals: reality is always more complex than our metrics. Whether in education, business management, software development, or even artificial intelligence, we need to be aware that any measurement system can be "hacked" if it becomes the main target.
The solution isn't to stop measuring — metrics are valuable tools that help us understand and improve our systems. The key is to use them intelligently: combining multiple metrics, reviewing them periodically, and above all remembering that they are rough indicators of what we actually want to achieve, not the goal itself.
Top comments (1)
Your point that the reward can simply be watching the number go up is the version I run into with habit tracking. Nobody is paid for a streak and it still gets gamed. It is easy to end up ticking the smallest possible version of a habit late at night just so the count survives, and the habit quietly shrinks while the metric looks fine.
For one person tracking themselves, several metrics for the same goal is the strategy that holds up. Keeping it secret obviously cannot work when you are the one keeping score.