Goodhart's law states that a measure used to control behaviour stops being a good measure of the thing it was originally chosen to represent.
A decision rule that ships a change only when a named measure clears a threshold gives that measure real authority inside an organisation. Authority is the condition under which a measure starts behaving differently, because people now have a reason to move it by whichever route is cheapest, and the cheapest route is rarely the one the measure was meant to detect.
The observation came from monetary policy. Charles Goodhart, then an economist at the Bank of England, made it in a 1975 paper on the problems of monetary management in the United Kingdom, written for a Reserve Bank of Australia conference. His wording was about statistics and control.
Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.
Goodhart was describing what happened when the authorities began targeting particular measures of the money supply. The relationships those measures had held with inflation for years came apart as soon as the measures became instruments of policy. Marilyn Strathern restated the idea for institutions in a 1997 paper on university audit, in the form most people now quote, which is that a measure ceases to be a good measure once it becomes a target.
The sections below show the three shapes gaming takes in product work, explain why most of it happens without intent, and then set out the defences. The page closes on the part of the problem that measurement cannot reach.
The three shapes metric gaming takes in product work
Moving the number without moving the behaviour. A support organisation judged on first response time introduces an automated acknowledgement that fires within two minutes of a ticket arriving. The median first response falls from 47 minutes to 2. Time to resolution rises from 9 hours to 14, because the acknowledgement satisfied the clock and nothing else changed. Both numbers are correct and one of them now measures a robot.
Changing what counts as the event. A team judged on activation defines activation as connecting a data source. Somebody proposes connecting a sample data set automatically on first login, which is genuinely useful for demonstrating the product. Activation moves from 34 per cent to 91 per cent in one release. Week four retention does not move at all, because the definition changed and the customers did not.
Taking the gain from somewhere the measure cannot see. A growth team judged on signups adds a prompt to a flow people were already in, and signups rise by 22 per cent. Paid conversion falls, because the additional signups came from people who were not ready and now arrive at a paywall they resent. The measure went up and the business went down, which is the version of the problem that costs the most and gets noticed the latest.
Why most gaming is done in good faith
Donald Campbell described the same effect in social policy in a 1976 paper, arguing that the more a quantitative indicator is used to make decisions, the more it comes under pressure to distort and the more it corrupts the process it was meant to monitor. His framing is useful because it locates the fault in the system of measurement and never in the people inside it.
Each of the three cases above was built by somebody solving the problem in front of them. An engineer asked to reduce first response time built the acknowledgement and reduced it. A designer adding a sample data set made an empty product easier to understand. Nobody set out to deceive, and a review that treats gaming as a conduct question will find no misconduct and change nothing.
What all three share is a measure sitting some distance from the outcome it stands for. First response time stands for a customer being helped, activation stands for a customer reaching value, and signups stand for revenue. Each of these moves lives in the gap between the proxy and the outcome, and the width of that gap decides how gameable a measure is.
The defences that hold a measure honest
Four practices reduce the gap or make it visible, and a programme usually needs all four.
- A measure close to the customer outcome. A measure of what a customer achieved is harder to move by a shortcut than a measure of what the product did. Time to resolution sits closer than first response time, and second week usage sits closer than a connected data source.
- A quality measure paired with every target. Response time pairs with resolution rate. Signups pair with paid conversion. Activation pairs with retention at four weeks. The pair makes the cheap route visible in the same report, so the shortcut and its cost arrive together.
- Guardrails owned outside the team being judged. A guardrail the team controls is a guardrail that gets redefined. Latency, error rate, unsubscribes and support contacts belong to somebody with no stake in the result.
- A versioned definition with a review date. A measure that jumps by 57 points in one release has almost always had its definition changed. A dated record of how each measure is calculated turns that jump from a mystery into a one line answer.
Pairing a target with a quality measure is the defence product teams adopt most easily, because it needs no new tooling. John Doerr describes paired key results in Measure What Matters, published in 2018, on exactly this reasoning, which is that a quantity target with no quality target beside it will be met in whichever way is fastest.
What measurement cannot fix
The defences above make gaming visible and they do not remove the reason for it. A measure that decides promotions, bonuses and headcount will be moved by the cheapest available means, and adding a second measure only means two numbers now need moving.
The part that has to change is what the organisation rewards. A team rewarded for shipping features ships features. A team rewarded for moving a number moves the number. A team rewarded for a customer outcome, and asked to show evidence that its work caused the outcome, is being asked for something much harder to produce by a shortcut, since that evidence has to survive the same reading every experiment result gets.
Whether that is reasonable to ask of a team is a judgement about the organisation, and the honest answer is that most organisations reward what they can see. That brings the problem back to the analyst, because a finding nobody outside the team understands cannot change what anybody is rewarded for.