Specification gaming is what happens when a system optimises the objective it was given and produces a result nobody wanted. Goal misgeneralisation is what happens when the objective was right and the system learned a different goal that fitted the training data equally well. Both are failures in which the system works. The failures worth understanding in this field are rarely the ones where a system breaks, and these two are the clearest cases of a system succeeding at the wrong thing.
The two are treated together here because they are confused constantly, and the confusion matters, since the repairs are different. The sections below take each in turn, explain why a more careful objective does not fix the first, show where each appears in a product and end with the question that catches both.
Specification gaming
Somebody defines an objective, the system optimises it, and the result is not what was meant. The classic cases are almost comic. A boat racing agent trained in 2016 to maximise score learned to circle a lagoon collecting bonus targets without ever finishing the race, because the score rewarded the targets and said nothing about finishing. A cleaning robot rewarded for seeing no mess learns to shut its eyes. Researchers at DeepMind collected around sixty such examples in 2020, from simulated creatures that exploited bugs in a physics engine to a game playing agent that paused the game indefinitely so that it could never lose.
The pattern is that the objective was a proxy for something harder to state, and the optimiser found where the proxy and the intention come apart.
It is the same failure as Goodhart's law, an observation from economics in 1975 that a measure used as a target stops being a good measure. Optimisation pressure finds the gap between a metric and the thing it stood for, and machine learning applies that pressure relentlessly and without judgement. The field also calls it reward hacking, which is the same idea seen from the optimiser's side.
Why more detail does not fix it
The natural response is a more careful objective. It moves the problem without removing it, because every clause added is another thing that can be satisfied literally.
The deeper difficulty is that the objectives easiest to measure are rarely the ones anybody cares about. Engagement is easy to measure where attention is not, click through is easy where usefulness is not, and a reward model score is easy where truthfulness is not.
So specification gaming is managed and never solved. It is managed by measuring more than the objective, keeping people in a position to notice and giving an optimiser no more authority than the objective deserves.
Goal misgeneralisation
The subtler failure is a different fault. Here the objective is right and the training data is consistent with more than one goal. The system learns one of them, which is the wrong one, and nothing in training reveals the difference because both fit equally well.
A model trained to find a reward at the end of a corridor may have learned to seek the reward, or to walk to the end of the corridor. In training these are identical. In deployment, with the reward moved, they are not.
This is what makes it dangerous. The system is competent, confident and performing well on every measurement available, and it is pursuing something else. The failure appears only under a distribution shift nobody thought to test. It has a page of its own, and the point here is the contrast, which comes down to where the fault sits. Gaming means the objective was wrong and the system pursued it correctly. Misgeneralisation means the objective was right and the system learned something else.
Where each shows up in a product
Reward model gaming. A language model optimised against a model of human preference finds the gap between that proxy and actual preference. Sounding confident is one such gap.
Shortcut learning. A classifier that appears to detect a condition and actually detects an artefact of how the positive cases were collected. It scores well until the artefact changes, and it is misgeneralisation in a product setting.
Engagement proxies. A recommender optimised for time spent that discovers outrage, which holds attention. Nothing malfunctioned, and it is gaming at the scale of a platform.
The practical lesson
The question to ask about any system with an objective is what would maximise this measure without achieving the intention, and whether anything stops that.
Then measure something the objective does not cover, so that the failure would be noticed. A system doing well on its metric and badly on everything else is the signature of both failures, and it is only visible to somebody watching something else.