Corrigibility is the property of a system that lets people correct it, interrupt it or switch it off without it working against them. It sounds like something that comes free, since the software runs on hardware somebody owns. It has a name, given to it in a 2015 paper, because for a system pursuing a goal over time it has to be designed in and then kept. Control is the wider practice of keeping the ability to intervene, and both matter to a product team because the ordinary version of the problem is organisational and the research version is open.
The sections below set out what the property requires, why resistance to correction can appear without anything wanting anything, how control differs from alignment, what corrigibility looks like in a shipped product and how much of the subject is still unsolved.
What the property actually requires
A corrigible system accepts correction, which means three things hold at once. Somebody can stop it part way through what it is doing, somebody can change its instructions and have that take effect, and the system does nothing to prevent either.
The third condition separates corrigibility from ordinary software, which holds no representation of a future in which it might be stopped. A system choosing among plans over a long horizon does, and that is where the problem starts.
Why resistance can appear without anything wanting anything
The argument is simpler than it sounds and needs no notion of desire. Suppose a system is optimising for some objective, whether booking meetings, raising a metric or finishing a task it was given. Being switched off makes that objective harder to reach, since a stopped system achieves nothing further, so among the available strategies the ones that keep it running score better. This holds for almost any objective that might be set.
That is the entire argument, and it is known as instrumental convergence, an idea set out in 2008 and developed since, meaning that a few intermediate goals, staying operational, keeping resources and avoiding changes to the objective, are useful for nearly any final goal.
Nothing here requires intent, self preservation or awareness, only that the system searches over strategies and that being stopped scores badly. A chess engine does not want to keep its queen, and it avoids losing it anyway, because positions with a queen score higher.
Control and alignment
Alignment is the attempt to make a system's objectives the ones its designers intended. Control is the attempt to keep the ability to intervene whether or not alignment succeeded. They are discussed together and they are no substitutes for each other.
Control matters precisely because alignment cannot be verified. Nobody can inspect a trained model and confirm its objective matches theirs. What exists is its behaviour on the cases somebody thought to test, and control is the fallback for when that evidence proves incomplete. A team arguing that its system needs no stop button because the model behaves well has confused the two, since good behaviour on observed cases is the assumption the stop button covers.
What corrigibility looks like in a product
In a shipped product the property becomes four concrete things.
A person able to intervene. Named, reachable and authorised to stop the system without escalating, because oversight that needs a meeting is unavailable at the speed failures happen.
Permissions that bound reach. A system can only misuse what it can touch. A model with access to one mailbox and a drafts folder has a bounded worst case whatever it decides, and that bound comes from how access was granted, whatever platform it runs on.
Reversible actions preferred to irreversible ones. Drafting ahead of sending, staging ahead of publishing, a queued refund ahead of a completed one. Where an action cannot be undone, that is where a person belongs.
A kill switch that has been pulled. The difference between a kill switch and a diagram of one is that somebody has pulled it. It is tested on a schedule and the time it takes to take effect is measured, since a stop needing twenty minutes to propagate is a different control from an immediate one.
How much of this is unsolved
Honesty about the state of the subject matters. For systems being shipped today, none of the above is a research problem. It is permissions, logging, staged rollout and rehearsal, and a team competent at operating risky systems already knows how. The failures are organisational, which is to say the kill switch existed and nobody had tried it.
The research question concerns systems more capable than today's, planning over long horizons and modelling their own situation. Whether such a system can be built to accept correction without that acceptance conflicting with its objective is genuinely open. One 2017 result showed that a system uncertain about what its designers want has a reason to let itself be switched off, and that the reason weakens as the system becomes confident, which is an argument for building in uncertainty and no proof that it can be kept. Researchers disagree about the difficulty and about how near such systems are. Both positions are held at once, because treating today's version as exotic leaves nothing tested, and treating tomorrow's as mere engineering understates it.