A capability evaluation is a measurement of what a model can do, as distinct from what it will do when asked politely. It is harder than it sounds, because the answer depends as much on how the evaluator asked as on the model itself. Every laboratory releasing a large model now runs evaluations before release, several governments have set up institutes to run their own, and the results are the evidence behind every claim that a model is or is not safe to release.
The sections below separate the three questions an evaluation can answer, explain why a negative result is weak evidence, cover public benchmarks and their limits, describe the dangerous capability evaluations behind safety frameworks, and end with what transfers to a product team.
The three questions an evaluation can answer
An evaluation can answer one of three questions, and the three need different methods.
Whether the model is good at the task. This is how well the model performs on work resembling what the team wants from it. It is the question a product team needs answered. Public benchmarks approximate it badly.
What the model can do at all. This is the upper bound of the model's ability under the most favourable conditions an evaluator can arrange. It is the question a safety assessment needs answered, because a safety assessment is about what an adversary could make the model do.
How the model behaves. This is whether the model refuses what it should refuse, stays consistent from one request to the next and holds up when a user pushes it. Behaviour is a property of training and says nothing about capability.
Mixing the three produces confusion. A model that declines to help with something has been measured on its behaviour, and that measurement says nothing about whether it could have helped.
Why a negative result is weak evidence
The question of what a model can do at all is the hard one to answer, because the same model can appear incapable or capable depending on how hard the evaluator tries. Several choices raise the result. A better prompt raises it, and so do a few worked examples placed in the prompt. Asking the model to reason step by step raises it again. Giving the model tools raises it, and so does letting it make several attempts and keeping the best. Each of those is a choice the evaluator makes, and together they can turn a model that failed a task into one that passes it.
A capability finding is therefore a finding about the model and about the method used to test it, taken together. A result showing that the model did the task is strong evidence, because the model demonstrably did it. A result showing that the model failed, which evaluators call a negative result, is weak evidence. It shows only that this method in the hands of this evaluator failed to make the model do the task, and somebody with more effort or better tools may succeed where this evaluator did not.
This imbalance between the two results is why the frontier laboratories spend serious effort on elicitation, meaning the work of drawing a capability out of a model, before they declare a capability absent. It is also why a model that appears safe in a quick test is evidence of very little.
Public benchmarks and their limits
Public benchmarks are the cheapest evaluation and the most quoted. A benchmark is a published dataset of questions with a published scoring method, covering knowledge, reasoning, coding or safety. Every model can be run against the same set, so benchmarks make models comparable.
Two limits apply. The first is contamination. A benchmark on the open internet is plausibly in the training data of every model released after it, so a high score may be partly recall of the answers. The second is construct validity, which is the question of whether a test measures what it claims to measure. Doing well on a curated set of questions differs from doing well on the messy requests a product meets, so a high benchmark score says only that the model is worth testing on the team's own work.
A benchmark produces a shortlist, and the team decides on its own cases.
Dangerous capability evaluations
Frontier laboratories now test specifically for abilities that would be harmful if present. Three categories come up in every framework. The first is help that materially increases what an attacker could do in a cyber attack or in a biological or chemical one. The second is manipulation and persuasion. The third is autonomy, meaning whether a model can acquire resources, copy itself or operate unsupervised over long periods.
The reason these abilities are tested separately from behaviour is that capability is durable and willingness is not. Refusal training can be removed by further fine tuning, so a laboratory that knows what its model could do knows what an adversary would have after removing the refusals.
Dangerous capability evaluations are what safety frameworks are built on. The first such framework was published in 2023 and the others followed within a year, and all of them work the same way. A threshold is defined in advance, the model is tested against it, and crossing the threshold triggers measures agreed beforehand, with no discussion needed at the time. The United Kingdom and the United States set up institutes in 2023 and 2024 to run evaluations independently of the laboratories, so that the claim of safety is checked by somebody other than the party making it.
What transfers to a product team
Three parts of this transfer to a product team. The team builds an evaluation set of its own cases, because no public benchmark describes its work. It tries hard to draw out a failure before concluding that the failure is absent, since a negative result from a lazy attempt means nothing. And it reruns every evaluation at each model version, because capability arrives unevenly and last month's result does not carry forward.