Most writing about AI risk is about systems nobody has built yet. This is about the ways the thing you are shipping next quarter will fail, roughly in the order you will meet them.
Confident wrong answers
The model produces something plausible and untrue, in the same tone it uses when it is right. There is no signal in the output that separates the two, which is what makes it dangerous rather than merely inaccurate.
It follows from the objective. A model trained to produce the likeliest continuation optimises for plausibility, and plausibility and truth are different properties that usually coincide. Newer models do it less often and none of them stop.
Answers that are worse for some people
Aggregate accuracy hides subgroup performance completely. A system at 94 per cent overall can be at 97 for one group and 71 for another, and nobody finds out until somebody complains or a regulator asks.
The cause is usually the data. Historical records encode historical decisions, and groups thin in the training data get worse results because there was less evidence about them.
Instructions arriving as content
Anything the system reads can contain instructions. A customer email, a retrieved document, a web page. Because instructions and data share one channel, no wording makes yours authoritative, and the problem stops being theoretical the moment the system can act rather than only answer.
Misuse
People will use it for things you did not intend, including deliberately. Extracting information it should not give, generating content you would not want associated with the product, or automating something at a volume you did not design for.
Quiet degradation
Nothing breaks. The world moves away from what the model learned, the questions people ask change, the provider updates the hosted model underneath you, and answers get worse while every operational dashboard stays green.
This is the failure most likely to run for months, because the absence of complaints is not evidence of quality. Most people who get a poor answer stop using the feature rather than reporting it.
Over reliance
The one that is hardest to control, because it is not a property of the system at all. People accept output they should have checked, and they check less as the system gets better, so accuracy and complacency rise together.
Reading the list
The useful exercise is not to score a system against these. It is to take each one and ask what it would cost here.
A confident wrong answer in a brainstorming tool costs nothing. The same failure in a tool that tells somebody whether their insurance covers a procedure is a different thing entirely, and no amount of model quality changes which of those you are building.
Practise this
You need one feature of your own that is live or nearly live, the six failure modes above, and fifteen minutes.
Take each mode in turn and write the sentence describing how you would first hear that it was happening, naming the actual alert, sampling routine or report rather than the general idea of one. Then mark the row with the letter for what that answer really is. Write what happens today rather than what is planned, because a control that exists in a ticket detects nothing.
Feature ..........................................
Failure mode How I would first hear about it Source
Confident wrong answer ............................... A M U N
Worse for some people ............................... A M U N
Instructions in content ............................... A M U N
Misuse ............................... A M U N
Quiet degradation ............................... A M U N
Over reliance ............................... A M U N
A an automated alert fires
M a person looks at something on a schedule
U a user tells us
N nothing, we would not find out
Count of U ____ Count of N ____
Count the U and N rows before anything else. A U means the harm reached somebody who then took the trouble to complain, which makes your users the monitoring and makes the reported cases a small fraction of the real ones. An N is worse and commoner than teams expect, and it lands most often on quiet degradation and over reliance, neither of which produces an error, a latency spike or a complaint.
The quarter of an hour teaches you which of your failure modes currently have no detection at all, which is a shorter and more useful list than one ranking them by how much they worry you.