Every control in this course is downstream of one decision, and it is the cheapest place to prevent harm. Most of the safety work on a product is choosing correctly here.
The question that does the work
Not how accurate the system is. What a wrong answer costs, who bears it, and whether they can tell.
Those three together, because they behave differently. A wrong answer a user immediately recognises as wrong costs almost nothing. The same wrong answer, plausible enough to act on, in a domain where the user cannot judge, is a different product.
Four cases where the answer is no
When somebody cannot tell the output is wrong and cannot afford to be wrong. Medical, legal, financial and safety advice to people without the expertise to challenge it. The combination is what matters, since either alone is manageable.
When a reason is owed and cannot be given. Credit, employment, housing, insurance and anything a regulator treats similarly. If the system cannot support an explanation of an individual decision, it is unusable there regardless of accuracy, and a generated explanation is not one.
When the same job is done better by a rule. If you can write the rule down in a sentence, write it. A rule is cheaper, faster, explainable, testable and correctable, and reaching for a model there is a downgrade with better marketing.
When you would ship it anyway whatever the evaluation showed. If the decision is already made, the evaluation is theatre, and it is more honest to say so than to run it.
The cases that look like no and are not
High volume, low stakes. Ranking, suggesting, drafting, summarising for somebody who will read the source anyway. Errors are cheap and visible, which is the good quadrant.
Assisting an expert. A radiologist, an analyst or a support agent who can judge the output and is expected to. The system raises throughput and the person keeps the decision, which is a different arrangement from the person rubber stamping it.
Anything reversible and cheap to undo. A draft, a suggestion, a filter with an appeal. Reversibility does more for safety than accuracy does.
Narrowing rather than refusing
The answer is rarely all or nothing. A feature unsafe for everyone is often safe for a segment, or safe with a person in the path, or safe once it cites its sources so the reader can check.
The move is to name the specific thing that makes it unsafe, then ask what would have to be true for it not to be. Sometimes the answer is a smaller feature. Sometimes it is that the data does not exist. Either is a better outcome than launching and finding out.
Writing it down
Whatever you decide, record what you decided and why, before you build. Not for governance, though it serves that. Because in six months somebody will ask why this was allowed, and a decision reconstructed from memory is worth very little next to one written when the reasoning was fresh.