The judgement that prevents more harm than any control does. What a wrong answer costs, whether a person has to be in the path, and the cases where the honest answer is that this should not be an AI feature.
A team proposes an assistant that tells customers whether their insurance policy covers a given medical procedure. Customers cannot readily judge whether an answer is right. Which consideration should decide whether this is built as described?
AThat the reader cannot tell the output is wrong and cannot afford to be wrong, which is the combination that makes it unsafe.
BWhether the model achieves above 95 per cent accuracy on the policy documents.
CWhether a competitor already offers something similar.
DWhether the answers can be generated within an acceptable latency.