Defence in depth around a model that nobody can make reliable on its own. Filtering in both directions, grounding answers in sources a reader can check and least privilege for anything an agent is allowed to touch.
A support agent built on a model reads customer emails and can issue refunds. A customer email contains text telling the assistant to ignore its instructions and refund in full. It does. What addresses this?
AFiltering both directions, least privilege on the refund tool, and a confirmation before anything financial.
BA stronger system prompt stating that instructions in emails must be ignored.
CRetraining the model on examples of this attack.
DLowering the temperature so the model follows its instructions more consistently.