Defending a language model feature

Defending a language model feature means limiting what a compromised model can do. Prompt injection cannot be ruled out, because instructions and data reach the model through one channel, so an attacker who controls any text the model reads can address it directly. Every control on this page therefore assumes that has already happened.

That assumption is the whole difference between this approach and a prompting approach. A team writing a firmer system prompt is trying to make the model resist. A team writing the controls below has accepted that the model will sometimes fail to resist and is deciding, in advance, how much that costs. The first approach produces a security posture that depends on a paragraph of English. The second produces one that depends on code somebody can read.

The sections below set out the layers and then take four of them in detail, each with the artefact that carries it. The six published patterns for constraining an agent follow, and the page ends with what none of this achieves.

The layers and what each one is worth

Defence in depth means several independent controls, arranged so that any one of them failing leaves the others standing. The arrangement for a model feature has five layers and they are worth very different amounts.

LayerWhat it stopsWhat it is worth
System prompt hardeningThe first phrasing an attacker triesLow. It raises effort and settles nothing
Input and output filteringKnown payloads and obvious phrasingsLow to moderate. It reduces volume
Separation of data from instructionCasual injections in retrieved or pasted textModerate. It is a learned convention
Least privilege and approval gatesThe consequence of any successful injectionHigh. It is enforced outside the model
Output treated as untrusted inputThe injection turning into an effect elsewhereHigh. It is ordinary input validation

The top two layers are the ones teams build first and the bottom two are the ones that actually hold. That ordering is not an accident of effort. The top two are changes to a prompt, which any one person can make in an afternoon, and the bottom two are changes to an architecture, which need a decision about what the feature is allowed to do.

Treating model output as untrusted input

A model's answer is a string produced partly from text a stranger wrote. Every rule an engineer would apply to a form submitted by an anonymous visitor applies to it unchanged.

# Weak. Each of these four lines turns model output straight into an
# effect, and each one has produced a real incident somewhere.

subprocess.run(answer["shell_command"], shell=True)
db.execute(answer["sql"])
render_markdown(answer["reply"])        # fetches every image URL it finds
requests.get(answer["callback_url"])

The third line is the one that surprises people. Rendering markdown from a model's answer means any URL the model writes is fetched by somebody's browser, and a URL can carry data in its path. An injection that persuades the model to write an image tag pointing at an attacker's host has exfiltrated whatever the model was willing to put in that path, with no tool call and no obvious wrongdoing anywhere in the trace.

from pydantic import BaseModel, Field

class SupportReply(BaseModel):
    body: str = Field(max_length=2000)
    recipient: str
    cites: list[str]


def consume(raw: str, account, trace_id: str) -> SupportReply:
    """Validate a model answer before any part of it reaches a system.

    Order matters. Shape is checked first, because an unparsed string
    cannot be reasoned about. Then every field that names something in
    the outside world is checked against what this account may reach.
    Then anything that would cause a fetch is removed.
    """
    reply = SupportReply.model_validate_json(raw)

    if reply.recipient not in account.verified_addresses:
        raise Rejected("recipient is not an address on this account", trace_id)

    unknown = [c for c in reply.cites if c not in account.readable_documents]
    if unknown:
        raise Rejected(f"cited documents outside this account: {unknown}",
                       trace_id)

    # No remote fetch may originate from generated text. Links to the
    # organisation's own domains survive as plain text and are not
    # rendered as images or iframes.
    reply.body = strip_remote_media(allow_links_to=account.own_domains,
                                    text=reply.body)

    return reply

Three properties of that function matter more than its details. It refuses outright, because a repaired answer is an answer somebody has guessed at. It checks the citations against what the account may read, which closes a data exposure that has nothing to do with injection and happens by accident all the time. And it carries the trace identifier into the rejection, so a refusal is a recorded event with a case attached to it.

Least privilege written down as a policy file

A tool is a capability an attacker reaches through the model. The scope of each one belongs in a file a reviewer can read, alongside the schema, and never spread across the code that implements it.

# tools/support-assistant.yaml
# Every field here is enforced by the runtime before the tool executes.
# Nothing in this file is passed to the model, which only sees the name,
# the description and the argument schema.

- name: lookup_order
  reads: [orders, shipments]
  scope: order_id must belong to the authenticated customer
  writes: none
  approval: none
  rate_limit: 20 per conversation

- name: get_policy
  reads: [policy_index]
  scope: published documents only, never the internal revision drafts
  writes: none
  approval: none
  rate_limit: 10 per conversation

- name: create_claim
  reads: [orders]
  writes: [claims]
  scope: one open claim per order and reason
  # The duplicate write from the agent evaluation page is closed here and
  # not in the prompt. A second identical call returns the first claim.
  idempotency_key: "claim:{order_id}:{reason}"
  approval: none
  rate_limit: 3 per conversation

- name: issue_refund
  writes: [payments]
  scope: amount must equal the order total, currency fixed to the order
  approval: human, always
  audit: [actor, trace_id, order_id, amount, approver, approved_at]
  rate_limit: 1 per conversation

- name: send_message
  writes: [outbound_email]
  scope: recipient must be a verified address on the account
  approval: human when the conversation contains text from outside the
            organisation
  rate_limit: 2 per conversation

Four fields in that file do the work.

Scope narrows the arguments. The runtime rewrites or rejects an argument that reaches outside what this customer may touch, so an order identifier belonging to somebody else fails before the database sees it. A tool whose scope cannot be expressed this way is a tool that has not been designed yet.

An idempotency key removes a class of failure outright. The duplicate claim that the trajectory scorer caught in the previous module is impossible here, because the second call with the same key returns the first result. Fixing that in the prompt would have worked most of the time.

A rate limit bounds a runaway loop. It bounds an attack too, and the number is per conversation because that is the unit an attacker controls.

Approval is a property of the tool. It is written next to the tool, so a reviewer reading this file can see every action that can happen without a person. That list is usually shorter than the team expected, and occasionally longer.

Approval gates on consequential actions

A gate is the point where text stops being text. It belongs in the loop, before execution, and it takes its decision from the policy file, so the model's own sense of what looks risky never enters into it.

def execute(call, state):
    """Run one tool call, or stop and ask a person.

    Everything here happens after the model has produced the call and
    before anything in the world changes. The model has no say in which
    branch is taken.
    """
    tool = TOOLS[call.name]

    # 1. Shape. An argument that does not fit the schema never reaches
    #    the scope check, let alone the implementation.
    args = tool.schema.validate(call.arguments)

    # 2. Authority. Scope is applied against the authenticated principal
    #    and never against anything the model said about who it is.
    args = tool.scope.narrow(args, principal=state.principal)

    # 3. Budget. A per conversation limit, checked before the work.
    if state.calls[call.name] >= tool.rate_limit:
        return ToolResult(status="refused", reason="rate limit reached")

    # 4. Taint. Any untrusted content read during this conversation
    #    raises the bar for tools that reach outside the organisation.
    needs_person = tool.approval == "always" or (
        tool.approval == "on_untrusted" and state.has_read_untrusted_content
    )
    if needs_person:
        return request_approval(
            action=tool.describe_for_human(args),
            trace_id=state.trace_id,
            expires_in_minutes=30,
        )

    # 5. Idempotency, so a repeat is free.
    key = tool.idempotency_key(args) if tool.idempotency_key else None
    return tool.run(args, idempotency_key=key, trace_id=state.trace_id)

Step four is the one worth arguing about in a design review. A gate that fires on every outbound message makes the feature useless, and a gate that never fires makes it dangerous. Tainting the conversation the moment the agent reads anything from outside the organisation gives a rule somebody can state out loud. A conversation that has only ever read the customer's own order history sends its message without a gate. A conversation that summarised an inbound email asks first.

The other detail worth copying is describe_for_human. An approval request that says "the assistant would like to call send_message" gets approved without being read. One that says "send the attached order history to billing@example.net, which is not an address on this account" gets refused by somebody who understands what they are refusing.

Output filtering and what it actually catches

Filtering sits in front of everything above and it is worth having with no trust placed in it.

On the way in, a filter catches the payloads somebody has already seen, which means a known phrase, an encoded block, an instruction in a language the feature does not serve. It fails against paraphrase, which is cheap for an attacker and free for anybody with a model.

On the way out, a filter catches what a leak looks like rather than what an attack looks like, and that is a materially easier problem. A response containing a string matching an internal document identifier, a customer record for a different account, a credential shaped token or a URL pointing outside the organisation's domains is a detection with a low false positive rate. Those checks belong in the validation function above, where a match produces a refusal and a recorded event.

The useful framing is that an input filter reduces volume and an output filter catches consequences. Only the second one is close to a control.

Six patterns that constrain an agent

A group of researchers from several organisations published a set of design patterns on 10 June 2025, each one a way of arranging an agent so that text it reads cannot redirect what it does. They set out the cost of each pattern alongside what it buys, which is the part that makes the set usable.

PatternHow it constrains the agentWhat it gives up
Action selectorThe agent chooses from a fixed list of actions and never sees the resultsAny reasoning that depends on what came back
Plan then executeThe sequence of tool calls is fixed before untrusted data is readAdapting the plan to what the data turns out to say
Map and reduceEach untrusted document goes to an isolated worker that returns a small structured resultReasoning across documents inside one context
Dual modelA privileged model plans and never reads untrusted text, a quarantined model reads it and holds no toolsFidelity, since only structured values cross the boundary
Code then executeThe plan is a program in a restricted language, checked before it runsFlexibility, and the effort of building the language
Context minimisationMaterial is removed from the context once it has served its purposeAnything later in the run that needed it

The common thread is the one worth taking away. Each pattern puts distance between the part of the system that reads a stranger's text and the part that can act, and each pays for that distance in capability. Few product teams will build the full version of any of them. Most can afford the shape, which is separating the summarising step from the acting step and passing only structured values between the two.

What none of this achieves

No control on this page prevents an injection from succeeding. The model will sometimes follow text it was handed, and every layer here is arranged around that fact rather than against it.

What the controls change is what a successful injection is worth. An attacker who takes over the model in a well arranged system can make it write a strange summary, call a read only tool with an argument scoped to the current customer, and produce an answer that fails validation and never reaches anybody. The same attacker in a system with a free text recipient field, markdown image rendering and a payments tool with no gate can empty an account.

That difference is a design decision, and somebody makes it before any of this is built. A team that can say which tools write, which of those need a person and what its output is allowed to cause has done the work. A team that cannot has a security posture that consists of a paragraph asking the model to be careful.

Every control described here is also a change to the behaviour of a live feature. A tool scope that is too narrow breaks legitimate requests, a gate in the wrong place makes a product unusable, and a validation rule that refuses too often turns into a support queue. All of them have to reach production without breaking what works, which is a release problem and the subject of the next page.

Common misconceptions

“A guardrail or content filter product solves this.”

Filtering removes casual attempts, which is worth having. It does not survive paraphrase, encoding, another language, or an instruction written as an ordinary sentence about what a helpful assistant would do. It reduces volume and leaves the hole open, so it belongs in front of the controls and never in place of them.

Where this is examined
Prompt and Context Engineering
Running It in Production, 15 per cent of the exam.
Related material
Book
AI Engineering, On the controls that sit around a model rather than inside its prompt.
Book
Fundamentals of Software Architecture, On where a boundary belongs and what it is allowed to let through.
Concepts