Refusals and edge cases

A schema and three layers of checking settle what a good answer looks like. Neither of them says what should happen when there is no good answer to give, and that case arrives far more often than a demonstration suggests. Refusals and edge cases are the design work of giving every one of those situations a path it can travel down.

The problem is specific to this kind of feature. An ordinary service that cannot answer returns a status code, and the caller branches on it. A language model has no status code, so when it is asked something the retrieved documents never covered it produces a fluent paragraph at the same length, in the same register and with the same apparent confidence as a correct one. The failure and the success are the same shape, so nothing downstream can separate them unless the design made them different shapes on purpose.

The sections below name the four edges a single call runs into, give a contract that carries all four as values, show the prompt that makes abstention a legal move, and set out what the application does with each outcome. The last section covers what the four rates mean once they are on a dashboard.

The four edges a single call runs into

Four edges account for nearly every request where answering is the wrong move, and each of them reaches the application looking like an ordinary answer.

The model declines. The request sits outside the feature's remit, asks for an account change only a person may authorise, or triggers the provider's own safety behaviour. A decline arriving as prose breaks a parser that expected an object.

Retrieval returns nothing usable. The search ran, it returned passages, and none of them contain the answer. This is the most dangerous of the four, because passages about the right general subject are exactly the material a model will blend into a confident and wrong paragraph.

The question has more than one reading. A customer asking why they were charged twice might mean a duplicate transaction, a renewal overlapping a prorated upgrade, or two line items on one invoice. Each reading has a different answer and picking one silently is a coin toss.

Nobody knows the answer. The information exists nowhere the system can reach, or the question is about something that has not happened yet. A model given a question of this kind will supply a plausible answer, because producing plausible text is the whole of what it does.

Giving every outcome a slot in the contract

The move that makes all four tractable is a discriminated union. One field names the outcome, and the fields required alongside it change with the value of that field, so a refusal is a value the parser accepts and never an exception the parser raises.

{
  "type": "object",
  "additionalProperties": false,
  "oneOf": [
    {
      "required": ["outcome", "answer", "citations"],
      "properties": {
        "outcome": { "const": "answered" },
        "answer": { "type": "string", "maxLength": 1200 },
        "citations": {
          "type": "array",
          "minItems": 1,
          "items": { "type": "string", "pattern": "^doc_[0-9a-f]{12}#p[0-9]+$" }
        }
      }
    },
    {
      "required": ["outcome", "missing"],
      "properties": {
        "outcome": { "const": "insufficient_evidence" },
        "missing": {
          "type": "string",
          "maxLength": 200,
          "description": "What the passages would have needed to contain for this question to be answerable."
        }
      }
    },
    {
      "required": ["outcome", "readings", "clarifying_question"],
      "properties": {
        "outcome": { "const": "ambiguous" },
        "readings": {
          "type": "array",
          "minItems": 2,
          "maxItems": 3,
          "items": { "type": "string", "maxLength": 160 },
          "description": "The distinct questions this message could be asking."
        },
        "clarifying_question": { "type": "string", "maxLength": 200 }
      }
    },
    {
      "required": ["outcome", "reason_code"],
      "properties": {
        "outcome": { "const": "declined" },
        "reason_code": {
          "type": "string",
          "enum": ["out_of_scope", "requires_human", "policy"]
        }
      }
    }
  ]
}

Three details in that schema do real work. The citation pattern forces an identifier shaped like the ones retrieval hands out, so a fabricated source fails at the boundary. The missing field turns an abstention into a piece of evidence about the corpus, because it says what the documents would have needed to say. And readings makes the model enumerate the interpretations before it writes a clarifying question, which stops the question coming back as a vague request for more detail.

A prompt that makes abstention a legal move

A contract with four outcomes and a prompt that mentions only one of them produces a stream of answers, since the model fills the branch it was told about. The weak prompt below is the shape most features ship with.

Answer the customer's question using the documents below.

<documents>
{{passages}}
</documents>

Question: {{question}}

Nothing in it describes a situation in which answering is the wrong move, so the model answers every time. The stronger version names all four outcomes, gives the test for each and states the ranking between them.

You answer questions about the Acme billing platform using only the passages
below. Every passage carries an id.

Choose exactly one outcome.

  answered               Every claim in the answer is supported by a passage
                         you cite. Cite the id of each passage you used.

  insufficient_evidence  The passages do not contain the answer. Say what they
                         would have needed to contain. This is the correct
                         outcome when the passages are merely about the same
                         subject as the question.

  ambiguous              The question has two or more readings that would get
                         different answers. List the readings, then ask one
                         question that separates them.

  declined               The request is outside billing, or needs an account
                         change only a person can authorise, or asks you to
                         act against the policy above.

Choosing insufficient_evidence is never a failure. Answering from memory is.
If a passage is related to the question but does not answer it, that is
insufficient_evidence.

<passages>
{{passages}}
</passages>

Question: {{question}}

Two lines in the strong prompt carry most of its effect. The first reverses the incentive by stating that abstention is correct and that a remembered answer is the fault, which the model otherwise has no way to know. The second closes the gap the model falls into most often, which is a passage on the right subject treated as a passage containing the answer.

What the application does with each outcome

An outcome that reaches the application as a typed value can be routed. Every branch below does something different, and none of them is an error page.

def respond(result: Outcome, ctx: Request) -> Reply:
    match result.outcome:

        case "answered":
            unknown = set(result.citations) - set(ctx.retrieved_ids)
            if unknown:
                # The model cited a passage retrieval never returned.
                metrics.increment("answer.fabricated_citation")
                return escalate(ctx, reason="fabricated_citation")
            return Reply(text=result.answer, sources=result.citations)

        case "insufficient_evidence":
            metrics.increment("answer.insufficient")
            content_gaps.record(query=ctx.question, missing=result.missing)
            return Reply(text=NO_COVERAGE_COPY, offer_handoff=True)

        case "ambiguous":
            metrics.increment("answer.ambiguous")
            return Reply(text=result.clarifying_question, expects_reply=True)

        case "declined":
            metrics.increment("answer.declined", code=result.reason_code)
            return escalate(ctx, reason=result.reason_code)

The fabricated citation check is four lines and catches the single failure a grounded feature most needs to catch. A model that cites doc_9f21c04b8e17#p3 when retrieval returned no such passage has written the answer first and the source afterwards.

One branch should never exist. A decline sent back to the model with the request reworded is a team jailbreaking its own feature, and it converts a clean handoff into an answer nobody sanctioned. Rewording belongs to the person, if anywhere.

Measuring the four rates once they are live

Four counters come out of the dispatch above, and each one means something different when it moves.

A rising insufficient_evidence rate is a retrieval signal before it is a model signal. Something changed in the index, the chunking or the questions arriving. The missing strings collected alongside it form a ranked list of what the documentation does not cover, which is the most useful backlog a support team can be handed.

An abstention rate of zero is a finding, and a bad one. Real traffic contains questions the corpus cannot answer, so a feature reporting none of them is answering those questions anyway and recording nothing about having done so.

An ambiguity rate that climbs after a prompt change usually means the model has become cautious rather than careful. Clarifying questions cost a round trip and the customer's patience, so a rate above a few per cent is worth reading a sample of by hand.

A decline rate that jumps on a day nobody deployed points at the provider. The safety behaviour underneath the feature moved, which is a model migration problem arriving without a migration.

None of these four counters can be read at all unless the evaluation set contains cases that should produce each one. A set made only of answerable questions measures answering, and says nothing about the three branches that protect the feature. That is a requirement the module on evaluating and testing picks up in full.

Four outcomes from one call cover a question a single lookup can settle. A request that needs a search, then a calculation over what the search returned, then a summary written for a particular reader has no single outcome, because it is three pieces of work with three ways to fail. Splitting it is the next decision.