Context rot describes a loss of useful performance as an LLM's context grows or accumulates distracting material. A model may miss a relevant passage, repeat an earlier mistake or apply an outdated instruction even though the input fits within its advertised window.
The term is informal. It covers several possible failure patterns, so diagnosis should identify the actual cause rather than treating every long session error as the same problem.
What long context research establishes
Lost in the Middle found that the location of relevant information affected performance on the models and tasks studied. Chroma's Context Rot report examined how input length, distractors and other context properties affected a range of models.
These studies support testing beyond maximum window size. They do not establish a universal length at which every model fails, nor prove that every additional token hurts. Extra context can supply missing evidence and improve an answer.
For an application, the important measurement is task success as length and composition change. Keep the answer bearing evidence constant while varying irrelevant material, placement and conversation history.
Prevent unsupported claims from becoming memory
Context poisoning occurs when false or misleading material enters the context and influences later work. It may result from an attack, an erroneous tool result or the assistant's own unsupported statement.
Consider an assistant inventing a 12% goodwill cap. It calculates 12% of 59.97 as 7.20 after rounding and stores that amount as an approved limit. The arithmetic is correct, but the policy premise is unsupported.
Later repetition makes the claim look established. Prevent that progression by storing the policy reference, observation source and verification status with the derived value. Authoritative business rules should come from controlled application data.
Reduce distraction and ambiguous tool choices
Irrelevant history can obscure the current goal. Redundant documents can consume budget without adding evidence. Several tools with overlapping names can also make selection harder.
If an agent repeatedly chooses between search_orders, find_order and lookup_order_history incorrectly, review the tool contracts. State when each tool applies, the required identifiers and the shape of its result. Offer the tools needed for the current stage when that can be determined reliably.
Removing tools is not always the answer. An overly narrow selection can hide the only valid action. Compare tool choice accuracy before and after the change.
Resolve context conflicts with source rules
A later statement is not automatically more authoritative. A current order service response may supersede a cached response, but a new customer message cannot rewrite the organisation's policy.
Define precedence by source and field. Record explicit corrections, document versions and timestamps where freshness matters. If the available evidence does not resolve a conflict, the assistant should ask for clarification or hand off the case.
For example, a summary should preserve “the customer corrected the delivery date to March 5.” It should not retain March 3 simply because that date appeared earlier or more often.
A practical long context test
Build a small suite around a task with a known answer. Run these variants:
- Supply only the necessary evidence.
- Add unrelated passages while retaining the same evidence.
- Move the evidence between the beginning, middle and end.
- Add an outdated version with a clear version label.
- Add an earlier mistaken assistant statement followed by a correction.
Grade the answer, cited evidence and any resulting action. Repeat runs when output variation could affect the conclusion. A failure only in the conflicting version case calls for a different fix from a failure caused by missing retrieval.
Choose a repair that matches the evidence
Use context selection and compression to reduce irrelevant material while preserving necessary facts. Refresh stale observations. Give long, bounded subtasks a separate context when their handoff can be verified.
Track these changes through traces. A fluent answer and a successful HTTP response do not establish task success. Conversely, an explicit statement that the evidence is insufficient can be the correct outcome. The goal is reliable use of evidence, including reliable abstention when an answer is not supported.