An agent loop appends to a transcript on every pass, and that transcript is the whole of what the agent knows. Nobody designed it, because it simply grows with each observation until a budget stops the run or the window fills. Memory and state are the work of replacing that accident with a structure somebody chose.
The vocabulary comes from two directions and it helps to keep them apart. State is what the system holds while one run is in progress, which is the transcript, the plan and whatever the tools returned. Memory is what survives a run and reaches the next one, which is a fact about a customer, a preference somebody stated or a record of what the agent did last Tuesday. The first is a buffer and the second is a store, and treating either as the other produces a recognisable kind of failure.
The sections below take the transcript as the state nobody designed, give the compaction that keeps it inside a budget, cover writing state outside the window entirely, and then move to memory across runs, its write policy and the reading that makes it usable.
The transcript as the state nobody designed
At the end of a seventeen step run the window holds everything the agent has done, in the order it did it, at full length.
Before compaction, at step 17
system prompt 400
tool definitions 1,100
the goal 60
steps 1 to 14 38,900 observations and reasoning
steps 15 to 17 6,600
------
47,060 tokens
Three faults sit in that column and none of them is the total. The failed attempt at step 4 is still present at full length, so the model reads a stale error on every later pass. The search result at step 7 holds fifty rows and the agent used one of them. And the account id established at step 2, which is the single most load bearing fact in the run, is buried thirty thousand tokens back among material that competes with it for attention.
Cost makes the same point from the other side. Every pass resends the whole column, so the seventeenth step costs several times the first, and the pages on context rot and on cost between them explain why a long window makes both problems larger rather than smaller.
Compaction and the trigger that fires it
Compaction replaces the middle of the transcript with a summary and keeps the two ends. The ends are chosen deliberately. The head carries the instructions and the goal, which nothing may paraphrase. The tail carries the last few steps verbatim, because the model is usually in the middle of something.
COMPACT_AT = 45_000 # tokens
KEEP_VERBATIM = 3 # most recent steps, two messages each
PINNED_KINDS = {"identifier", "confirmed_fact", "constraint"}
def compact(transcript: list[Message], pinned: list[Fact]) -> list[Message]:
if count_tokens(transcript) < COMPACT_AT:
return transcript
head = transcript[:2] # system prompt and goal
recent = transcript[-KEEP_VERBATIM * 2:]
middle = transcript[2:-KEEP_VERBATIM * 2]
summary = model(SUMMARISE_PROMPT, steps=middle,
model=SMALL, schema=StepSummarySchema)
# Identifiers and confirmed facts are copied across word for word. A
# summary that paraphrases an account id has destroyed it.
carried = [f for f in pinned if f.kind in PINNED_KINDS]
metrics.observe("agent.compaction",
before=count_tokens(transcript),
after=count_tokens(head) + len(carried))
return head + [system(render(carried)), assistant(summary.text)] + recent
The prompt that writes the summary is the part that decides whether compaction helps or quietly breaks the run.
Summarise the steps below for an agent that has to continue this task.
Cover three things, in this order.
1. What has been established as fact, and which tool established it.
2. What has been tried and failed, and the exact error each attempt
returned.
3. What is still outstanding.
Copy every identifier, amount and date exactly as written. Never paraphrase a
number. Do not include anything you are inferring from what you read.
Write at most 200 words.
<steps>
{{steps}}
</steps>
The second item is the one teams leave out, and leaving it out has a specific consequence. An agent holding a summary of only what worked keeps no record of the four things that failed, so it tries them again, spends its budget and ends at the step cap having made no progress since the compaction.
After compaction
system prompt 400
tool definitions 1,100
the goal 60 pinned, never summarised
pinned facts 180 account id, plan, dates confirmed
summary of steps 1 to 14 1,400 written by a small model
steps 15 to 17 verbatim 6,600 the recent window stays exact
------
9,740 tokens
The run continues on a fifth of the tokens, and the facts that matter are now near the front of the window where they compete with very little.
Writing state outside the window
Compaction is lossy by construction, so anything that must survive it belongs somewhere other than the transcript. This is the write move from the page on the four things a team can do with a context budget, and in an agent it becomes a pair of tools.
[
{
"name": "write_note",
"description": "Record a finding so it survives compaction and is available to later steps. Use this for anything you would be unable to recover by calling a tool again. Overwrites any note with the same key.",
"parameters": {
"type": "object",
"additionalProperties": false,
"required": ["key", "value"],
"properties": {
"key": { "type": "string", "maxLength": 40 },
"value": { "type": "string", "maxLength": 500 }
}
}
},
{
"name": "read_notes",
"description": "List the keys and values written during this run.",
"parameters": {
"type": "object",
"properties": {},
"additionalProperties": false
}
}
]
Two properties make this worth the two tools. A note lives in the application's own storage, so it survives compaction, a crash and a process restart. And the notes are small and keyed, so the agent reads the three it needs at a cost of a hundred tokens where the original observations cost twelve thousand.
The same mechanism covers a large artefact. An agent producing a long document writes it to a file and passes the path, so the document itself never enters the window at all. What the agent carries is a reference, and the tool that reads it can return one section at a time.
Memory that survives a run
Everything so far dies when the run ends. Memory is the part that does not, and it is a store with a schema rather than a longer transcript.
{
"type": "object",
"additionalProperties": false,
"required": ["id", "subject", "kind", "statement", "source",
"written_at", "basis"],
"properties": {
"id": { "type": "string" },
"subject": {
"type": "string",
"description": "The account or person this memory is about."
},
"kind": {
"type": "string",
"enum": ["preference", "account_fact", "episode"]
},
"statement": {
"type": "string",
"maxLength": 200,
"description": "One sentence, written so somebody can read it alone in six months and understand it."
},
"source": {
"type": "string",
"description": "The message id or tool result this came from. Required, so any memory can be traced to where it started."
},
"written_at": { "type": "string", "format": "date-time" },
"expires_at": {
"type": ["string", "null"],
"format": "date-time",
"description": "Null only for something that cannot change, such as the date an account opened."
},
"basis": {
"type": "string",
"enum": ["stated", "inferred"],
"description": "stated means the customer said it. inferred means the agent concluded it."
}
}
}
Three fields in that schema carry the weight. The source field makes every
memory traceable, so a wrong one can be followed back to the moment it was
written and the write rule that allowed it can be fixed. The basis field
separates something a customer said from something an agent worked out, which
matters because the second kind is a guess wearing the clothes of a fact. And
expires_at exists because most memories are perishable. A preference stated
eighteen months ago describes a person who may have changed their mind, and an
account fact describes a state of the world that has moved on.
The three kinds are worth keeping separate for the same reason. A preference says how somebody wants the organisation to treat them. An account fact says something about their situation. An episode records what happened in a past interaction, which is the kind that stops an agent asking the same question in three consecutive conversations.
The write policy and the fact that never expires
An agent asked to decide what to remember remembers far too much, because writing a memory costs it nothing and it has no view of the next twelve months. Four rules keep the store usable.
Nothing gets written from a single ambiguous message. A customer saying they are in a hurry today is not a customer who prefers short answers.
Anything inferred is labelled as inferred. An inferred memory can be read and it never outranks a stated one, and the labelling is what makes that possible.
A system of record wins. Where the customer relationship system holds the billing address, the memory store does not hold a second copy of it. A memory duplicating an authoritative source is a cache that nobody invalidates.
Everything gets an expiry unless it genuinely cannot change. The date an account opened is permanent. Almost nothing else is.
The reason these rules earn their place is that a memory fault outlives the
conversation that created it. A poisoned context is one bad run. A poisoned
memory is every run afterwards, each one reading the wrong statement and acting
on it, until somebody traces a customer complaint back to a sentence written
nine months earlier. That is context poisoning with a much longer life, and the
source field is what makes the trace possible at all.
Reading memory as a retrieval problem
Loading every memory for a customer into the window undoes the whole exercise, since a store of four hundred statements is another undifferentiated block competing for attention. Reading is a retrieval problem and it takes the same treatment as any other.
The agent retrieves by subject first, then by relevance to the current goal, and takes a small capped number. The selected memories then enter the window inside a labelled block of their own.
<memory subject="acct_4b71c9e20d8af315">
stated, 2026-03-04, expires 2027-03-04
Prefers email over telephone for anything about billing.
stated, 2026-07-19, no expiry
Finance contact is Priya Raghavan, who approves any credit note.
inferred, 2026-08-02, expires 2026-11-02
Appears to run a monthly close in the first week of each month.
</memory>
Statements above are what this system has recorded previously. They may be
out of date. Anything marked inferred was concluded by an earlier run and was
never confirmed. Where a memory conflicts with a tool result, the tool result
is correct.
The closing instruction is the one that stops a stale memory beating a live lookup. A model shown a remembered plan name beside a plan name returned by the billing tool will otherwise pick whichever reads more confidently. The memory always reads more confidently, since it arrives as a flat statement while the tool result arrives as a row in a table.
Memory keeps one agent coherent across a long run and across the runs that follow it. Neither compaction nor a memory store does anything about a task whose material genuinely cannot fit in one window, or about a subtask whose noisy intermediate work would pollute the window of everything around it. That takes a different move, which is giving the subtask a window of its own.