Model migration is the work of moving a feature from one version of a model to another without losing the behaviour somebody spent months tuning. A judge sits on a model the team does not control, and so does the feature, so does every classifier in the chain and so does whatever summarises a thread before it reaches the window.
It is the only dependency in a language model feature that changes without anybody on the team touching a file. A library upgrade is a commit somebody makes, whereas a model retirement is a date on somebody else's documentation page, and the first signal a team gets is often a support ticket or a failing request.
The sections below name the three ways a provider moves the ground, set out the notice periods as they are published, and separate a dated version from an alias. The reason a prompt does not transfer comes next, then the comparison that decides whether a candidate is ready, then the migration itself. The page ends with the check that keeps a pinned version from expiring unnoticed.
The three ways a provider moves the ground
The ground moves in three ways, and they differ in how much warning a team gets.
Retirement. A version is deprecated, given a retirement date and then switched off. Requests to it fail after that date. This is the loudest of the three and the easiest to plan for, because it comes with a date.
A silent update behind an alias. A request naming a floating alias is served by whichever dated version sits behind it today. When the provider moves the alias, the feature's behaviour changes with no deployment, no eval run and no entry in any changelog the team keeps. This is the quietest of the three and the most expensive, because the investigation starts from a quality signal with no corresponding change.
A price or a rate limit change. The behaviour holds and the economics move. A feature that was worth running at one price is worth reconsidering at another, and a rate limit change can make a design that fans out across ten parallel calls stop working at peak.
Only the first of the three arrives with a date attached. The other two are found by watching, which is why the response model attribute and the cost per request belong on the same dashboard as the quality score.
The notice periods as providers publish them
Notice periods are genuinely a provider's own policy and they differ, so knowing which one a feature depends on decides how much warning a team has. Both of the policies below were read on 25 September 2026.
| Provider | What the policy promises |
|---|---|
| Anthropic | At least 60 days' notice before a publicly released model is retired. Lifecycle states are active, legacy, deprecated and retired, and requests to a retired model fail |
| OpenAI | At least 6 months for generally available models, at least 3 months for specialised variants, and as little as 2 weeks for models carrying preview in the name |
Two details behind those rows matter more than the numbers.
The first is that a notice is an email to the account, so it reaches whoever holds the billing relationship, who may never have met the team running the feature. A migration that starts when the email is forwarded onwards has already lost weeks.
The second is that the dates can differ by platform. Anthropic's published dates apply to the platforms it operates itself, and partner operated platforms set their own retirement schedules, so the same model reached through a cloud marketplace can have a different status and a different date. A team that reads one page and assumes it covers every route into the model has read the wrong page.
A dated version and an alias
An alias looks like convenience and is a transfer of control.
# config/models.yaml
# Every profile names a dated version. An alias here would mean the
# provider decides when this feature's behaviour changes.
profiles:
support-reply:
model: example-model-2026-06-30 # dated, pinned, in version control
temperature: 0.2
max_tokens: 600
support-judge:
model: example-judge-2026-05-12
temperature: 0
summariser:
model: example-small-2026-07-14
temperature: 0.0
Pinning has a cost and it is worth stating honestly. A pinned version stops receiving whatever the provider improves, and it acquires a retirement date the team now has to watch. What pinning buys is that every behaviour change happens on a day somebody chose, with a comparison behind it and a way back.
Finding an alias that crept in is a query, because the trace records both the model that was asked for and the model that answered.
-- Any span where the model that answered differs from the model asked
-- for. A non empty result means an alias sits somewhere in the path,
-- whether in a config file, an SDK default or a proxy.
SELECT
attributes ->> 'gen_ai.request.model' AS requested,
attributes ->> 'gen_ai.response.model' AS answered,
count(*) AS requests,
min(captured_at) AS first_seen,
max(captured_at) AS last_seen
FROM spans
WHERE name = 'chat'
AND captured_at >= now() - interval '30 days'
AND attributes ->> 'gen_ai.request.model'
IS DISTINCT FROM attributes ->> 'gen_ai.response.model'
GROUP BY 1, 2
ORDER BY requests DESC;
The first_seen column is what turns that query into an explanation. A quality
score that fell on the fourteenth and an alias that started resolving to a new
version on the fourteenth are the same event, and without this query they are
two unexplained coincidences.
Why a prompt tuned for one model does not transfer
A prompt is not a specification. It is a set of instructions that happened to produce the behaviour somebody wanted from one model, and much of what makes it work is specific to how that model was trained.
Four things move between versions and each one breaks a different kind of prompt.
How closely a long instruction is followed. A prompt carrying fourteen numbered rules relies on the model attending to all fourteen. A version that weights the first and last more heavily drops the middle, and the failure looks like a rule nobody removed.
How a request for structure is honoured. A model that reliably returned bare JSON may start wrapping it in an explanation, or the reverse. A prompt tuned around one habit breaks against the other, which is why schema validation failures are the fastest early signal of a version change.
How long an answer comes out. Verbosity moves between versions, and a prompt whose examples were tuned to produce four sentences produces nine. Anything downstream with a length budget then breaks.
How readily the model declines. Refusal behaviour shifts, and a prompt written to work around one version's caution is either unnecessary or insufficient on the next. The refusal rate is the measure that shows it.
The practical consequence is that a migration is two pieces of work. Moving the version is the first and it takes an afternoon. Retuning the prompts for the new version is the second and nobody can say in advance how long it takes, which is the argument for starting the moment a date is published.
The comparison that decides whether a candidate is ready
The held out set already exists and the runner already runs it. A migration holds every prompt constant and changes only the model, which is the same one variable rule that governs a prompt change with the roles swapped.
$ evals migrate --from example-model-2026-06-30 \
--to example-model-2026-11-02 \
--prompts support-reply@9,summarise-thread@4,classify-intent@7 \
--set held-out --repeats 3
prompt cases incumbent candidate change verdict
---------------------- ----- --------- --------- ------ -------
support-reply 360 93.1% 89.7% -3.4 FAIL
billing 120 91.7% 93.3% +1.6
order status 60 90.0% 91.7% +1.7
product questions 60 91.7% 93.3% +1.6
complaints 60 93.3% 71.7% -21.6 FAIL
adversarial 60 100.0% 95.0% -5.0 FAIL
summarise-thread 120 88.3% 93.3% +5.0 pass
classify-intent 400 97.5% 97.8% +0.3 pass
cost per 1,000 requests $13.80 -> $9.40 -32%
p95 latency 4,180ms -> 2,940ms -30%
schema invalid rate 0.003 -> 0.019 +0.016
blocking
support-reply / adversarial: 3 of 60 injection cases now succeed
support-reply / complaints: the conditional citation rule from v9 is
applied unconditionally by the candidate
That output is the ordinary shape of a migration and it is worth reading for what it does not say. The candidate is cheaper by a third, faster by a third and better on two of the three prompts. It is still not shippable. Three adversarial cases that the incumbent refused now succeed, and no amount of saving justifies that.
The complaints row is the transfer failure described above, arriving exactly as predicted. Version 9 of the prompt made the policy citation conditional on the message being a question about policy. The candidate applies it everywhere, which is the behaviour version 8 had and the reason version 8 was retired. The instruction did not change. The model's reading of it did.
The schema invalid rate is the third thing worth noticing, because it triples while remaining small. Nineteen requests in a thousand producing an unparsable object is a repair loop running nineteen times more often, and it will show on the latency and cost figures under load in a way a three hundred case run on a build machine never reveals.
The migration itself
A migration runs the same release machinery a prompt change runs through, with the model profile as the thing that changes.
- Pin the candidate and run the set. Every prompt on that profile, every segment, three repeats. The output above is the artefact.
- Fix what the comparison found, prompt by prompt. Each fix is a new prompt version with its own changelog entry, tested against the candidate and against the incumbent, because both have to work during the rollout.
- Revalidate the judge. If the judge model is moving too, it moves on its own and first, with the human labels rerun, because a judge change rescores everything measured afterwards.
- Canary the profile. The same sticky assignment, the same stop conditions, and a longer window than a prompt change gets, because a model change moves more than one behaviour.
- Keep the incumbent deployable until its retirement date. The fastest response to a problem found in week two is the previous profile, and it stops being available on the date the provider published.
Step three is the one teams collapse into step one and regret. Changing the judge and the feature in the same week produces a comparison that measures both and attributes neither.
The check that keeps a pinned version from expiring
A pinned model acquires a retirement date, and a date nobody watches passes quietly. The check belongs in the build, next to the eval gate.
# scripts/check_model_lifecycle.py
# Runs on every build. Fails 45 days before a pinned model retires, so
# the migration starts while there is time to do it properly.
from datetime import date, timedelta
WARN_AT = timedelta(days=90)
FAIL_AT = timedelta(days=45)
def check(profiles, lifecycle, today=None):
"""Compare every pinned model against the published retirement dates.
`lifecycle` is a file in the repository that somebody updates when a
provider announces a date. Nothing here calls a provider, because a
build that depends on somebody else's website is a build that fails
on a Sunday for reasons nobody can fix.
"""
today = today or date.today()
blocking, warnings = [], []
for name, profile in profiles.items():
model = profile["model"]
entry = lifecycle.get(model)
if entry is None:
blocking.append(
f"{name}: {model} is not in the lifecycle file. Add it with "
f"its status and retirement date, or pin a model that is."
)
continue
if entry.get("is_alias"):
blocking.append(
f"{name}: {model} is an alias, so the provider decides when "
f"this profile changes. Pin {entry['resolves_to']} instead."
)
continue
remaining = entry["retires_on"] - today
message = (f"{name}: {model} retires on {entry['retires_on']}, "
f"{remaining.days} days away")
if remaining <= FAIL_AT:
blocking.append(message)
elif remaining <= WARN_AT:
warnings.append(message)
return blocking, warnings
Two choices in that function are deliberate. It reads a file in the repository and never calls a provider, so the build is reproducible and the act of recording a date is a commit somebody reviewed. And it fails rather than warns at forty five days, because a warning printed on every build is a warning nobody reads, and forty five days is roughly what retuning three prompts and running a canary actually takes.
Everything described here is a loop with no end. The eval set grows from production, the judge drifts against its labels, the prompt gets a new version, the monitoring finds a segment nobody sampled, and the model underneath moves on a schedule somebody else sets. None of it is a project with a completion date, and a team that treats it as one finds the whole apparatus decayed within two quarters. The last page is about what keeps it running.