LLM model migration replaces a model or serving configuration while preserving the behaviour the application requires. It may follow a retirement notice, a cost change, a new capability or an evaluation showing that another model performs better.
Treat migration as a system change. Prompts, sampling controls, context accounting, tool protocols and llm structured output support may behave differently on the candidate.
Inventory every model dependency
The visible assistant may be only one dependency. Classifiers, summarisation steps, embeddings and automated graders may use separate models.
Record each model's identifier, endpoint, owner, purpose and replacement constraints. Track retirement information through the provider's official notices and the organisation's service agreement. Notice periods vary by product and release type.
Where available, use a version identifier that supports controlled change. A pinned version still needs lifecycle monitoring; it does not guarantee indefinite availability.
Check the API contract before comparing quality
Confirm supported message roles, sampling settings, output limits, tool call representation, streaming events and refusal handling. Validate the candidate's schema support using the actual application schema.
A request that succeeds syntactically can still behave differently. For example, a new output budget may include reasoning usage, or a model may require a different tool result message format.
Changing an embedding system is a separate migration concern. Query and document representations must remain compatible, often requiring reindexing and a cutover plan.
Run a controlled model comparison
Start with the current prompt and data configuration to isolate the model difference as far as possible. Run the same evaluation cases and graders against both models.
Record task success, segment regressions, abstention, tool correctness, latency and full cost. Repeated trials help when output variation matters.
If the candidate needs prompt adaptation, treat that as a second experiment. Evaluate the adapted model prompt pair on fresh held out cases before release. A prompt that transfers poorly does not prove the model is unsuitable, but tuning against the release set can hide the problem.
Test the failure paths again
Include invalid input, missing evidence, provider refusal, truncation, rate limiting and tool failures. Confirm that the application interprets the new response statuses correctly.
For agents, test the execution path as well as the final answer. A model may select a different valid sequence, while a parser or fixture assumes the old sequence.
Use agent evaluations and contract tests to separate legitimate behavioural differences from integration failures.
Roll out with a usable fallback
Use the same release discipline as a prompt deployment, with immutable configuration, reviewed evidence, limited exposure and monitored promotion.
Verify that the fallback remains available during the migration window. Rolling back to a retired endpoint is not a recovery plan.
If shadow testing uses live requests, ensure that the candidate cannot duplicate writes. Route its proposed actions to fixtures or a non executing path, and account for the additional data processing and cost.
Exercise: write a migration acceptance table
Create rows for API compatibility, task quality, critical segments, tool safety, latency, cost and fallback availability. For each, record the required evidence and acceptance condition.
Then simulate one incompatible response shape and one retired endpoint error. Check that the product reaches a deliberate fallback rather than treating an empty response as a valid answer.
Finish with a release decision that states which configuration was tested and what remains uncertain. Model names alone are not enough to explain a migration.