Prompt versioning records which instructions and configuration produced a response. Prompt deployment moves a reviewed version into use through a controlled release process.
A prompt change can alter decisions, tool use and refusal behaviour even when the application code is unchanged. Treat it as a behavioural change with an owner, evidence and a recovery path.
Version the full release configuration
A release record should identify the prompt template, examples, model configuration, output schema, tool definitions and relevant retrieval settings.
{
"release": "support-2026-09-25.1",
"prompt": "support-v12",
"model_profile": "support-model-v4",
"schema": "support-response-v3",
"tools": "order-tools-v6",
"retrieval": "policy-index-config-v8",
"eval_run": "eval-184",
"owner": "support-platform"
}
These are illustrative identifiers. Resolve them to immutable artefacts or snapshots. A versioned database registry can serve this purpose as well as files in source control.
Record the resolved configuration in each trace. A moving alias by itself may be insufficient to reconstruct a historical request.
Compare the candidate with the baseline
Run the required regression evaluation before rollout. Review new failures, critical segments and resource changes.
When the purpose is to isolate a prompt change, hold the model and other components constant where possible. Some changes must ship together, such as a new schema and its parser. Evaluate that bundle explicitly.
An evaluation result should refer to the exact candidate being released. Editing the prompt after the evaluation invalidates that correspondence.
Use a canary with meaningful comparison
A canary exposes a limited cohort to the new version while retaining a baseline for comparison. Assign consistently at the appropriate level, often a session, user or tenant, so one workflow does not mix incompatible behaviour.
Google's canary release guidance explains controlled exposure and evaluation during rollout. For LLM features, include behavioural quality alongside service health.
A small cohort still needs enough cases and important task coverage. Do not promote solely because a short observation window contained no visible errors.
Define stop and promotion criteria
Specify the thresholds, sample requirements and observation period before rollout. Critical violations may trigger an immediate stop; noisier quality estimates need sufficient labelled examples.
Compare like cohorts and account for differences in task mix. A canary handling easier requests can look better even if the model behaviour has not improved.
Monitoring should identify the release version so an alert can be attributed and acted on.
The release workflow has two decisions. Offline evidence determines whether limited exposure is justified; observations from that exposure determine whether the release should expand.
Rehearse rollback and compatibility
Rollback should restore a known configuration quickly. It cannot unsend an email or undo every action already taken. Track and reconcile consequential effects separately.
Confirm that the previous version can still operate with current stored data and schemas. If a release changes memory format or tool contracts, include compatibility handling or a migration plan.
Long running sessions need a defined policy. Finish under the old release, migrate state or restart with an explicit handover. Avoid accidental mixing.
Exercise: prepare a release packet
Create a candidate manifest, attach its evaluation report and define a canary cohort. Write the exact conditions for promotion, pause and rollback.
Then rehearse restoring the previous configuration in a test environment. Verify that a new request resolves to the old version and that an in progress workflow follows the documented policy.
The packet is complete when another engineer can identify what will change, judge the evidence and carry out the rollback without reconstructing decisions from a chat thread.