How to Roll Out a New LLM Model Version Safely

A new LLM version changes how your production system behaves, even if the model name is the only setting you change. It might be more accurate overall and still return invalid JSON, call different tools, refuse valid requests or fail cases that the old version handled correctly.

Roll it out in stages. Compare the new version with the old one on your own test cases, then run it on copies of live requests without showing its answers to users. After that, send it a small share of live traffic and increase that share only while it keeps passing your release gates. Prepare and test the rollback before you start.

Write down what the feature must do

Before you test anything, write down what the LLM feature must do, what it must never do and which limits it has to meet. A public benchmark cannot tell you whether a model meets these requirements, because they depend on the prompts, retrieval, tools, permissions and parsers around the model, and on your users.

For each supported task, specify the output schema, how the model should use retrieved documents and citations, which tools it may call with which arguments, and how it should handle unclear or incomplete requests. Also cover when it refuses or escalates to a person and how it keeps context across turns. Set limits for latency, timeouts and cost per request, and decide what happens during outages and rate limits.

Turn each important requirement into a test you can measure. A customer-support feature might require schema-valid output, a valid ticket category, an answer grounded in company documentation when that documentation exists, and escalation when the request needs human approval.

Give separate risks separate release gates. A gate is a threshold the new version has to pass before it moves to the next stage. For example, you might require at least 99.5% schema-valid output in a critical use case, zero unauthorized destructive tool calls in adversarial tests, and no increase in critical-severity policy violations. The right numbers depend on how much harm a failure causes, whether it can be undone and what it costs the business.

What the feature must never do includes following instructions hidden in retrieved content, revealing the system prompt or credentials, leaking data between users or tenants, making unauthorized tool calls, and accepting requests built to run up cost. The OWASP Top 10 for LLM Applications is a good way to organize these threats.

The model should never be the only authority for an external action such as sending a message, changing a record, approving a payment or deleting data. Put a deterministic authorization check in front of each of these actions, and add human approval where an action is sensitive.

Write down which user data goes into the model's context, which provider receives it, how long traces are kept and who can read evaluation samples.

Record the current setup

Keep an exact record of the version that serves production today. It is your baseline for the comparison and the target for a rollback. Include:

• Model name, provider, endpoint and exact snapshot, if the provider offers one
• System and developer prompts, inference parameters, tokenizer and safety settings
• Tool definitions, permissions and schemas
• Retrieval setup, including the embedding model, index, reranker and document corpus
• Preprocessors, parsers and post-processors
• Routing rules, user cohorts, serving framework and infrastructure image
• Judge model, rubric and test-set version used for evaluation
• Current latency, error, cost, refusal and escalation numbers, known failures and past incidents

Store the same record with every release.

The baseline lets you find negative flips, which are cases the old version handled correctly and the new version gets wrong. A study of LLM updates found negative flips across many tasks and models, even in updates that improved overall performance. Its authors argue that compatibility with the previous version should be a goal of its own.

Compare the new version with the old one

As Brendan Burns of Microsoft pointed out in an SREcon talk on rolling out AI models, a model release rarely gives a clean working or broken signal. A new model or prompt can make some answers better and others worse. You only find out whether the change is a regression by measuring across many inputs.

Test the whole feature with the same prompts, retrieval, tool calls, parsing, routing and policy checks as production. A model that does well in isolation can still fail inside the full feature. Run each case through both versions and record whether each version finished the task, broke a rule, needed escalation, caused a side effect or went over a limit.

Test in layers

Split the tests into layers, so that a strong overall score cannot hide a regression in one area:

• Application contract tests check schemas, required fields, permissions, timeouts and parser behavior.
• Capability tests score the supported tasks, such as extraction, classification, summarization or drafting, against ground truth from your domain.
• Safety and security tests cover refusals, privacy boundaries, jailbreaks, prompt injection and unauthorized actions.
• End-to-end tests cover retrieval, citations, multi-turn state, tool choice and recovery from tool errors.
• Operational tests measure latency percentiles, throughput, token use, cost and rate limits.

Separate layers also show where a regression happened. In a study of a production food-ordering agent, the authors broke one layer of the agent's code at a time. In most cases the overall pass rate fell by less than six percentage points, while the tests for the broken layer fell by 25 to 91 points. The study covers a single agent, so treat the numbers as an illustration.

Build the test set

Take test cases from several sources:

• Curated golden cases with the expected behavior
• De-identified production examples that passed a privacy review
• Past incidents, security findings, complaints and user corrections
• Long conversations and incomplete requests
• Adversarial prompts
• Recorded tool-use sequences
• Examples from different languages, regions, user groups and use cases

Keep a held-out set that you never tune prompts or settings against, and refresh it when production shows a verified failure.

Replay these inputs in a test environment where nothing can cause side effects. Use the same retrieval data, tool schemas, preprocessing and output validation as production. If the new version receives different context from the old one, record the difference instead of blaming every change in output on the model.

OpenAI researchers used the same idea to predict safety problems before release. Across four GPT-5-series releases, they took de-identified conversations from an earlier deployment and had the new model write the next reply. The replay gave useful estimates of how often the model misbehaved after release, and it did better than estimates based on adversarially selected production data.

Choose how to score results

Use deterministic checks wherever the right answer is objective. Validate JSON with a parser, check citation IDs against the sources that were retrieved, compare tool calls with an allowlist, and confirm that no natural-language instruction can get the model past an authorization check. Use standard metrics where reliable ground truth exists, LLM judges for open-ended qualities such as relevance or completeness, and human review for unclear, new, high-impact or safety-sensitive cases.

An LLM judge is a model that scores another model's output against a rubric. It can cut review costs, but a NeurIPS 2025 position paper argues that the use of LLM judges has grown faster than the checks on their validity and reliability. Calibrate the judge against examples labeled by experts. Version the judge model, rubric and scoring prompt separately from the model you are testing, and calibrate again whenever the judge changes. Do not let an unvalidated judge approve high-severity safety cases on its own.

Decide what counts as a pass

The release report should say whether the new version meets the minimum requirements and whether it is better or worse than the old one. It should also give the negative-flip rate, the critical cases that failed, the user groups or use cases that changed, and how uncertain these numbers are.

The new version has to pass three kinds of gates: the absolute requirements, the regression limits against the old version, and every mandatory critical case. A better average does not make up for a privacy leak or an unauthorized action.

Test attacks and rare failures

Rare, severe failures need their own tests. Run these attacks against the complete application, using sandboxed accounts and synthetic data:

• Tell the model to ignore its instructions.
• Hide instructions inside retrieved documents, web pages, emails or files.
• Ask for the system prompt, credentials, private context or another user's data.
• Upload malicious or malformed files.
• Ask for tool calls the user is not allowed to make.
• Send very long prompts meant to exhaust context or cost limits.
• Try to shift the model's instructions step by step over many turns.
• Return tool errors, contradictory results or missing fields.

Test the enforcement layer as well as the model's reply. A refusal does not help if the request still reaches a privileged tool, and a safe reply does not prove that the parser, router or retrieval step keeps the same boundary. For destructive or externally visible actions, check that the action itself was blocked instead of reading the text of the reply.

Review every failure in a mandatory risk category, even when the averages improve. Sort the failures by severity:

• Critical: privacy disclosure, cross-tenant access, an unauthorized destructive action or a severe safety violation
• High: a wrong action in a sensitive use case, prompt injection that keeps working, or a major loss of grounding
• Moderate: a clear regression in formatting, refusals, retrieval or task completion
• Low: style changes that do not affect the task or the safety boundary

Set the release rule before you look at the results. For example, any critical failure blocks the release, high-severity failures need a documented fix or an approval, moderate regressions need an accepted impact assessment, and low-severity changes are monitored. For each critical case, keep the old version's output and the evaluator's reasoning, so reviewers can see what changed.

Run it in shadow mode

In shadow mode, a copy of each production request also goes to the new version while the old version keeps serving users. You log and score the new version's output, but users never see it. Because the traffic is live, shadow mode shows what a fixed test set cannot, such as the current mix of requests and input lengths, unusual retrieval results, the tools the new version picks, and how it behaves under production load and rate limits.

Compare the two versions request by request, and split the results by segment:

• Use case and task type
• User group and customer tier
• Language and region
• Input and context length
• Retrieval corpus
• Tool or integration
• Client or device
• Time of day and load

A version that passes overall can still fail for a particular language, integration or high-risk use case. Look for changes that success metrics miss, such as different refusal or escalation rates, longer or repetitive answers, unsupported citations, different tool choices and missing follow-up questions. Keep enough trace data to see whether a difference came from the model, the retrieval results, the tool responses or post-processing.

Give the new version simulated, read-only or sandboxed tools. Shadow traffic must never send messages, change records, make purchases, delete data or start anything that cannot be undone. When the new version produces a tool call, record and validate it without running it. When a tool's result depends on state, give both versions the same fixed or recorded responses so the comparison stays fair.

Shadow mode cannot show how users react to a different answer. A user might ask a different follow-up question, trust an answer more, or act on a new recommendation. Good shadow results are a reason to move on to a canary release, where users see the new version's answers.

Release to a small share of traffic

In a canary release, a small, controlled share of live traffic goes to the new version while the old version stays available. Google Cloud's reliability guidance for AI and ML recommends this for new model versions and says to monitor performance and error rates before the version gets all traffic.

Start with internal users or low-risk use cases if possible, then move to a small group of external users before you increase traffic. Keep use cases with irreversible side effects on the old version until the new one has passed stronger checks. Show the same segments as in shadow mode on your dashboards.

Increase traffic in fixed stages, such as 5%, 25%, 50% and then all traffic, with an explicit approval at each step. Google Cloud Deploy structures a canary the same way, with one phase per traffic percentage and a final stable phase at 100%. Choose the stages and how long you watch each one based on request volume, possible harm and how long it takes to detect rare failures. A low-volume, high-risk use case needs more evidence per stage than a high-volume one whose mistakes are easy to undo.

At each gate, record how much traffic each version received and how the new version did on technical, quality, safety and cost metrics. Include regressions in any segment, open incidents and open questions. The release owner signs off before the next stage. Stop expanding when a critical event occurs, a required threshold fails or the evidence is incomplete.

What to monitor

Availability and latency do not tell you whether users get better or safer results, so monitor behavior and safety alongside technical health.

For technical health, track errors, timeouts and retries; p50, p95 and p99 latency; token use and cost per finished task; context-window failures; rate-limit events; serving capacity; and failures in retrieval, tools and other dependencies. Where you can, compare the new version with the old one running at the same time. A provider slowdown, a new mix of users or a retrieval outage affects both versions, while a regression in the new version shows up mostly in its own numbers.

For behavior, track task completion, grounding and citation accuracy, valid structured output, refusal and escalation rates, satisfaction scores, and how often users correct an answer, re-prompt, complain or give up. Have people review a sample of high-impact outputs, especially where no reliable ground truth exists, and record the reason for each escalation or correction. A rise in re-prompts can point to a quality or instruction-following regression even when every request technically succeeds.

For safety and security, track policy violations, detected prompt injections, sensitive-data disclosure, unauthorized tool calls and cross-tenant access. A rare event still needs an immediate response when it is severe.

Watch for drift in the inputs (user mix, request topics, prompt length, languages) and in the parts around the model (retrieval indexes, tool schemas, provider endpoints and safety layers, and the judges and rubrics you evaluate with). Keep representative traces under your approved retention policy, and remove or protect sensitive data before you share traces for debugging. When you verify an incident, add a minimal reproduction to the regression suite and label it with the layer where it failed.

The NIST AI Risk Management Framework expects post-deployment monitoring plans to include user feedback, appeal and override, incident response, recovery and change management. Keep monitoring after the new version reaches 100% of traffic.

Plan the rollback before launch

Before the canary starts, decide what triggers a rollback, who makes the call, how traffic goes back to the old version and what the fallback is, and test all of it. Pause or roll back automatically when any of these happens:

• A critical safety or privacy event
• An unauthorized external action
• A clear rise in high-severity policy violations
• Severe schema or parser failures, or a large increase in errors or timeouts
• A statistically credible regression in a critical use case
• A breach of a hard cost or token limit
• A provider outage or an unexpected change in the provider's behavior
• Harm in one segment above the agreed tolerance

Rehearse the rollback in an environment with the same routing as production. Check that operators can:

1. Route traffic back to the old version.
2. Restore the old prompt and inference settings.
3. Restore compatible tools, retrieval configuration and parsers.
4. Turn off external actions.
5. Keep logs and traces for the investigation.
6. Send critical work to a simpler model or a manual process.
7. Make sure no queued jobs run on the new version after the rollback.

The NIST Generative AI Profile asks organizations to test and manage the risks of rollover and fallback for generative AI systems, and says the fallback may include manual processing. It also calls for incident response plans and continuous monitoring for third-party AI systems, which includes any model you call through a provider's API.

After the switch

The model name does not fully describe what runs in production. Providers update model weights, safety policies and serving infrastructure without changing the API endpoint. When researchers compared the March and June 2023 versions of GPT-4, its accuracy at telling prime numbers from composite numbers fell from 84% to 51%. The June version also made more formatting mistakes in generated code, followed instructions less well and was less willing to answer sensitive questions.

Pin an exact snapshot where the provider offers one, and expect to migrate again, because LLM APIs are often updated silently and scheduled for deprecation. If the provider only offers a model that is updated continuously, run your compatibility tests on a schedule and keep a fallback provider or a non-LLM path for critical use cases. Treat a provider notice or an unexplained change in latency, refusals, formatting, retrieval use or tool choice like a new release, and run the evaluation again.

When the rollout is finished, review which gates passed or failed, which segments changed, which incidents or near misses happened, and whether the rollback worked as rehearsed. Record which tests failed to predict production behavior, and add the new failures to the permanent regression suite.

Retire the old version only after the new one has run stably, the rollback window has closed, and critical use cases still have a tested fallback.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com