How to Maintain AI Workflows During System Changes
A business AI workflow connects an AI model to documents and business software, then uses the model’s output to do things such…
In a business workflow, an AI can read a document and then update a customer record, send a message, or create an invoice. If the request times out, the person responsible cannot tell from the timeout alone whether the destination completed the action. If the workflow tries again, it might repeat the action. If the document changed while the workflow was running, the AI result might also be out of date.
The workflow should recognize the same intended action across retries and save its progress. It should also check important inputs before making a consequential change. Idempotency means that repeating an action does not produce an additional business effect. A workflow can still execute code more than once; the goal is to prevent those repeated executions from creating duplicate payments, records, or messages.
A stable action ID identifies one business intent, such as creating a particular invoice from a particular approved document revision. Reuse that ID when retrying the same action with the same intended inputs. Create a new ID when the source document or approved instruction changes; the new ID marks a new generation of the action.
For example, an invoice workflow could derive an action ID from the invoice request, the action type, and the document revision used to prepare it. The exact format is a design choice, not a universal standard. Keep the ID tied to the intended business action rather than a workflow run: a run ID changes when the workflow restarts, even if the intended invoice is the same.
Pass the same idempotency key to the connected service when it supports one. Stripe explains that a service can use a repeated key to recognize a retry and return the result already stored for that request. If the service does not support idempotency keys, the workflow needs another way to check whether the action already happened, such as looking up the resulting record before creating another.
A timeout does not prove that a request failed. The destination may have completed the action while the response was lost. In that case, retrying under a new ID risks creating a duplicate. Reuse the original ID and check the destination for the result before treating the action as failed.
A checkpoint records which workflow steps have finished and saves results that later steps need. When a worker stops unexpectedly, the workflow can resume from the last saved progress instead of repeating every step. Saved workflow histories, such as Temporal’s event history, show how a workflow can recover recorded progress.
A checkpoint does not prove that an external action happened only once. A worker might send a request, the destination might complete it, and the worker might stop before saving a success result. The workflow then has a checkpoint that does not reflect the destination’s state. Idempotency and checkpoints solve different problems: the checkpoint tracks what the workflow knows; the idempotency key or destination lookup helps prevent a repeated external effect.
Keep a deduplication log for side-effecting actions such as sending a customer message or creating an invoice. For each action ID, record a fingerprint calculated from the request, its status and timestamps, the destination record or operation ID, and the result. Use clear statuses such as “in progress,” “succeeded,” “failed,” and “outcome uncertain.” A local log helps recovery, but it cannot by itself guarantee that the destination did not complete an action just before a crash. When the log says “outcome uncertain,” compare it with the destination’s records before sending again.
Separate long steps into smaller actions when doing so lets the workflow save progress between them. If a multi-step task fails halfway through, smaller checkpoints limit the work that must be repeated. Temporal’s activity guidance also notes that retryable activities need to tolerate repeated execution.
Retry a request when the failure looks temporary and the intended inputs have not changed. A network interruption or temporary service error may clear on another attempt. Use a bounded retry policy, with a delay that increases between attempts where appropriate. The exact retry count and delay depend on the workflow platform and the destination service; AWS Step Functions, for example, lets operators configure retry limits, backoff, and error handling in its error-handling rules.
Stop automatic retries when the error points to a problem another attempt will not fix, such as invalid input or insufficient permission. A fail state marks the workflow as stopped, records the error, and prevents it from looping without progress. Route the item to a person when the workflow cannot determine whether a consequential action happened, when required information is missing, or when a changed result needs judgment.
The same distinction applies to queues. Amazon SQS standard queues use at-least-once delivery, which means a message can be delivered more than once. A workflow that consumes a queued task still needs deduplication or idempotent actions, even if the queue normally delivers messages once. See the SQS visibility timeout documentation for that service’s delivery behavior.
Save the source document’s identity and revision or version when the workflow reads it. If the source system does not provide a dependable revision, save a fingerprint of the content the workflow actually used. A fingerprint helps detect a change in the content; it does not decide whether the change matters to the task.
Before committing a consequential action, check the source again. If the relevant content has changed, mark the previous AI result as stale. Rerun the analysis when the changed content affects the answer, or pause for review when the workflow cannot determine that safely. A document-change notification may only indicate that an update exists; for example, Google Drive requires a client to retrieve change details from its change feed.
Also check whether the destination record changed after the workflow read it. A person or another integration may have edited the record while the AI prepared an update. Some services support writes that check the record’s version using an entity tag, or ETag, which identifies that version. With the HTTP If-Match condition, a client can ask the server to apply an update only if the record still matches the version the client read. The server rejects the update when the condition fails, helping prevent a stale write from overwriting newer data. This method works only when the destination supports and honors it; RFC 9110 defines the HTTP behavior.
If the destination does not support conditional updates, fetch the record again before writing. Compare the fields the workflow plans to change. If those fields changed, pause and reconcile the difference rather than blindly overwriting the newer value.
Save enough information to identify how an AI result was produced: the model identifier, prompt or template version, input references, relevant settings, output, and validation results. When an API call times out after producing a result that the workflow captured, reuse that saved result for a retry rather than generating a replacement without cause.
An intentional rerun after a model or prompt change is different. Treat the new output as a separate generation, or version, and compare it with the prior result before replacing downstream work. Model outputs can vary, and providers can change model behavior between snapshots, or specific released model versions. OpenAI recommends pinned model versions and evaluations to improve consistency, while noting that generated outputs remain variable in its API guidance. Pinning supports control; it does not guarantee identical output.
For a low-impact classification, a validated new result might update the record automatically. For a customer-facing message or financial action, require a person to review a materially changed result before the workflow commits it. Set that boundary based on the consequences of an incorrect action.
| Situation | Response |
|---|---|
| Temporary error with unchanged inputs | Retry with the same action ID and bounded backoff. |
| Worker stops after saved steps | Resume from the checkpoint and reuse recorded results. |
| Source document changes | Refresh it, identify affected work, then rerun or pause for review. |
| Destination record changes | Use a conditional update if supported; otherwise fetch and reconcile before writing. |
| Model or prompt changes intentionally | Create a new output generation and validate it before replacing prior results. |
| Persistent error or uncertain external outcome | Stop automatic retries, record the state, and reconcile or hand the item to a person. |
Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.
Take a look at vroni.com