Does Setting a Seed Make LLM Output Reproducible?

No. A seed can improve repeatability, but it does not guarantee identical large language model (LLM) output. It initializes the pseudorandom number generator used in token sampling; it does not lock the model, prompt construction, hardware, numerical calculations, or hosted infrastructure.

To replay output exactly, you usually need the same model revision, tokenizer and chat template, fully rendered prompt, decoding settings, random state, runtime, hardware behavior, batching conditions, and dynamic inputs. Hosted APIs generally offer best-effort repeatability. Self-hosted deployments can provide stronger guarantees only when engineers pin and verify the full inference environment.

What a Seed Controls

An LLM generates text one token at a time. At each step, the model calculates a probability for every possible next token. The decoder then selects one token from that distribution.

With sampling enabled, decoding parameters determine how the distribution is used:

• Temperature controls how strongly the probabilities favor high-ranking tokens.
• Top-p restricts sampling to the smallest group of tokens whose cumulative probability reaches a chosen threshold.
• Top-k restricts sampling to the highest-ranked k tokens.
• Repetition controls, stop sequences, output limits, and structured-output rules also affect generation.

The seed initializes the pseudorandom sequence used to select tokens from the resulting distribution. If the model produces the same distribution at every step, the same seed and decoding configuration produce the same sampling choices.

That is why engineers should still set seeds for debugging, prompt comparisons, regression tests, and controlled evaluations. A seed removes avoidable sampling variance. It does not control the entire function that produces the output:

\text{output} = f(\text{weights}, \text{prompt}, \text{tokenizer}, \text{decoding}, \text{random state}, \text{runtime})

Seeds Are Not Replay Keys

A numeric seed has meaning only within a particular implementation. Different providers and inference engines can use different pseudorandom generators, tokenizers, sampling algorithms, and seed semantics.

A seed identifies neither the model weights nor the probability distribution from which the decoder samples. Reusing seed 42 with another model does not reproduce the first model’s choices. Changing the model deployment, prompt template, tokenizer, or external input also changes the token probabilities, so the same random sequence no longer produces the same response.

Microsoft documents Azure OpenAI seeded generation as a best-effort attempt at deterministic sampling, not a guarantee. An archived OpenAI Cookbook example similarly tied repeatability to matching request parameters and system_fingerprint, and applied its guidance to specific 2023 model versions rather than to all current models.

Treat a seed as one field in a replay record, not as a portable identifier for a response.

Why Zero Temperature Varies

Temperature zero normally selects the highest-probability token instead of deliberately sampling among alternatives. This is called greedy decoding. Greedy decoding removes one source of randomness, but it does not guarantee that the model computes exactly the same probabilities on every run.

The model calculates logits, numerical scores for each possible next token. Floating-point arithmetic rounds those scores. Parallel GPU operations can also combine values in different orders. Because floating-point addition is not perfectly associative, changing the operation order, numerical precision, kernel, batch, or hardware can slightly change a logit.

If two tokens have nearly equal scores, a small numerical difference can reverse their ranking. Once one token changes, the model receives a different sequence on the next step. The remaining completion can then diverge substantially.

A seed has little direct effect during greedy decoding because the decoder selects an argmax, or highest-scoring token. The seed cannot correct a changed argmax.

Sampling and Numerical Variation

Sampling randomness and numerical nondeterminism are separate problems.

A fixed seed repeats the same random draw only when the decoder uses the same probability distribution. If rounding or a different execution path changes that distribution, the same random stream can select a different token.

Temperature zero removes the random draw but still depends on stable token scores. Therefore, deterministic random-number generation and deterministic model computation require separate engineering controls.

A serving engine combines multiple requests into a batch to improve hardware utilization. Different batch contents, request order, padding, or scheduler decisions can select different GPU kernels or reduction orders. The visible prompt stays the same, but the arithmetic path changes.

The Thinking Machines Lab report on LLM nondeterminism describes this mechanism and reports an experiment in which a Qwen model generated 80 unique completions across 1,000 temperature-zero attempts under default settings. With batch-invariant kernels enabled, all 1,000 completions matched. This is engineering evidence from one implementation, not a universal benchmark, but it demonstrates why serving behavior matters.

Variables That Change Output

Exact reproduction requires more than matching the visible prompt and seed. The complete execution contract includes the following variables.

Model state

Record and pin:

• Model weights and deployment revision
• Quantization artifacts
• Tokenizer files and version
• Chat template
• Inference-engine version and serving configuration
• Model routing or region, if the provider exposes those choices

A provider can update a backend without changing the model name. A local deployment can also change output after an engine upgrade, a new quantization artifact, or a modified chat template.

Request construction

The effective prompt includes more than the user message. Preserve:

• System, developer, and user messages after templating
• Retrieved documents
• Tool inputs and tool outputs
• Structured-output schemas
• External data
• Application-side transformations
• Conversation history and truncation decisions

Two requests that look identical in an application log can produce different prompts if retrieval results, timestamps, tool responses, or formatting steps differ.

Decoding settings

Record every generation control, including:

• Seed
• Temperature
• Top-p and top-k
• Maximum output length
• Stop sequences
• Repetition penalties or related controls
• Structured-output settings
• Tool-choice settings
• Request-specific response-format options

Changing a single control changes the probability distribution or the point at which generation stops.

Serving behavior

The serving layer can change computation through:

• Dynamic batch formation
• Request order and concurrency
• Scheduler decisions
• Prefix caching
• Speculative decoding
• Chunked prefill
• Tensor parallelism
• Padding and memory layout

Hosted API callers generally cannot freeze these variables. A self-hosted deployment can control more of them, but only if its serving mode exposes the necessary settings.

Numerical environment

For local inference, record:

• GPU model, GPU count, and GPU topology
• CUDA and driver versions
• Kernel libraries
• Numerical precision, such as BF16, FP16, or FP32
• Deterministic-kernel settings
• Batch size and concurrency
• Inference-engine commit

Different GPU counts or precision formats can produce different arithmetic paths even with identical weights and prompts.

Evidence From Recent Studies

Two 2025 studies show why a fixed seed and temperature zero are insufficient as universal guarantees.

Hosted APIs

The ACL paper Non-Determinism of “Deterministic” LLM System Settings in Hosted Environments tested five hosted LLM systems across eight tasks. The researchers used identical inputs, temperature=0, top_p=1, and fixed seeds, with ten runs per condition. The experiments took place in February 2025.

The study found accuracy variation of up to 15 percentage points across repeated runs under settings intended to be deterministic. It also reported a particularly large maximum-to-minimum gap for Mixtral-8x7B on one college-math task, where accuracy ranged from 75% to 3% across the tested runs.

Raw text agreement was often lower than parsed-answer agreement. Different explanations or formatting could still contain the same selected answer, while exact-string comparisons treated them as different. This distinction matters for downstream parsers, evaluators, and snapshot tests.

The study covered only five systems, eight tasks, and ten runs per condition, so its measured variation does not establish a failure rate for every provider or workload. It does establish that fixed seeds and zero temperature do not guarantee stable hosted output.

Hardware and precision

The NeurIPS 2025 study, Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference, varied GPU type, GPU count, batch size, and numerical precision.

For DeepSeek-R1-Distill-Qwen-7B under BF16 precision, the researchers reported up to 9 percentage points of accuracy variation across tested runtime configurations. On MATH500, more than 90% of examples diverged under BF16 across those configurations, compared with 2.2% under FP32. They also observed response-length differences of up to 9,000 tokens in tested cases.

The results do not mean that FP32 guarantees determinism in every deployment. They show that lower-precision arithmetic and runtime configuration can materially affect greedy decoding. The same seed cannot compensate for a different numerical path.

Hosted and Local Inference

Hosted APIs are convenient but expose only part of the reproducibility contract. A provider might expose a model identifier, seed, request parameters, request ID, or backend fingerprint. The caller generally cannot freeze the provider’s GPU type, batching, scheduler, kernel libraries, numerical precision, routing, or deployment revision.

Record the provider’s available metadata, especially a system or backend fingerprint. Microsoft recommends monitoring system_fingerprint because backend changes can affect repeatability. A matching fingerprint can provide useful evidence that the backend configuration did not change, but it is not proof that every internal computation was identical.

Self-hosting provides stronger control, not automatic determinism. vLLM’s version 0.10.2 reproducibility documentation conditions reproducibility on using the same hardware and vLLM version and warns that ordinary online serving does not guarantee reproducibility. The documentation describes its newer batch-invariance feature as a beta feature designed to reduce dependence on batch size and request order, and limits it to supported NVIDIA hardware.

Choose the deployment model based on the required level of repeatability:

• Use a hosted API when best-effort repeatability and task-level stability are sufficient.
• Use controlled local inference when exact replay has operational, testing, or audit value.
• In either case, measure the desired property with repeated tests instead of inferring it from the presence of a seed.

Define Reproducibility First

“Reproducible” describes several different targets. Choose the target before designing a test.

A system can produce identical text and still produce an incorrect answer. It can also produce different wording while preserving the same parsed JSON value or classification label.

Test the Actual Failure Mode

For JSON extraction, measure schema validity, parser success, field-level agreement, and required-value accuracy. Raw string equality is a poor primary metric if whitespace, field order, or explanatory text does not affect the downstream result.

For classification, repeat representative inputs and measure label disagreement, accuracy, and calibration. A stable label is not enough if the model is consistently wrong.

For assistants, evaluate task success, safety behavior, required facts, and important constraints across repeated runs. For benchmarks, report repeated-run distributions rather than one score when exact replay is not assured.

For audit-sensitive workflows, store the original request, response, model metadata, dynamic inputs, and evaluation result together. That record supports both exact comparison and task-level investigation.

Record the Full Environment

For a hosted API, log at least:

1. Exact model or deployment identifier
2. Fully rendered request payload
3. Seed and every decoding parameter
4. API and SDK versions
5. Timestamp, region, request ID, and the provider fingerprint
6. Retrieved documents and external data
7. Tool inputs and outputs
8. Raw response and normalized parsed result

Snapshot dynamic inputs rather than relying on the original request alone. A changed retrieval result or tool response changes the effective prompt even when the user message remains unchanged.

For self-hosted inference, pin the model and tokenizer revisions, chat template, quantization artifacts, inference-engine commit, CUDA and driver versions, kernel libraries, GPU topology, precision, and scheduler configuration. Control or record batch size, concurrency, request order, speculative decoding, caching, tensor parallelism, and prefill behavior.

Then run a representative regression suite repeatedly under production-like concurrency. Compare output hashes for string-level changes and task-level metrics for meaningful behavior changes. A seed belongs in that audit trail, but it is not evidence that the full system is replayable.

Practical Takeaway

Setting a seed is useful because it reduces sampling variance and makes debugging and regression testing more controlled. It does not make LLM output inherently reproducible.

For exact text, freeze and verify the complete model-serving environment. For most production systems, test parsed answers, labels, schema validity, task success, safety behavior, or statistical performance across repeated runs instead.

FAQ

Does setting a seed guarantee the same response?

No. A seed controls pseudorandom choices during sampling, but model updates, backend behavior, numerical computation, batching, and dynamic inputs remain outside its control.

Should identical requests usually match?

They often match on a stable deployment with fixed decoding settings, but exact equality is not guaranteed. Treat repeated matching results as evidence of observed repeatability, and consult provider or serving-stack documentation for any stated guarantees.

Does temperature zero eliminate variation?

No. Temperature zero removes deliberate sampling randomness through greedy decoding. Numerical rounding, GPU execution paths, batching, scheduling, or backend changes can still alter the highest-scoring token.

Can the same seed be reused across providers?

No. Providers can use different tokenizers, model distributions, pseudorandom generators, sampling implementations, and seed semantics. A seed is provider-specific rather than a portable replay key.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com