How to Test a Nondeterministic LLM Application
When an application calls an LLM, the same input can produce a different response on the next run. A test that compares the response with one expected string…
Read articleArticles
Page 2 of 2.
Is the Same Prompt Always the Same LLM Input?
No. The text you type into a chat box is only one part of what a large language model (LLM) receives. The full request can also contain system…
Read articleTests Come for Free Now, and Most of Them Are Pointless
For years everybody agreed that tests make software better, and almost nobody wrote them. In the first ten years or so of my career I probably didn't write…
Read articleWhat Is a System Fingerprint in LLM APIs?
When an LLM API response includes a field such as systemfingerprint, it is identifying the provider’s backend configuration, not your prompt, account, device, or system message. The value…
Read articleWhat Is Batch Invariance in LLM Inference?
Batch invariance means a request produces the same inference result when the server runs it alone, alongside other requests, at a different batch position, or under another supported…
Read articleWhat Is the Difference Between a Model Alias and a Versioned Model ID?
If a configuration contains production, latest, or stable, it probably uses a model alias. An alias is a named pointer that tells a platform which model version currently…
Read articleWhen Does an LLM Agent Need Persistent Memory Instead of a Larger Context Window?
An LLM agent is software that uses a large language model to answer questions or carry out tasks, sometimes across multiple conversations. Its context window is the amount…
Read articleWhy Ollama Prints Tool-Call JSON Instead of Calling the Tool
When you use Ollama to run a local model, the model might print JSON that looks like a function call even though the tool did not run. A…
Read article