How to Connect a CLI Coding Agent to a Local LLM With Ollama

To use a coding assistant in your terminal while its language model runs on your computer, connect a CLI coding agent to Ollama. A CLI, or command-line interface, lets you run a tool by typing commands in a terminal. Ollama downloads and runs the language model on your computer. The agent reads project files and proposes edits; it may also run commands. An API, or application programming interface, carries prompts and responses between the agent and Ollama.

If Ollama supports your coding agent, run ollama launch to set it up. To configure the connection yourself, point the agent to Ollama’s local server and select a model that Ollama has downloaded. Because the agent controls access to your files and commands, review its actions even when the model runs locally.

Launch a supported agent through Ollama

Install Ollama and your chosen coding agent. Then check that your version of Ollama supports the launch command. Ollama’s January 2026 announcement documents ollama launch for Ollama v0.15 and later, with integrations for OpenCode, Claude Code, Codex, and Droid. The model name after --model tells Ollama which model to use.

ollama pull qwen3-coder
ollama launch opencode --model qwen3-coder

The first command downloads the model. The second configures and starts OpenCode through Ollama. If you want Ollama to configure OpenCode without starting it, use:

ollama launch opencode --model qwen3-coder --config

The documented launch commands for other supported agents include ollama launch claude and ollama launch codex. Check your installed versions and each agent’s installation requirements if a command is unavailable. Ollama’s launch instructions describe the supported integrations and model selection.

Configure an agent manually with Aider

With manual configuration, Ollama runs the model on your computer, and Aider sends requests to Ollama’s server. Aider’s documentation recommends the ollama_chat/ prefix when you specify an Ollama model.

In a Unix-like terminal, download the model and start Ollama’s server with a context length setting:

ollama pull 
OLLAMA_CONTEXT_LENGTH=8192 ollama serve

Leave the server running. Open a second terminal, change to your project directory, and start Aider:

cd /path/to/your/project
aider --model ollama_chat/

Replace <model> with the name you pulled. For example, if you downloaded qwen3-coder, use ollama_chat/qwen3-coder. Aider’s Ollama setup instructions list http://127.0.0.1:11434 as the usual local API address and document OLLAMA_API_BASE for setting a different address. Shell syntax and service management differ on Windows, so use the equivalent settings for your environment.

The 8192 value above is Aider’s documented example, not a universal recommendation. A context window is the amount of text a model can consider in one request. It includes instructions and conversation, as well as code. Aider warns that a small window can leave out repository information. Set the window to suit the model and coding task, and check that the running Ollama server uses the setting you intend.

Match the agent’s API to Ollama

Some coding agents connect directly to Ollama. Others connect using a request format they already support. Ollama provides local endpoints at http://localhost:11434/api for its native API and http://localhost:11434/v1 for its OpenAI-compatible API. “Compatible” means the endpoint follows some of another provider’s request format; it does not guarantee that every feature works the same way.

For example, Ollama documents tool support for OpenAI Chat Completions, but its compatibility table marks tool_choice as unsupported for that API. A coding agent’s ability to call tools also depends on the agent and the model. Check Ollama’s API compatibility details when an agent connects but fails to use a feature such as tool calls.

Claude Code uses a different setup. Ollama’s Claude Code instructions require Ollama v0.14.0 or later for Anthropic Messages API compatibility. In a Unix-like shell, set the local endpoint and token before starting Claude Code:

export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_BASE_URL=http://localhost:11434
claude --model qwen3-coder

Ollama’s Claude Code instructions give this configuration and recommend at least 32K tokens of context for that integration. Follow the chosen agent’s instructions if it expects different settings.

Choose context and model size for your machine

A model’s advertised context capacity can differ from the context size Ollama uses. The model may support a large window, while Ollama or the agent uses a smaller one. More context lets a coding agent include more project information in a request, but the computer also needs memory to load the model and process that context.

Ollama’s January 2026 launch announcement recommends at least 64K tokens for coding tools. Its Claude Code post recommends at least 32K for that particular integration. These recommendations apply to the configurations described in those posts; they are not a universal minimum. Start with a context size that fits your model and hardware, then increase it if the agent needs more project information and your computer has enough memory.

Model size also affects memory and speed. For example, Ollama’s library lists qwen3-coder:30b as a 19 GB download with a 256K context window. Those figures describe that model variant, but they do not show whether it will fit in your computer’s memory or run fast enough for your needs. Before choosing a larger variant, check the model listing and the context size the model will use.

Check the connection and keep local use explicit

If the agent cannot connect, first confirm that Ollama is running and that the agent points to the correct local address. For manual setups, compare the agent’s configured base URL with Ollama’s documented local endpoint, then verify that the model name in the agent command matches the model you downloaded. A connection error points to the server address or availability; a model lookup error points to the selected model or its name.

Also confirm where the model processes each request. A request sent to a local Ollama endpoint uses the model running on your computer. When local inference is your requirement, check that both the selected model and endpoint are local. Ollama also offers hosted models, so choosing a model through an integration alone leaves the request’s processing location uncertain. Ollama documents the distinction between local and hosted API endpoints.

Local inference does not limit what the coding agent can do in your project. An agent may read or edit files and run commands, depending on its permissions and configuration. Review proposed changes and understand the agent’s command access before letting it work in a repository.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com