Built-In Tools vs Custom Tools in LLM Agents

When building AI agents, a tool can be anything that the model can ask to use such as a search engine, a database lookup, a shell command, etc. The model doesn’t directly operate those tools, but rather decides what tool would be helpful, requests a tool use with parameters, gets the result back, and then continues it's work.

A custom tool is a tool that you define and run on your own infrastructure. You provide a schema such as search_web or get_customer_balance to the model, the model requests it, and your application runs the actual code on your infrastructure.

A built-in tool is different. The provider runs it in its own runtime. In practice this mostly means hosted web search. OpenAI and Anthropic both offer provider-run web search in their APIs. That means if you use ChatGPT or Claude (or Codex CLI or Claude Code), they can just search the web and access websites, which internally means they use their built-in web search tool. Other providers like DeepSeek also support tool calls (any llm does so more or less) but don’t offer a hosted web-search tool themselves, so you’ll have to bring and run the search tool yourself.

So a built-in tool feels more “native” mostly because it happens "magically" under the hood. The question I'm interested in here is, what's the difference between using built-in tools vs your own custom tools that you give the LLM. In other words, what's the difference between using ChatGPT/Codex with its integrated web search vs. using DeepSeek while providing my own custom made web search tool.

1. The basic architecture

The architecture for a normal tool you provide is roughly like this:

The key boundary here is that the model doesn’t execute your function. It returns a structured request that says, effectively:

{
  "tool": "search_web",
  "arguments": {
    "query": "latest Nvidia earnings"
  }
}

Your program sees that, executes something, and sends the result back.

The current function-calling documentation from OpenAI describes this five-step flow pretty much exactly: send tools, get tool call, execute on your application side, send tool output back, get continuation/final answer.

---

2. What changes with provider-hosted web search?

Now move the orchestrator inside the provider.

The entire loop can happen in one API request from your end.

Anthropic’s current web-search flow works like this: Claude decides when to search, the API searches and returns results to Claude, and this can happen multiple times in the request before Claude returns the final answer.

OpenAI also refers to the use of a reasoning model for web search as agentic search, where the model manages the search process, analyzes the results, and can choose to continue searching. The search events are exposed in their Responses API as web_search_call items.

So that distinction you're noticing is real.

---

3. But is the tool actually invoked "during reasoning"?

This is where terminology gets tricky.

Conceptually, yes:

OpenAI documentation currently describes agentic web search as being able to do searches “as part of its chain of thought.”

But do not interpret that as an HTTP request that somehow occurs between transformer layer 63 and transformer layer 64. That’s almost certainly not the right mental model.

A better abstraction would be the generated text hitting a tool boundary, where the external runtime does the tool work and the model continues from the returned observation.

The generation is effectively interrupted/suspended at a tool boundary, where an external system gets an observation and generation proceeds with that observation available.

The exact internal implementation (KV cache handling, worker scheduling, separate inference passes, etc.) is a provider-private implementation detail. Do not assume that the tool is literally implemented within one transformer forward pass.

---

4. And that's surprisingly similar to your own agent loop

At the abstract level:

LLM → action → environment → observation → LLM

So, no new cognitive operation is magically available only with provider tools.

This distinction is important.

---

5. Then why can provider-native tools sometimes work better?

This is where the practical difference becomes significant.

There are several advantages a provider can have.

A. The model may have been specifically trained for that tool

This is potentially the biggest difference.

Imagine these two tool schemas.

Your tool:

internet_lookup(
    query,
    search_depth,
    domains,
    freshness
)

Provider tool:

web_search(...)

The provider may have trained the model on millions of trajectories like the following during post-training:

question
→ reason
→ web_search
→ inspect
→ reason
→ web_search
→ inspect
→ answer with citation

They can optimize things like:

Should I search?
What query should I issue?
Should I search again?
Which result should I open?
Which information is relevant?
Should I trust it?
When do I have enough evidence?
How do I cite it?

That is much more than learning JSON syntax.

It is tool-use policy learning.

For example, Anthropic discusses interleaved thinking for supported tool-use modes. In this case, the model can reason between tool calls to decide what to do next.

So, if you plug in your own unfamiliar search tool, the model might generalize very well, but it might not have exactly the same amount of post-training on your particular interface.

---

6. Provider search can also be much more than a search API

This is another important point.

Suppose your implementation is:

results = bing.search(query)
return results[:10]

The model receives 10 chunks.

A provider's web_search might conceptually be closer to:

query generation
       ↓
multiple search backends
       ↓
ranking
       ↓
fetch pages
       ↓
extract readable content
       ↓
deduplicate
       ↓
spam/quality filtering
       ↓
reranking
       ↓
token-budget optimization
       ↓
citation metadata
       ↓
model context

And there can be additional loops around it.

Anthropic’s current web search is a good concrete example. Newer versions can get Claude to run code to filter search results before they get into the model context, so irrelevant content takes up fewer context tokens.

OpenAI also allows controls over search context and returned-token budgets, and its search in reasoning mode can perform search, page opening, and find-in-page tasks.

So when comparing:

provider web_search

versus

mySearchTool()

you may actually be comparing two very different retrieval systems.

---

7. There's also a latency advantage

Your loop may involve model inference, an API round trip to your application, your own process, a search API call, another process step, another API round trip, and then model inference again. Perhaps repeated 5 times.

A hosted loop may look more like a model worker that calls an internal tool service and then continues on the model worker.

The provider has opportunities to optimize for scheduling, networking, result serialization, caching, streaming, etc.

That can make multi-step research materially faster.

I would not, though, assume things like “they definitely preserve the exact KV cache across searches” unless the provider explicitly documents it. That is implementation-specific.

---

8. Context handling is another subtle advantage

With a client tool, suppose you return this:

{
  "results": [
    { "title": "...", "content": "8,000 tokens..." },
    { "title": "...", "content": "10,000 tokens..." },
    ...
  ]
}

You've now dumped a mountain of text into the model context.

A tool integrated with a provider can tightly control search corpus, filtering, extraction, reranking, selection, compression and model context. And metadata can potentially be kept separately.

That often gives you a better:

useful-information / context-token

ratio.

This can have a surprisingly large effect on agent quality.

---

9. But a self-hosted tool can actually be BETTER

Provider-hosted doesn't inherently mean superior.

Imagine you're building a programming agent.

Instead of generic web search, you give it:

search_github_code()
search_stackoverflow()
search_npm()
fetch_package_docs()
search_internal_docs()
lookup_symbol()

plus carefully optimized schemas and result formatting.

That system may dramatically outperform generic web search for your application.

Or for a financial agent:

get_sec_filing()
get_realtime_price()
get_earnings_transcript()
query_bloomberg()

is probably preferable to blindly searching the internet.

So provider tools tend to win on:

general-purpose integration
zero setup
latency
citations
model/tool co-optimization

while your own tools win on:

control
domain specificity
private data
deterministic APIs
custom ranking
observability
security boundaries
cost control
provider independence

---

10. There's a continuum rather than two categories

Modern agent architectures actually have something like four levels.

Level 1: A client function. Level 2: A remote tool or MCP server. Level 3: A tool hosted by a provider. Level 4: A full product-level agent environment with shell, filesystem, browser, search, git, task state, etc.

At this point you're not really comparing models anymore.

You're comparing agent systems.

And that's becoming increasingly important.

---

11. What if the model provider has no built-in web search?

Suppose you have:

Model A
- excellent reasoning
- excellent function calling
- no web-search feature

and:

Model B
- excellent reasoning
- native web search

You can absolutely build this around Model A:

while True:
    response = model(messages, tools=tools)

    if response.tool_call:
        result = run_tool(response.tool_call)
        messages += [response.tool_call, result]
        continue

    return response.text

Architecturally, you've recreated the same agent loop.

There's no fundamental reason Model A couldn't do excellent web research.

The main variables become:

reasoning ability
×
tool-use training
×
quality of your search stack
×
context management
×
agent-loop design

not simply:

native web search: yes/no

---

12. But a model not trained for tool use is a different story

There is a more fundamental distinction here.

Consider three models:

Model A
Excellent reasoning
Excellent native/function tool use

Model B
Excellent reasoning
Tool calling supported but mediocre

Model C
Plain text model, no meaningful tool-use training

You can bolt tools onto all three.

For C you could say:

When you want to search, output:

<search>
query
</search>

and parse it.

Technically it works.

However, agent quality may be much worse if the model hasn’t learned a robust policy for:

when to act
which action
argument construction
result interpretation
error recovery
multi-step exploration
stopping

That’s one reason that modern “agentic models” feel qualitatively different from older LLMs, even if they can both emit JSON on paper.

---

13. A useful way to think about the model itself

I'd model an agentic LLM as approximately implementing a policy:

\pi(a_t \mid s_t)

where its current state is something like:

s_t =
\{
\text{prompt},
\text{conversation},
\text{reasoning state},
\text{previous observations},
\text{available tools}
\}

and the next action might be:

a_t \in
\{
\text{text},
\text{web_search},
\text{shell},
\text{read_file},
\text{write_file},
...
\}

A tool execution changes the environment:

o_{t+1} = Tool(a_t)

and that observation is fed back:

s_{t+1} = s_t + a_t + o_{t+1}

Then the model chooses again.

From this perspective, provider search and your custom search are mathematically the same kind of operation.

What differs is who implements:

Tool(a_t)

and who implements the surrounding control loop.

---

14. Why "native search" can nevertheless increase apparent model intelligence

This is a particularly important consequence.

Suppose the naked model has capability M.

You might think:

AgentCapability = M

But it's more like:

AgentCapability =
f(
M,
tools,
tool\ policy,
retrieval,
context\ management,
orchestration
)

A good search-enabled model may seem much smarter because it can repeatedly turn uncertainty into information:

That's qualitatively different from classic one-shot RAG.

The model controls retrieval dynamically.

That's a huge part of the power.

---

15. And that is probably the distinction you were sensing

Traditional RAG works by retrieving before inference. Agentic retrieval allows the model to retrieve, see, continue, and then retrieve again.

The information-gathering policy itself becomes part of the reasoning process.

This architecture is particularly easy and efficient with provider-hosted search, since the provider owns that entire inner loop.

But you can implement precisely this pattern yourself.

---

Practical comparison

	Provider web search	Your own web-search tool
Execution	Provider infrastructure	Your infrastructure
Agent loop	Usually provider-side	Usually your orchestrator
Extra API round trips	Fewer/exposed less	Usually yes
Model specialization	Often highly optimized	Depends on model
Retrieval pipeline	Provider-controlled	Fully yours
Search ranking	Provider-controlled	Fully yours
Context optimization	Often built in	You implement it
Citations	Usually first-class	You implement them
Observability	Limited	Excellent
Customization	Limited/moderate	Unlimited
Private sources	Limited to integrations	Excellent
Vendor lock-in	Higher	Lower
Domain-specific quality	General purpose	Can be much better

---

The shortest mental model

If I were designing agents, I'd think of it like this:

MODEL
  ↓ action
ORCHESTRATOR
  ↓ execute
TOOLS
  ↑ observation

The conceptual algorithm is the same.

The differences are in integration, post-training, latency, retrieval quality, context management, and operational control.

And so I would not choose an LLM provider purely because it has native web search. If a model has good reasoning and good generic tool-use capabilities, you can build a great (or for specialized use cases, better) search agent on top of it. The native tool is much more important if you want strong general-purpose research with minimal engineering.

The most interesting next layer here is how reasoning models are actually trained to decide when to call tools -- i.e. how tool use appears in SFT/RL trajectories and why that produces better agents than merely teaching a model a JSON function-calling grammar. That's where the architecture starts connecting directly to the model training itself.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com

Email updates

Usually a new article and a few links I found interesting.

Email address

Subscribe

No spam. Unsubscribe with one click.

Related posts

More writing around this topic.

Sep 13, 2026 LLMNews

My First Week With GPT-6 Astra

GPT-6 Astra was my main Codex driver for the last week, and I am back on GPT-5.5. That sounds harsher than my…

Read post

Aug 11, 2026 AI EngineeringGuides

How Floating-Point Determinism Affects LLM Reproducibility

Large language model (LLM) inference is the process of producing output tokens from an input prompt. It uses floating-point arithmetic for matrix…

Read post

Aug 11, 2026 AI EngineeringGuides

Can Provider Routing Change LLM Outputs?

Provider routing can change an LLM's output when a request reaches a different model version, fallback model, parameter configuration, precision level, inference…

Read post