EP12 · April 2, 2026 · 7:21 · AI Dev Tools: The Crazyrouter Podcast

Building AI Agents with Tool Calling — A Practical Guide

Tool calling is what turns a chatbot into an agent: the model decides which function to call and with what arguments, your code runs it, and the result feeds back into the conversation. This episode explains the think-act-observe loop, how OpenAI, Anthropic and Google approach tool calling, how to build a first agent in four steps, and four lessons for running agents in production.

Key takeaways

  • Remember that the model never executes tools itself; it returns a structured request and your code runs the function and sends the result back.
  • Start with one tool and one model, and get the four-step loop (define, request, execute, respond) working before adding complexity.
  • Write specific tool descriptions that say what each tool does and when to use it, because the model chooses tools from those descriptions.
  • Return clear error messages from failing tools so the model can retry, change approach, or explain the problem to the user.
  • Cap the number of tool calls per turn (around ten is a sensible default) to stop agents from looping forever.
  • Log every tool call, its arguments and its result; agent behavior is non-deterministic and logs are your main debugging tool.

Show notes

What tool calling is#

Tool calling, also called function calling, means giving a language model a list of tools it may use and letting the model decide when and how to call them. Instead of only producing text, the model can trigger actions: searching the web, querying a database, sending an email, running code or hitting any API you define. Each of those capabilities is just a function you describe to the model.

A plain chatbot is like a person who can only talk. An agent with tools is a person who can talk and also use a computer. The key detail is that the model does not execute anything. It outputs a structured request such as "call this function with these arguments." Your code runs the function, returns the result, and the model continues reasoning with that information. The pattern is a loop: think, act, observe, repeat.

How the major model families handle it#

All the major providers support tool calling, but their designs differ. The examples below reflect the models discussed at the time of recording (April 2026).

  • OpenAI (GPT-5, o3): tools are defined as JSON schemas in a tools parameter and the model replies with tool_calls. It is the most widely adopted format.
  • Anthropic (Claude): a similar approach, with particular strength in multi-step reasoning about which tools to call and in what order. Claude Opus 4 was singled out as a strong choice for complex agent workflows.
  • Google (Gemini): supports function calling with optional automatic function execution, where the SDK loops through calls for you. Useful for simpler agent patterns.

If you call models through an OpenAI-compatible gateway, the tool format is standardized, so you write your tool definitions once and swap models freely.

Your first agent in four steps#

Take a weather assistant as the minimal example.

  1. Define the tool. Write a JSON schema with a name such as get_weather, a description, and a location parameter.
  2. Send the request with tools attached. The user asks "What's the weather in Tokyo?" and the model answers not with text but with a tool call: get_weather with location Tokyo.
  3. Execute and return. Your code catches the tool call, calls a real weather API, and sends the data back to the model as a tool message.
  4. Let the model respond. With the data in hand, the model writes a natural-language answer for the user.

That is the entire core loop. Using the OpenAI Python SDK it comes to roughly thirty lines of code.

Notice that the model's reply in step two contains no prose at all; it is purely a structured request. Your code therefore checks every response: if it contains a tool call, execute it and loop again; if it contains text, return that to the user.

Multi-tool agents#

Agents get powerful when they have several tools. A support agent might have search_knowledge_base, get_order_status, create_ticket and send_email. When a user asks where their order is, the model calls get_order_status. When the user then asks for a refund, it calls create_ticket and follows up with send_email to confirm. A single turn can involve zero, one or several tool calls, chained in whatever order the model judges sensible, all inside the same think-act-observe loop.

You do not write if-else routing for any of this. You define the tools and the model works out the workflow from the conversation. That shift, from hand-coded branches to model-driven routing, is what separates agents from traditional chatbots.

Four lessons from production#

Agents behave differently from ordinary request-response code. Four habits make the difference once real users depend on them:

  • Make tool descriptions precise. The model picks tools from their descriptions, so vague wording leads to wrong choices. State what each tool does and when it should be used.
  • Handle errors gracefully. Tools fail and APIs time out. Return a clear error so the model can retry, try something else, or tell the user what happened.
  • Limit the loop. An agent allowed unlimited tool calls will occasionally get stuck. Set a maximum per turn; ten is a reasonable default.
  • Log everything. Record which tools were called, with which arguments, and what came back. Without that trail you cannot debug or improve non-deterministic behavior.

Matching models to agent tasks#

Different models are good at different parts of tool use. In the episode's experience, Claude excelled at complex multi-step reasoning, GPT-5 at structured output and precise schema-following, and Gemini at fast, cost-effective handling of simpler tool patterns. A practical setup uses a stronger model for a planning agent and a smaller one, such as GPT-5-mini, for simple lookups.

Because the tool format is shared behind an OpenAI-compatible API such as Crazyrouter, you can prototype on one model, benchmark others and route each agent to a different model by changing a single model string. Start simple with one tool and one model, then add tools and experiment as your agent matures. Models are getting better at using tools with every update, so a model-agnostic design lets you adopt those improvements without rewriting your agent.

Questions from this episode

Does the LLM actually run the function in tool calling?
No. The model only returns a structured request naming the function and its arguments. Your application executes the function, sends the result back as a tool message, and the model uses it to continue. Some SDKs, such as Google's automatic function calling, run that loop for you.
How many lines of code does a basic tool-calling agent need?
A minimal single-tool agent, such as a weather lookup, is about thirty lines of Python with the OpenAI SDK. It covers defining the tool schema, sending it with the request, executing the returned call and passing the result back. Multi-tool production agents add error handling, loop limits and logging on top.
How do I stop an AI agent from looping on tool calls?
Set a hard maximum number of tool calls per turn; around ten is a reasonable default. Also return clear error messages when a tool fails so the model can change approach instead of repeating the same call. Logging every call makes it easy to spot loops when they happen.

More episodes

All episodes with show notes →

Try it on your own traffic

Crazyrouter gives you one OpenAI-compatible API key for Claude, GPT, Gemini, DeepSeek and more, billed pay-as-you-go.

Get an API key →