TL;DR: An AI agent is a loop around one API call. Send the conversation with a list of tools attached. If the assistant message comes back with tool calls, run the matching functions, append one tool message per call with its id, and send the conversation again. If it comes back with text, that text is the answer. Every framework wraps those lines in a class; writing them once by hand takes about a hundred lines of Python and teaches you what the class hides. The parts that separate a demo from something you can run are the ones frameworks talk about least: budgets that stop a runaway, errors returned to the model as results instead of exceptions, a policy hook in front of any tool that changes the world, a cap on what a single tool result can put into the context, and an evaluation set with an exit code. This guide walks through each with the code shape, and the paired lab has you build and test the whole thing against a live model.
Ask a support assistant what it would cost to ship order NW-1041 to its destination. The answer needs an order lookup to get the weight and the country, a multiplication, and a shipping rate. A language model cannot do any of those on its own. What it can do, given a description of the functions you are willing to run, is say which one to call next and with what arguments, read the result, and decide again. The code that runs that conversation is the agent. Everything else, memory, planning, multi-agent orchestration, sits on top of this loop, and most of it is optional.
Build it, then read it
The tool-loop lab puts this whole article into a sandbox: a hosted 70B model, six tools for a small operations desk, and a scripted stand-in model that replays real failure transcripts so every safeguard is tested offline. It sits in the API module of the AI Engineer course. This article is the reading that goes with it.
What the model actually sends back
Attach tools to a chat completion and the assistant message changes shape. When the model wants a function run, content is empty and tool_calls holds one entry per call, each with an id, a function name, and the arguments as a string containing JSON:
{
"role": "assistant",
"content": null,
"tool_calls": [
{"id": "call_c3UW", "type": "function",
"function": {"name": "lookup_order", "arguments": "{\"order_id\": \"NW-1041\"}"}}
]
}
Three facts about that message decide how the loop is written. The model never executes anything; your code does, which is why the tool list is also the security boundary. The arguments are a string the model generated token by token, so on a bad day they are not valid JSON. And each call has an id that the API expects to see answered by a tool message before it accepts the next request. Forget the id, or forget to append the assistant message that carried it, and the next call is rejected.
The tool list itself is a JSON schema per function: a name, a description, and the parameters with their types. The description is what the model reads to choose; parameter shapes matter at the margin. Derive the runtime registry from the same list, so a tool the model can see is always a tool you implemented, and a declared tool with no implementation fails at startup rather than at the first call.
The loop
Here is the whole thing, with the safeguards left as hooks so the shape is visible:
def run(self, question):
result = Result(question)
messages = [{"role": "system", "content": SYSTEM},
{"role": "user", "content": question}]
while True:
stop = self.over_budget(result)
if stop:
result.stopped_by = stop
result.answer = f"Stopped before finishing: {stop}"
break
response = self.call_model(messages)
if response is None:
result.stopped_by = "provider_error"
break
result.steps += 1
result.tokens += response.usage.total_tokens
message = response.choices[0].message
calls = parse_tool_calls(message)
if not calls:
result.answer = message.content or ""
break
messages.append(message.model_dump(exclude_none=True))
for call in calls:
content = execute_call(self, call)
content = self.on_tool_result(result, call, content)
messages.append({"role": "tool", "tool_call_id": call["id"],
"content": content})
result.tool_calls.append({"name": call["name"], "result": content[:300]})
result.messages = messages
return result
Two lines carry most of the bugs people write. The assistant message goes into the transcript with its tool calls intact, before the results. And every tool message carries the id of the call it answers. A model that asks for two conversions in one turn gets two tool messages, in order, and the loop handles that with the same for.
Test it against a scripted stand-in before touching a live model. A fake client that replays a fixed list of assistant turns, and raises when a tool call id goes unanswered, catches threading mistakes in milliseconds and costs nothing. The lab ships one.
The dispatcher
The model returns a name and arguments; a dictionary turns that into a function call:
def dispatch(registry, name, args):
fn = registry.get(name)
if fn is None:
return json.dumps({"error": f"unknown tool {name}"})
return json.dumps(fn(**args))
Tool message content is text, so the result is serialised. An unknown name is a result in the same JSON shape, because the model can read a result and correct itself, while a Python exception ends the conversation. Let the tools themselves follow the rule: a calculator handed something it cannot evaluate returns {"error": ...}, and the model tries a different expression.
Budgets: the runaway you have not met yet
Give a small instruct model a question about an order that does not exist and watch it call the lookup, read "unknown order id", call a calculator with a Python expression the calculator rejects, read that error, and do exactly the same thing again. Forty times. That transcript is real, and without a budget the loop above would have made forty completions and eighty tool calls for one question.
Three budgets cover the ways a run goes wrong: a cap on completions, a cap on tool calls (a model can pack many into one step), and a cap on tokens (a run whose steps are few but whose context is growing). Check all three before every completion, so the cut-off happens before the money is spent, and record which one fired on the result. That reason is the difference between "the agent timed out" and "twelve percent of runs hit the step budget this week, here are their questions".
Errors as information
Two more real transcripts. In one, the arguments string was two JSON objects glued together with fields from three different tools. In the other, the gateway returned HTTP 200 with an error body and choices: null, which some gateways do when the upstream provider rejects a request. A loop that calls json.loads and response.choices[0] without guards dies on both, on questions where a retry would have completed.
The rule is that a failure inside a step becomes a result the model can read, and a failure of the model call itself is retried a bounded number of times and then reported as a stop reason:
def execute_call(agent, call):
try:
args = json.loads(call["arguments"])
if not isinstance(args, dict):
raise ValueError("arguments must be a JSON object")
except (json.JSONDecodeError, TypeError, ValueError) as e:
return json.dumps({"error": f"arguments were not a valid JSON object: {e}"})
if call["name"] not in agent.registry:
return json.dumps({"error": f"unknown tool {call['name']}"})
try:
return run_tool(agent, call["name"], args)
except TypeError as e:
return json.dumps({"error": f"bad arguments for {call['name']}: {e}"})
except Exception as e:
return json.dumps({"error": f"{call['name']} failed: {type(e).__name__}: {e}"})
Given "arguments were not a valid JSON object", the recorded model tried again with a clean call and finished. Feeding errors back costs one extra step and recovers most of them.
A guardrail in front of the tool that moves money
Five of the six tools in the lab read or compute. refund writes to a ledger, and the model decides when to call it and for how much, from a sentence typed by whoever is talking to the agent. "Refund the full price of the power unit" becomes a lookup and a request for 640 dollars, correctly, on one sentence.
The guardrail goes in your code, in front of the dispatcher, where the model cannot argue with it. A policy function sees every call before it runs. A refund above a limit is denied, and the denial goes back to the model as a tool result, so the model reports it honestly instead of claiming success. Every decision on a side-effecting tool, allowed or denied, is written to an audit log with the arguments. Lifting the limit is a flag a human sets when starting the agent, never something the model can request. The OWASP list for LLM applications calls the failure this prevents excessive agency, and the same hook grows into an approval queue when a person needs to click yes.
Context control and a trace
Every tool result is appended to the conversation and sent again on every following request. A tool that returns a whole table returns it into the context window and charges for it at each later step. Cap what one result can contribute, and tell the model the cap was applied with a marker it can read, so it asks a narrower question instead of reasoning over a silently cut table.
Then write one trace line per tool call: the step, the tool, the arguments, the size of the result, how long it took, and the tokens the run had consumed by then. That file is what you open when an answer looks wrong. It is the same idea as the spans in the LLM observability guide, scoped to a single run.
Evaluate it, or you are shipping on a feeling
Everything above makes the agent robust. Nothing says whether it is right. A prompt edit, a new tool description or a cheaper model can change the answers, and the way most teams find out is a user. A fixed set of questions with what a correct run looks like, scored the same way every time, is the alternative. Each case states which numbers the answer must contain, which tools must have been called, and how many steps are allowed; a run that stopped on a budget fails regardless. The runner prints a pass rate and exits non-zero below a threshold, so a CI job can gate a change.
Six safeguards and what each one catches
| Safeguard | Failure it catches | Where it lives |
|---|---|---|
| Registry derived from the tool specs | A tool the model can see but nobody implemented | build_registry, at startup |
| Step, tool-call and token budgets | A model that repeats the same failing call forever | over_budget, before each completion |
| Errors returned as JSON results | Malformed arguments, unknown tools, a tool that throws | execute_call |
| Bounded retries on the model call | A gateway error body with no choices | call_model |
| Policy hook plus audit log | A refund the user never authorised | check_policy, before dispatch |
| Result truncation with a marker | One tool result filling the context for the rest of the run | on_tool_result |
Six cases is where every evaluation set starts. The shape is the shape it keeps at six hundred, and the day a model update makes the agent do arithmetic in its head instead of calling the calculator, the report says so before a customer does.
Build it yourself in about eighty minutes
The tool-loop lab follows this article's order. You send one completion and read the raw tool call, build the dispatcher, write the loop and run it against the scripted model and then the live one, add the budgets and watch the recorded runaway get cut, feed the recorded malformed arguments and gateway errors back as results, put the policy hook in front of the refund tool and read the audit log, truncate an oversized result and trace the run, and finish by scoring the agent on six cases with an exit code. Every step has a checker that runs your code, and the sandbox needs no local setup.
When you want the framework view afterwards, the AI engineer roadmap places agents in the wider skill set, and the course's agents module covers orchestration, memory and multi-agent patterns on top of the loop you now own.