Step 3: Tools and the loop
How the agent loop runs, what the harness controls, and when several agents are worth their cost.
The agent finds the right evidence and still goes wrong. It calls the wrong tool, repeats itself or stops too early. What controls how the agent acts, if the model is not the problem? The answer is the loop around the model. An agent is not one call to a model. The model asks for something and the system fetches it. The model reads the result and decides what to do next. Everything that runs that loop is the harness. Teams often skip it on the way from "the context is fine" to "we need to train a model". The evidence says the harness is usually what was missing. This is the third step on the intelligence ladder.
How the loop runs
The model is stateless. It remembers nothing between calls. Every turn, your code sends the whole conversation again. The model replies with either an answer or a request to use a tool. If it is a tool request, your code runs the tool and appends the result. This repeats until the model answers or you stop it.
Four design choices follow, and together they are harness design.
The context grows every turn. Each tool result is appended, so a long investigation fills the context window with old evidence. The context window is the model's working memory for one task. You must decide what to do about that, and the choice matters. Compaction summarises the history. It loses detail and can quietly drop what the model needed. Reset starts fresh with a stated goal and a short handover. It keeps lasting state in files or a record, not in the conversation. Reset is harder to build and generally more reliable. A summary of a summary degrades, while a written record does not.
What outlives the run is memory. A record kept only for the task is working data. Once it is kept for later runs, it becomes memory. It then needs a named owner and must carry its source. It waits in quarantine before later runs trust it, and deletion must reach it. Step 2 sets out those rules.
Nothing stops the loop unless you stop it. A model can call a tool that returns an unhelpful result, decide to try again, and do that forever. Real harnesses cap the number of turns and the elapsed time. Each run also has a cost cap within the agent budget. Harnesses check for cancellation between turns and offer a stop hook, a point where code can end the run. These are not edge cases. They are the normal path.
Tools are an interface you design. A tool with a vague description gets called at the wrong times. A tool that returns ten thousand tokens of JSON fills the context window the reasoning needs. Design tools for a reader with no memory of your system, because that is what the model is. Every tool call also passes the tool gateway. The gateway checks that the agent may make the call. The gateway is one of the seven capabilities an agent platform provides.
foundation · loop
One turn of an agent loop
The model is stateless: every turn resends everything, and nothing stops the loop unless the harness stops it.
- 1
Model decides
Answer, or ask for a tool.
- 2
Gateway checks
May this caller do this, now?
- 3
Tool executes
Your code runs; the model waits.
- 4
Result appended
Context grows every turn.
- 5
Caps checked
Iterations, wall clock, cost.
- 6
Repeat or stop
Reset context; keep state in records.
The final evidence feeds the next controlled pass.
Diagram description: The agent loop as a cycle: the model decides, a tool executes, the result is appended, the harness checks its caps, and the loop repeats or stops. The sequence contains 6 stages: 1, Model decides: Answer, or ask for a tool.; 2, Gateway checks: May this caller do this, now?; 3, Tool executes: Your code runs; the model waits.; 4, Result appended: Context grows every turn.; 5, Caps checked: Iterations, wall clock, cost.; 6, Repeat or stop: Reset context; keep state in records..
The harness changes what the agent does
Hold the task, the environment and the model constant. Harness choices still measurably change the beliefs that drive the agent's later decisions. This holds even where the final success rate looks the same (arXiv 2607.04528, July 2026). In retrieval-heavy work, the harness can matter more than the retrieval design itself (arXiv 2605.15184, May 2026).
Two things follow. Published comparisons of models run in different harnesses are confounded. A claim that model A beats model B may really be a claim about the harness around each. And your own agent's behaviour depends on your loop, not only on the model you licensed.
A third finding is worth carrying. When several agents fail together, the cause is usually the design, not the model. One study built a list of failure types from 150 traces of agent runs. It checked the list on more than 1,600 traces across seven frameworks. It found fourteen failure modes in three groups, and design failure is its central conclusion. A stronger model does not fix a badly designed loop.
When several agents pay off
Splitting work across several agents buys breadth in parallel and keeps each agent's context separate. It costs roughly 1.4 to 20 times as much. It does not buy better reasoning per token.
One study tested 10,000 SEC filings across 25 field types. It compared five models and four ways of coordinating agents. A supervisor directing worker agents reached 0.921 F1 at 1.4 times the cost. F1 is a 0 to 1 score that balances missed answers against wrong ones. That design gave the best accuracy for its cost. A pattern where agents check and correct their own work reached 0.943 at 2.3 times the cost. A hybrid got 89 percent of the accuracy gain at 1.15 times the cost (arXiv 2603.22651, March 2026).
The threshold that follows is the only published one. Several agents are justified when the accuracy you need is above about 0.92 F1. Below that, you pay a multiplier for what a single well-built loop delivers.
Worked example: a vegetation candidate at a utility
A vegetation candidate arrives with a location and an observation date. The agent checks the asset map, recent inspections and existing work orders. If the image cannot show the required detail, the agent prepares a field-inspection request. It does not guess the exact clearance.
Each tool returns a bounded result with source references. The loop has a deadline, a cost limit and a stopping rule. Permission checks stay outside the model, as set out in enforcement outside the model. How far the agent may act on its own is set in the autonomy contract. Creating a request is not the same as completing the inspection.
Begin with one agent or a fixed workflow. Add specialist agents only when they improve the result, or useful parallel work, enough to justify the coordination. See vegetation inspection.
Where to go deeper
The agent platform layer (R07) covers the harness, recovery after a crash, portability and what to build or buy. The Agentic OS sets out the parts of an agent platform, which you buy and which you build. Its harness section covers what a platform provides here and what stays yours.
When this step is done
The loop is instrumented, and failures are traced to a specific cause. What remains is the model not knowing your domain.
"Traced" is the test. If you cannot say which turn failed and why, you cannot know the model is at fault. Every option that changes the model costs more than finding out. For an inspection, a trace should show the sources consulted, the tool responses, the checks made and the confirmed action.
Evidence and limits
Independently measured The harness and multi-agent findings are independent published work, not ours. They are arXiv papers from 2025 and 2026. The 0.92 threshold comes from one study on one task, extracting fields from filings. Treat it as a guide and test on your own work.
Next step
Decide whether the task needs a workflow, one agent or several, on workflow, one agent, or several.
Explore this topic
- Harness choice changing downstream decisions: arXiv 2607.04528, July 2026.
- Harness outweighing retrieval architecture: arXiv 2605.15184, May 2026.
- The multi-agent cost-accuracy frontier: arXiv 2603.22651, March 2026.
- Fourteen multi-agent failure modes in three groups (MAST): Cemri et al., arXiv 2503.13657, first posted March 2025. Title: "Why Do Multi-Agent LLM Systems Fail?"
- What a record kept past the run becomes: from data to memory and the memory-pipeline architecture.
- Depth on the agent platform layer (R07).
- Industrial scenarios on this page are proposed designs, illustrated in the use cases.
Cite this page
Written by Murali Sid, Founder and AI Systems Architect, last updated 15 October 2026. Licensed CC BY-SA 4.0, so quote it freely with attribution.
Murali Sid (2026). Step 3: Tools and the loop. Agentic Architecture Skills. https://www.agenticarchitectureskills.com/ladder/tools-and-the-loopMachine-readable version of this page: /ladder/tools-and-the-loop.md