# Choose what to change

Diagnose the failure before choosing a model or training method.

Author: Murali Sid (https://linkedin.com/in/muralisid)
Source: https://www.agenticarchitectureskills.com/ladder/choosing-an-approach
Published: 2026-09-17
Updated: 2026-10-15
Licence: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/
Cite as: Murali Sid. Choose what to change. Agentic Architecture Skills. https://www.agenticarchitectureskills.com/ladder/choosing-an-approach

---


An agent schedules an inspection that was completed yesterday. Should we buy a more capable model, rewrite its instructions, or train it? **First find out what prevented it from doing the job correctly.**

<TeachingIllustration src="/images/ladder/diagnosis-software-v1.webp" title="Find what failed before changing the model" alt="An explicitly labelled AI model produces an inspection plan without the completion record. Diagnose unclear instructions, missing evidence, failed tools, or remaining model limitations before selecting a change." caption="Illustrative inspection case. The AI model is software; the inspection is its task. The same wrong answer can have several different causes." />

## A stronger model may not fix the problem [#a-stronger-model-may-not-fix-the-problem]

It is tempting to treat every wrong answer as insufficient intelligence. In this example, the completion record may never have reached the agent. It may concern a different inspection case. Or a failed tool call may have been mistaken for a successful lookup. Changing the model without checking these possibilities leaves the cause unresolved.

The [intelligence ladder](/ladder) explains the available ways to build capability. This page helps choose which part to change. The aim is a better completed task at an acceptable full cost.

## Start with a failed task [#start-with-a-failed-task]

Keep the request, allowed actions, sources available at the time, tool results, and actual outcome. Ask the inspection owner what an acceptable result would have been. A useful answer might have three parts. Identify the completed inspection. Check unresolved findings. Request a further inspection only if the evidence supports it.

That gives us something concrete to test. "Make the agent more reliable" does not.

## Run a comparison that separates the causes [#run-a-comparison-that-separates-the-causes]

<TeachingIllustration src="/images/ladder/comparison-software-v1.webp" title="Test a specific explanation for the failure" alt="Two runs use the same task and model. One lacks the completion record; the other receives the correct evidence. Compare decisions, human review, time, and full cost, then repeat on held-out cases." caption="The outputs are examples, not measured results. Keep other settings comparable and repeat runs to account for variation." />

First supply the correct completion record to the same model, keeping the task and other settings comparable. If its answer improves consistently, investigate the information path. This is evidence for a retrieval or context problem, not proof that the whole system is fixed.

Then test the proposed fix through the real retrieval path. Include inspections that were not completed, old records, wrong asset identifiers, and missing information. Supplying the answer manually is a diagnostic test; it is not the production solution.

## Check the information [#check-the-information]

Could the agent access the correct source? Did parsing retain the completion date and asset identity? Was the current version retrieved? Did the evidence actually reach the model? Use [context testing](/patterns/how-to-test-a-context-design) and [memory organization](/memory/information-to-memory) to examine this path.

## Check the task and tools [#check-the-task-and-tools]

If evidence is adequate but the requested result is unclear, try clearer instructions and examples. If the agent selects a sensible action but cannot complete it, inspect permissions, tool inputs, errors, and confirmation. A submitted inspection request is not a completed inspection. Test retries without creating duplicate work.

## Check the workflow [#check-the-workflow]

If the same steps are always required, compare a fixed workflow. If the next check depends on the result of the previous one, compare a bounded investigation. If separate investigations are useful, evaluate [one agent or several](/ladder/single-and-multiple-agents). Changing coordination should address an observed difficulty.

## Then consider changing the model [#then-consider-changing-the-model]

When the task, evidence, and tools are adequate, compare another model, better examples, or training against the remaining limitation. If capability is already good but expensive, test whether a smaller model or [distillation](/ladder/distillation) can preserve the required quality.

Use tasks held apart from development. Count unsuccessful attempts, expert review, latency, and maintenance alongside model charges. Retain changes that improve the agreed outcome enough to justify their cost. See [value and full cost](/use-cases/value-and-investment).

A new model, prompt or tool is a change to a running agent. It re-runs the evals at the change stage of [the agent lifecycle](/architecture/agent-lifecycle). [The learning flywheel](/architecture/learning-flywheel) sets out how the change is tested and promoted.

## Evidence and limits [#evidence-and-limits]

This diagnostic order and the inspection case are teaching proposals, not reported experimental results. Anthropic's [Building effective agents](https://www.anthropic.com/engineering/building-effective-agents) supports starting simply and testing added complexity (19 December 2024, reviewed 18 September 2026).

## Next step [#next-step]

Start with the cheapest change, [step 1: instructions](/ladder/instructions).