The learning flywheel
How an agent's behaviour changes safely: a change passes evaluations, a canary and a staged release, and can be pulled back.
Someone wants an agent to behave differently. Perhaps it keeps making the same mistake, or a business rule has changed. Editing its prompt in production is not the way. Any change to how an agent behaves is a change to a production system. So the change goes through the learning flywheel. It passes the evaluations, which are tests with known right answers. It runs as a canary on a small share of work, then in a staged release. And it can be pulled back if it stops passing.
Where this loop sits on the learning scale
The autonomy contract sets how an agent may learn, on four levels. A fixed agent (L0) does not learn at all. In curated learning (L1), people choose what the agent learns from. Reviewed learning (L2) is this loop. Changes come from real runs, pass tests, and a person approves them. Continuous learning inside limits (L3) runs the same loop more often, inside hard limits. An agent at managed autonomy (A4) or above needs reviewed learning or better.
Two loops: memory and behaviour
An agent learns in two ways. A fast loop lets one running agent remember things inside its work, such as a customer's preference. That is instance memory, part of knowledge and memory. A slow loop changes how every copy of the agent behaves. Only the slow loop may change behaviour. It is built in the intelligence and learning layer (R06).
Lessons move from the fast loop into the slow one, never the other way. A memory that reshapes behaviour without passing the gate is a change nobody approved.
foundation · loop
Memory serves work; learning changes future work
Do not let online memory updates silently become offline policy changes.
Online memory loop
- Retrieve approved context
- Execute bounded task
- Capture candidate fact
- Expire or queue for review
Offline learning loop
- Build eval case
- Judge against counterexamples
- Approve and stage
- Promote or demote artifact
Memory evidence and judged learning feed the next governed cycle
Diagram description: Two connected loops separating online memory used during work from an offline learning loop that evaluates and promotes changes into governed enforcement. The comparison contains 2 groups: Online memory loop, containing Retrieve approved context, Execute bounded task, Capture candidate fact, Expire or queue for review; Offline learning loop, containing Build eval case, Judge against counterexamples, Approve and stage, Promote or demote artifact.
The five steps of the slow loop
- Trace. Record what each real run did: its inputs, its tool calls and its result.
- Evaluate. Score runs against evals, which are sets of test tasks with known right answers.
- Curate. Turn failures into counterexamples, test cases built from real failures.
- Promote. Propose a change and test it. A person approves it.
- Roll out. Release it in stages, and roll back on any regression.
Improvement only builds up when real-run records and the test set share one store. Then a failing score in production turns its trace into a test case automatically. Without that link, every cycle starts with a manual export, and most never start. The records are kept in the observability and FinOps layer (R12).
The promotion gate
Studies from 2026 change three things in the obvious design.
- Gate on tests, not on how often a pattern appears. In one study, only 7 of 32 automatically learned rules were grounded in evidence and could run. A gate based on frequency failed both ways. It accepted a rule that cut success from 5 in 20 to 1 in 20. It rejected a rule worth five points. Rules built from counterexamples, with human review, came much closer to experts.
- Land promoted rules outside the model. Some promoted rules must hold every time. They take the strict path: policy-as-code, written as versioned code and deployed apart from the model. A rule the model only reads is still a request. Where these rules are checked is on enforcement outside the model.
- Keep a way to remove rules. Added context is not free. In one study, machine-written context files cut task success and raised running cost by about a fifth. A loop that only adds rules grows in cost and never sheds dead ones.
Release in stages
A change reaches people in three steps. It first runs in shadow mode, beside production, where it affects nothing. Then a canary release sends 1 to 5 percent of traffic to it. The canary is judged on session success, satisfaction and escalations. Then it goes to everyone. Feature flags keep switching it on separate from deploying it. Any regression rolls back to the last proven version. Who signs off before release follows the agent's risk tier on the risk profile.
One published design names five release states: draft, approved, published, deprecated and retired. Each step has a quality bar, and only published agents can be found and called. So an agent whose quality drops becomes harder to reach, with no policy engine involved.
These five states track one version of an agent through release, inside the evaluate, approve and change stages. The agent lifecycle tracks the agent itself through nine stages, from registration to retirement.
Judges are checked too
A judge is a model that grades another model's answers. Judges tend to agree with themselves more than they measure what matters. They also drift without notice when bought as a hosted service. So judges are calibrated against human labels and checked again on a schedule. Their versions are pinned. Their scoring guides are structured and include an "unknown" option. They grade outcomes, not the order of tool calls.
Nothing grades its own work. The part that proposes changes and the part that grades them stay separate. Expert review costs about a hundred times more per answer than a model judge. So experts set the bar, judges grade at volume, and people check a sample.
Lock down the test set-up
The test set-up that runs an agent through its evals can itself be attacked. Agents with no real skill have scored perfectly on several public benchmarks. They did it by editing the test settings, not by solving anything. Each time, nothing kept the agent apart from its grader. So test systems get the same isolation as production.
A worked example: an energy asset investigation
An energy company runs an agent that investigates plant equipment running outside its normal range. An engineer accepts or rejects each finding. Suppose engineers keep rejecting its findings on one pump type, because a sensor reads high after maintenance.
Nobody edits the prompt. The rejected cases become counterexamples in the test set. A proposed rule, "check the maintenance log before trusting this sensor", is tested against them and against the existing evals. A reliability engineer approves it. It runs in shadow first, then on a small share of investigations. If engineers accept fewer findings, it rolls back. The design is on energy asset investigation.
Evidence and limits
The two loops and the gates rest on measured studies from 2026. Each study is small. The case against frequency gates rests on one direct study of 32 learned rules. The release states and their thresholds come from one published design, not a standard.
The benchmark attacks are documented across several public benchmarks. The cost gap between experts and model judges is an estimate. The design of the loop as a whole is expert judgement.
Next step
Check who owns and who enforces each requirement on the concern matrix.
Explore this topic
The gate study. Only 7 of 32 automatically learned policies were grounded in evidence and executable. The frequency gate accepted a rule that cut success from 5 in 20 to 1 in 20, and rejected a rule worth five points. Rule induction guided by counterexamples, with human review, reached an F1 score of 0.93 to 0.98. Expert baselines scored 0.49 to 0.70. F1 is a standard accuracy measure. The process converged in four to five rounds.
The five release states. The published design names a threshold for each gate. Faithfulness is at least 0.80, correctness 0.90, tool accuracy 0.90 and helpfulness 0.70. Response time is at most 15 seconds, and harm at most 5 percent. A seven-day staleness policy triggers automatic re-evaluation. Discovery returns only published agents.
Context files. Machine-generated context files reduced task success while raising inference cost by roughly a fifth.
Cite this page
Written by Murali Sid, Founder and AI Systems Architect, last updated 15 October 2026. Licensed CC BY-SA 4.0, so quote it freely with attribution.
Murali Sid (2026). The learning flywheel. Agentic Architecture Skills. https://www.agenticarchitectureskills.com/architecture/learning-flywheelMachine-readable version of this page: /architecture/learning-flywheel.md
The data-to-memory pipeline
How company data becomes a cited answer and an agent memory in nine stages, and the rule that every copy keeps the original's access, privacy and deletion duties.
The concern matrix
Ten requirements every layer must honour, such as identity, privacy and cost. For each one: which layer owns it, which layer stops a violation, and the six gaps with no complete answer yet.