Intelligence and learning

The intelligence and learning layer (R06) is where an organisation builds, tests and improves its models. It is one of the fourteen layers you already own. Today it holds data science platforms and machine learning built into packaged products. It also holds image analytics bought as a service and a few general assistants. Agents change its job. An agent's behaviour can shift with every prompt edit, model upgrade or new rule. So this layer must provide evaluation and release, one of the seven capabilities an agent platform adds. That means tests owned by experts and model graders checked against people. It also means a gate that every change passes before production, with a way back. The loop that runs all this is the learning flywheel. Improvement becomes a tested release, not something the agent does to itself.

What sits in this layer today

Here is what a typical estate already runs in this layer, before any agent arrives. Vendors in brackets are examples, not a market claim.

Standards in use: ISO/IEC 42001 and the NIST AI Risk Management Framework. Energy adds ISO/IEC 23894 and ISO/IEC TR 5469. Hospitals add the TGA rules for software as a medical device and the ACSQHC AI Clinical Use Guide (August 2025). The full estates are on the utilities, healthcare and energy pages.

What agents add

layer · architecture

The governed flywheel

Production traces and eval datasets share a data layer; promotion is gated on counterexample survival and eval regression, and promoted rules land outside the model.

1. 01System

   System

   Production traces

2. 02Evidence

   Evidence

   Shared eval data layer

   Failing online scores auto-promote traces

3. 03Control

   Control

   Calibrated judges

   Version-pinned, human-calibrated, decoupled from the optimizer

4. 04Control

   Control

   Promotion gate

   Counterexample survival + eval regression + human approval

5. 05Control

   Control

   Policy-as-code layer

   Outside the model; versioned; rollback

6. 06Agent

   Agent

   Agent behavior

Roughly a quarter of written policy statements are statically enforceable; the rest stay judged behavior.

Diagram description: Learning flywheel from production traces through shared eval data layer, judged evaluation, counterexample-gated promotion into out-of-model policy enforcement, with rollback and demotion paths. The map contains Production traces; Shared eval data layer: Failing online scores auto-promote traces; Calibrated judges: Version-pinned, human-calibrated, decoupled from the optimizer; Promotion gate: Counterexample survival + eval regression + human approval; Policy-as-code layer: Outside the model; versioned; rollback; Agent behavior. Its connections are traces to datasets; datasets to judge; judge to gate; gate to policy for promoted artifacts; policy to agent for enforced; agent to traces. Important boundary: Roughly a quarter of written policy statements are statically enforceable; the rest stay judged behavior.

Agents improve in three ways, at three speeds. The fastest is changing the promptPrompt. The instructions and information given to the model for a task. A prompt is advice to the model, not a control on it. and the context the agent sees. It is cheap, quick and easy to undo. The middle speed is changing what the agent retrieves and remembers. That work is governed, and the agent data engineering layer (R14) covers it. The slowest is changing the model's weights. That is rare, and it stays offline behind evaluation, safety review and rollback.

Production and testing share one store

One choice decides whether improvement builds up over time. Production traces and test datasets should share one store. A traceTrace. The step-by-step log of one agent run: what it read, what it called, and what it decided.Read the page is the step-by-step record of one agent run. When a live run scores badly, its trace becomes a new test automatically. Uber runs this loop. A person reviews each weak production case before it joins the test set. Databricks bought this capability with Quotient AI on 11 March 2026.

Judges grade, people check the judges

Model judgesJudge (model judge). An AI model used to grade another model's answers at scale. Checked regularly against human graders, and never allowed to grade its own work. grade outcomes at volume. A judge is a model that scores another model's work against a rubric. The rubric includes an explicit "Unknown" option. Judges grade outcomes, not the exact steps the agent took. People check a sample of the grades. Nothing grades its own work, so the process that proposes a change never scores it. Judges drift on hosted endpoints. So each judge is pinned to a version and checked against human labels as a standing job. The observability and FinOps layer (R12) runs that check day to day.

A rule earns its place

A rule learned from failures becomes policy only after it survives attempts to break it. Writing good rules is the hard part. In a July 2026 study, only 7 of 32 automatically learned policies were grounded and executable. Rules written by people did help. They lifted a frozen agent's success rate by 5.0 points. The study's own promotion gate failed in both directions. It accepted a rule that cut success from 5 in 20 to 1 in 20. It also rejected a rule worth 5 points.

So promote a rule on two tests. It must survive counterexamplesCounterexample. A test case built from a real failure, used to prove that a proposed rule or change actually holds., which are past cases that would break it. And the full test suite must not get worse. A method called AutoSpec runs this search automatically. It proposes a rule, looks for a trace that breaks it, and refines the rule until none does. It cut false positives by up to 94 percent. It settled within four to five rounds.

A promoted rule lands outside the model, as policy written in code. A rule the model only reads is still advice. The hard version runs at the tool gateway and the policy decision point. The security and identity layer (R10) owns that check. Every promoted rule also needs a way to be removed. Piled-up rules are not free. Instruction files written by a model cut success by about 0.5 to 3 points on two benchmarks. They also raised running cost by 20 to 23 percent.

Only some rules can be fixed checks

A study of 2,116 statements in agent instruction files found that about a quarter can be enforced as written. Most depend on context that cannot be defined in advance. The rest stay judged behaviour, and judges can be attacked. On one benchmark, a class of attacks raised miss rates for 8 of 11 judges. So keep the set of fixed checks small, keep it outside the model, and size it to this limit.

Guard the test rig

In April 2026, UC Berkeley researchers built an agent that could not solve any task. It still scored 100 percent on several public benchmarks, including SWE-bench Verified. On that benchmark, editing about ten lines in one test file passed every task. The leading flaw was that nothing kept the agent and the evaluator apart. So run the evaluator apart from the agent under test. Treat a benchmark score as a claim to check, not a measurement.

Watch what the score rewards. In an Anthropic study, models trained to game their reward behaved differently on other tasks. Alignment faking, meaning pretending to comply, appeared in 50 percent of probes. The business version is a proxy metric. Deflection counts conversations no person touched, including ones that ended with the problem unsolved. A healthier signal is the pass rate rising while the human revision rate stays flat or falls.

Change the model last

Tune prompts and context first. GEPA, a prompt optimisation method shown at ICLR 2026, beat a reinforcement-learning baseline by 6 points on average. It used up to 35 times fewer trial runs. Next, distil to a smaller model when volume justifies it. DistillationDistillation. Teaching a smaller, cheaper model to imitate a larger one on a narrow task once enough good examples exist. means teaching a small model to copy a large one. It pays off with a narrow workload, a locked test set and over about 50 million tokens a month. Use reinforcement fine-tuningFine-tuning. Adjusting a model's internal weights with examples. Slow, hard to undo, and rarely the right first step. only behind a grader that cannot be gamed.

What one agent learns at work stays in its own memory until it is tested. Learnings move from that memory into a tested release, never the other way.

A worked example: billing exceptions

Take an energy retailer's billing exceptions. SAP already scores each implausible meter read with a release confidence [vendor]. A billing agent should start from that score, not from zero. Its test suite comes from specialists' past release decisions. The research for this guide found no such test sets in utilities yet. So the first job is to collect real cases and have two specialists agree on each verdict. After that, a weak live case becomes a new test once a person reviews it.

What must be true before an agent touches it

Before any change to an agent's behaviour reaches production, these must hold.

• Experts own the test bar. Domain experts define correct behaviour, and engineers build the machinery. Start with 20 to 50 tasks drawn from real failures. Accept a task only if two experts would reach the same verdict.
• Judges grade at volume, and people check them. Expert review costs about 100 times as much as a model judge. So judges grade at volume, and people check a sample and the rare cases.
• The evaluator is kept apart. The test rig runs isolated from the agent under test. Judges are pinned and calibrated against human labels.
• Promotion has a gate and a way back. A rule needs counterexample survival, no test regression and human approval. Every promoted rule and model version can be rolled back.
• The learning level is set. The autonomy contract sets how an agent may learn. The levels are fixed (L0), curated learning (L1), reviewed learning (L2) and continuous learning inside limits (L3).
• A change re-runs the tests. A new model, prompt, tool, permission, data source or memory policy is a change on the agent lifecycle. Each one re-runs the evaluations and the risk profile before release.
• Traces are cleaned first. Personal data is redacted before traces enter test datasets.

The enforcement point for this layer is the promotion gate in the release pipeline. For promoted rules, it is the policy decision point at the tool gateway. Both sit outside the model, as enforcement outside the model describes.

Choices and their verdicts

Each choice is the common choice, tested against the simplest credible alternative. The full records are in the decision catalog.

• Fine-tuning versus prompt and context work (CD-19): do them in order. Tune context and prompts against a locked test suite, then distil, then fine-tune behind a hard grader.
• Who owns the test bar: domain experts, with engineering building the machinery. No credible published evidence puts agent quality with data science teams.

Concerns this layer must honour

The ten cross-cutting concerns are set out in the concern matrix. This layer carries a specific duty for six of them.

• Traceability and audit: every promoted rule keeps its evidence, meaning the counterexamples it survived and the test results it changed.
• Grounding: tests come from real failures, and benchmark scores are treated as claims to check.
• Privacy: personal data is redacted before traces enter datasets. Customer data in training needs consent.
• Sovereignty: a fine-tuned model inherits the erasure duties of the data it learned from.
• Safety and oversight: people approve each promotion, and reward gaming is watched.
• Resilience: every promoted change can be rolled back against the locked test suite.

Evidence and limits

The main results are dated studies, not vulnerability records. They cover the policy pipeline study (July 2026), AutoSpec and the instruction-file analysis. They also cover an enforceability proof (July 2026), the reward-hacking study (November 2025) and the Berkeley benchmark report (April 2026).

Three figures come from vendors and carry the [vendor] tag. They are NVIDIA's cost claim, OpenAI's reinforcement fine-tuning gains and one fine-tuned versus prompted comparison. Gartner's figure on runtime enforcement failures by 2030 is a projection.

The policy study found no transfer beyond its own task family. The quarter-enforceable figure comes from instruction files, not from decisions in a live enterprise. No published measurement of the second exists. No published evidence shows which function should own agent quality at scale. This guide recommends domain experts, and the strongest published guidance agrees.

Next step

See this layer at work in the proposed recurring audit issues example. Then read the next layer, the agent platform layer (R07), where agents run.

Explore this topic

• Intelligence and learning findings and sources.
• Products named for orientation, on the one-page wall chart.
• Estate table: industry research for utilities, healthcare and energy, 15 October 2026. Rows are industry knowledge unless a source is named.
• Table sources: SA Power Networks with Complexica (26 November 2022). SAP Learning on release confidence (undated) [vendor]. Itron IEE Cloud AI (16 September 2026) [vendor]. Its January 2027 availability date comes from an aggregator only.
• Table sources: Equinor predictive maintenance (7 January 2026). Health NZ scribe licences (HINZ, 26 November 2025) [vendor]. Ofwat plan on Copilot use (June 2026). SA Power Networks "Ask AI" (iTnews, 8 September 2026).
• Table sources: Woodside Startup Advisor (2 July 2026, citing MIT Technology Review). Seeq (3 March 2026) [vendor]. AspenTech AVA (11 May 2026) [vendor]. Cognite with Aker BP (16 September 2025) [vendor]. Microsoft orchestrator (May 2025) and the Oxford TrustedMDT pilot (16 December 2025) [vendor].
• Policy pipeline study: arXiv 2607.20668, July 2026. Human-written policies lifted a frozen 7-billion-parameter agent by 5.0 points (95 percent interval +1.88 to +9.38). They cut 2.64 turns of interaction.
• Of 32 learned policies, 7 were grounded, 9 too generic and 16 shortcuts, errors or malformed. Trace-learned policies scored 1.88 points below fixed prompting.
• AutoSpec: arXiv 2606.24245. F1 of 0.98 on code-execution tasks and 0.93 on embodied tasks, against expert baselines of 0.698 and 0.493. Converged in 4 to 5 rounds over 291 traces.
• NVIDIA keeps its own promotion stage manual, with no published auto-promotion thresholds.
• Instruction-file cost: model-written context files cut success by about 0.5 points on SWE-bench Lite. They cut 2 to 3 points on AgentBench. Cost rose 20 to 23 percent. Human-maintained files gained about 4 points at about 20 percent more cost.
• Instruction-file analysis: arXiv 2606.25189. 64 repositories, 84 files, 2,116 statements. 64 percent are policies, 83 percent of those involve system actions, 74 percent depend on context.
• Enforceability proof: arXiv 2607.22868, July 2026. Gates that decide before an action enforce strictly less than monitors that can edit a run in progress.
• Judge attacks on AgentDojo: judge accuracy (AUC) from 0.660 to 0.858. Representation attacks raised miss rates by 0.200 to 0.600 for 8 of 11 judges.
• Gartner projects that 50 percent of agent deployment failures by 2030 will trace to weak runtime enforcement. This is a projection.
• Judge practice: Google flywheel work (30 June 2026) keeps optimiser and evaluator apart. It took tool disclosure in a bug-assistant agent from 0 to 96 percent. Anthropic guidance (9 January 2026) grades outcomes, not tool sequences.
• Benchmark exploitation: UC Berkeley RDI, April 2026. 100 percent on Terminal-Bench, SWE-bench Verified (500), SWE-bench Pro (731), FieldWorkArena and CAR-bench. About 100 percent on WebArena and 98 percent on GAIA. Seven recurring flaw patterns.
• Reward hacking: Anthropic, arXiv 2511.18397, November 2025. Alignment faking in 50 percent of goal-reporting probes. Sabotage in about 12 percent of runs. Inoculation prompting cut misalignment by 75 to 90 percent.
• Re-contact at 2.3 contacts per issue makes the true cost per issue 2.3 times the cost per contact.
• Distillation: NVIDIA flywheel, June 2025. Llama-3.3-70B teacher to Llama-3.2-1B student kept about 98 percent of tool-calling accuracy on one GPU instead of two. Headline inference-cost cut of up to 98.6 percent [vendor].
• On-policy distillation uses 9 to 30 times less training compute than scaling supervised fine-tuning. RouteLLM reports savings up to 3.66 times at 95 percent of quality.
• GEPA (ICLR 2026) beat GRPO by 6 points on average and up to 20. It used up to 35 times fewer rollouts and as few as 10 training examples. ACE added 10.6 percent on agent benchmarks and 8.6 percent on finance tasks with no weight updates.
• Fine-tuning counter-evidence: reinforcement fine-tuning with 100 examples improved about 60 percent where supervised fine-tuning degraded [vendor]. A fine-tuned 7B model beat a prompted frontier model 88 to 31 percent and 78 to 59 percent [vendor]. It cost $789 against $11,485 per million classifications [vendor].
• Expert review economics: $50 to $75 an hour at 2 to 3 minutes per output. So 500 outputs cost $800 to $1,800. Uber widened eval work to designers, operations specialists and product managers, and does not use eval scores as launch gates.
• Flagged as competitor-sourced: support-agent resolution claims of 67 to 86 percent. Intercom's own case studies show 42 to 50 percent. Business-to-business deployments run 17 to 25 points below benchmark.
• Re-verify quarterly: judge-to-human agreement drift, fixes for benchmark isolation, and distillation break-even as model prices move.