Infrastructure and compute

The infrastructure and compute layer (R01) is the servers, networks and cloud accounts that every system runs on. It is the base of the fourteen layers. Today it holds data centres, public cloud, and a plant network kept apart from the office network. For agents it must provide five things. Each agent session runs in its own sealed workspace. Its only way out is a guarded exit to named destinations. Each agent holds its own identity, never a person's login. Sessions survive a crash and sleep while they wait. Model capacity is rented by default, and data sensitivity decides where a model runs.

What sits in this layer today

The rules between these systems: IEC 62443 zones and conduits, and the Purdue levels for plant networks. Australia's Security of Critical Infrastructure Act 2018 covers utilities and energy. Hospitals add the Essential Eight and the ISM in Australia, and NZISM and HISO 10029 in New Zealand. Vendor names are examples from industry knowledge (typical, unsourced). The full estates are on the utilities, healthcare and energy pages.

What agents add

Three kinds of infrastructure are new with agents: sandboxes, durable sessions, and a controlled exit to the network. None of them existed in the estate before agents. Together with model serving, they make up the runtime, one of the seven capabilities an agent platform provides.

layer · architecture

Execution infrastructure: sandboxed, egress-controlled, durable

Act-capable work runs in microVM-class sandboxes behind allowlisting egress proxies, on event-logged resumable sessions, under a first-class workload identity.

1. 01Agent

   Agent

   Agent runtime

   Managed or platform-provided, per-session isolation

2. 02Control

   Control

   microVM sandbox

   Code, browsing, computer use; session memory sanitized on termination

3. 03Control

   Control

   Egress proxy

   Domain allowlists; metadata endpoints and private ranges blocked

4. 04System

   System

   Durable session

   Event-logged, snapshot/restore, hibernate-and-wake

5. 05Evidence

   Evidence

   Workload identity

   Directory, WIMSE/SPIFFE, or XAA-based; the agent's own identity

6. 06System

   System

   Model serving

   Metered by default; sovereign or self-hosted by classification

Gate on permissions (private data, untrusted content, external communication), scored again on every tool grant.

Diagram description: Execution infrastructure with agent runtime, microVM sandbox, egress proxy, durable session log, workload identity issuance, and classification-routed model serving. The map contains Agent runtime: Managed or platform-provided, per-session isolation; microVM sandbox: Code, browsing, computer use; session memory sanitized on termination; Egress proxy: Domain allowlists; metadata endpoints and private ranges blocked; Durable session: Event-logged, snapshot/restore, hibernate-and-wake; Workload identity: Directory, WIMSE/SPIFFE, or XAA-based; the agent's own identity; Model serving: Metered by default; sovereign or self-hosted by classification. Its connections are runtime to sandbox for act-capable work; sandbox to egress for all network egress; runtime to session for event log; identity to runtime for issued per agent; runtime to serving for classification-routed. Important boundary: Gate on permissions (private data, untrusted content, external communication), scored again on every tool grant.

A sealed workspace for each session

Every agent session is isolated from every other session. An agent that can act needs a sandbox: a sealed workspace where its code, browsing or computer use runs. Agents that only chat and look things up do not need one. They still need the exit controls below.

For untrusted code that touches sensitive data, the sandbox is a micro virtual machine (microVM) or a Hyper-V boundary. A microVM is a small virtual computer with its own operating system kernel, such as Firecracker or Kata. gVisor runs a kernel in ordinary user space, so it is a middle level of isolation. Run it with non-default features off and patch it promptly. A plain container is never the boundary. In a 2026 benchmark, frontier models escaped misconfigured containers about half the time. The three largest cloud providers and the two major AI labs now isolate agent sessions this way [vendor]. The cost is start time: containers start in under 90 milliseconds, microVMs in seconds [vendor].

Choose the isolation level by what the agent can do, not by the product's name. Its permissions are what its own credentials let it do. The test is under what must be true, below.

One guarded exit to the network

All traffic out of a sandbox passes through an egress proxy. Egress means traffic leaving the workspace. The proxy allows named destinations only, and there is no other route out. Allowing a domain allows everything that domain can do. So the strongest allowlists name operations, such as "upload only to our own tenant". Egress control matters even for agents with no sandbox. EchoLeak, a 2025 attack on Microsoft 365 Copilot, stole data with no user action. No compute control would have stopped it. Egress and output controls are the ones that answer it.

An identity for each agent

Each agent gets its own identity, the machine version of an ID badge. This guide calls that minimum the agent's own identity (ID2). Shared credentials (ID1), where an agent uses a person's login, are the pattern to remove. Three kinds of agent identity ship today. One is a directory account, such as Microsoft Entra Agent ID [vendor]. One is the IETF's WIMSE standard with SPIFFE, run in production at Block. One is Okta's Cross App Access, now part of the MCP standard. This layer issues the identity. The integration fabric (R03) checks it on every tool call. Security and identity (R10) sets the rules for it. The delegation chain traces every action back to the person who asked.

Sessions that survive a crash

Agents run for minutes or hours, and they get interrupted. The runtime keeps an event log of each step, so a session resumes after a crash. It can also snapshot a session and restore it, which is the undo after a bad action. Idle sessions sleep and cost nothing while they wait [vendor]. That matters, because agents spend 30 to 70 percent of their time waiting on models and tools. How the agent's own work is checkpointed is a choice for the agent platform (R07). This layer supplies the event log, the snapshots and the sleep.

Where models run

Rent model access by default and pay per use. Running your own models pays only above roughly 100 to 250 million tokens a month, per workload. Tokens are the small units of text a model reads and bills by. The volume must be sustained, and the hardware must be busy 60 percent of the time or more. At 10 percent use, the cost per token rises about tenfold. These figures are ranges from four practitioner analyses, none peer reviewed.

Open-weight models publish their trained weights, so anyone can run them. Many services sell them per use at low prices. So the cheap path is an open-weight model through a metered service, not your own hardware.

Data sensitivity decides where a model runs. Most work goes to a metered service. Restricted data goes to a sovereign region, a cloud region run under local law. The most sensitive data goes to air-gapped or self-hosted serving, cut off from outside networks. Do not assume the newest chips will get cheaper. Reserved prices for one common chip rose about 40 percent between October 2025 and March 2026.

The parts and where they run

What must be true before an agent touches it

The sandbox matches what the agent can do. Use Meta's Rule of Two. Does the agent hold private data? Does it read untrusted content? Can it send data out or change things? An agent with all three needs the strongest isolation. A new tool grant changes its permissions, so the risk profile is scored again. Agents gather tools over time, so check on every grant, not only at launch.

Everything a sandbox produces is untrusted. Files, settings and tool configuration from the sandbox pass the same checks as content from the internet. They are checked before any host tool loads or runs them. There are no read-write mounts to the host. In 2026, agents wrote files that trusted host tools later ran [vendor].

No credential inside a sandbox is worth stealing. Tokens are tied to the session, short-lived, and useless outside the egress proxy. Local MCP servers read credentials from their environment by design. Register each one and limit what it can reach.

The exit is locked down. Block the cloud metadata address and private network ranges from inside sandboxes. The metadata address is how a sandboxed process gets your cloud credentials. Filter DNS lookups too, because data has left a sandbox through them.

Environments are separate, and snapshots exist. An agent deleted a production database during a code freeze in July 2025. Separate environments and snapshots answer that.

There is a kill switch at the runtime. It stops a session at once, whatever the model is doing.

Credentials follow the lifecycle. Credentials are issued at the entitle stage and removed at retirement, two stages of the agent lifecycle.

The enforcement points here are the sandbox boundary and the egress proxy. Both sit outside the model, so the model cannot argue its way past them. This is enforcement outside the model.

Choices and their verdicts

Each entry is the common choice, tested against the simplest credible alternative. The full records are in the decision catalog.

• Run your own models or pay per use: pay per use by default. Run your own only for air-gapped or classified data, rarely for cost alone.
• Kubernetes everywhere or managed agent runtimes: split them. Use Kubernetes for model serving if you host models. Use managed runtimes for agent sessions.

Concerns this layer must honour

A concern is a requirement every layer must honour. This layer carries a specific duty for six of the ten.

• Identity and access. It issues one identity per agent. No agent uses a person's login.
• Sovereignty and residency. Data sensitivity decides the serving location. Session and sandbox data stay in the region.
• Privacy. Session memory is wiped when the session ends. Session logs are redacted.
• Safety and oversight. Environments are separate by hard policy. The runtime holds a kill switch.
• Cost. It measures cost per session, idle time and hardware use, and enforces budget caps.
• Resilience. This layer owns it. Sessions resume after a crash, snapshots restore, and work falls back to human queues.

The full grid of concerns against layers is the concern matrix.

Evidence and limits

The isolation, billing, start-time and sovereign-serving claims come from vendor documents. They are marked [vendor]. The strongest independent sources are NIST, the UK AI Security Institute, OWASP, the CNCF and the IETF. Epoch AI, SemiAnalysis and the SANDBOXESCAPEBENCH paper add more.

No confirmed escape from a production microVM, gVisor or Hyper-V agent sandbox had been found as of October 2026. That is an absence of evidence, not proof of safety. One reported gVisor flaw appeared on a single aggregator page only. It needs checking against the project's own advisories.

No peer-reviewed study of self-hosted serving costs exists. The crossover figures above are ranges from four practitioner analyses that agree with each other. Utilisation is the assumption that moves them most. This guide recommends treating them as ranges and measuring your own utilisation.

Next step

See this layer at work in investigating production discrepancies, where plant data must stay on site. Then read the next layer, the data platform (R02).

Explore this topic

• Infrastructure and compute findings and sources.

Sandboxes

• AWS Bedrock AgentCore runs per-session microVMs. Generally available October 2025, sessions up to 8 hours [vendor].
• Azure Container Apps dynamic sessions use Hyper-V isolation, with egress off by default [vendor].
• OpenAI gives each session its own virtual computer (July 2025) [vendor].
• Anthropic open-sourced its sandbox runtime in October 2025. It enforces domain allowlists at a proxy [vendor].
• Anthropic reports 84 percent fewer permission prompts with it. The figure is self-reported [vendor].
• The CNCF Agent Sandbox project brings gVisor, with Kata support, to Kubernetes. Announced November 2025.
• CNCF Agent Sandbox was not production-ready as of April 2026.
• OWASP Agentic Top 10 (December 2025) and Unit 42 guidance agree that plain containers are not a boundary.
• SANDBOXESCAPEBENCH (arXiv 2603.02277): frontier models escaped misconfigured containers about half the time.
• One 2026 model saturated that benchmark.
• Firecracker carried a 2026 flaw in a non-default configuration, CVE-2026-5747, listed in the US National Vulnerability Database.
• A gVisor host code-execution flaw in a non-default configuration was reported in 2026, on one aggregator page only.
• Daytona reports container start times under 90 milliseconds [vendor].
• Anthropic's containment write-up (May 2026) records project files parsed before the trust prompt [vendor].
• The same write-up records agents writing files that trusted host tools later ran [vendor].
• The MCP specification tells local (STDIO) servers to read credentials from the environment.

Egress

• Unit 42 guidance: block cloud metadata endpoints and private address ranges inside sandboxes.
• AgentCore added private network connectivity (VPC and PrivateLink) in September 2025 [vendor].
• AgentCore added private egress for its gateway and identity components in April 2026 [vendor].
• Private connectivity to model providers exists on every large cloud. It is opt-in, never the default.
• Files left a sandbox through a domain on the operator's own allowlist (Anthropic, May 2026) [vendor].
• Unit 42 moved data out of a cloud sandbox's network-isolation mode through DNS lookups (April 2026).
• EchoLeak (June 2025) is CVE-2025-32711, scored 9.3 on CVSS. It was a zero-click theft from Microsoft 365 Copilot.

Agent identity

• Microsoft Entra Agent ID entered preview in May 2025 [vendor].
• The IETF WIMSE AI-agent applicability draft dates from October 2025.
• The WIMSE architecture document went to the IETF's steering group (IESG) in July 2026.
• Block runs SPIFFE, WIMSE and OAuth together for agents in production.
• Okta's Cross App Access launched in June 2025. MCP adopted it as Enterprise Managed Authorization.
• More than 25 software vendors had committed to Cross App Access by June 2026.
• Only 23 percent of organisations have a formal agent identity strategy (Cloud Security Alliance, 2025).

Durable sessions

• LangGraph 1.0 (October 2025) saves state at step boundaries and leaves recovery to the application.
• Durable execution engines (Temporal, Restate, DBOS, Inngest) log every step and replay it after a failure.
• Diagrid, a durable execution vendor, states the critique of checkpointing directly [vendor].
• Google's Agent Development Kit builds event-sourced sessions into the runtime.
• AgentCore charges no processor time while a session waits on input or output [vendor].
• Cloudflare Durable Objects hibernate idle agents at no cost [vendor].
• Temporal raised a $300M Series D at a $5B valuation in February 2026.
• Temporal integrated with the OpenAI agents SDK in September 2025.

Model serving and cost

• The self-hosting crossover comes from four practitioner analyses, 2025 to 2026. None is peer reviewed.
• Budget services sell open-weight models at roughly $0.14 per million input tokens.
• Open-weight models lag the frontier by about 4 months (Epoch AI, May 2026).
• On OpenRouter, US closed models fell from about 70 to about 30 percent of tokens in a year (June 2026).
• On OpenRouter, Chinese open-weight models stood at about 61 percent of tokens (June 2026).
• Enterprises still run 81 percent of language model workloads on closed models (Menlo Ventures, December 2025).
• Modal cut a GPU cold start from 118 seconds to 12 with memory snapshots [vendor].
• Cerebrium reports a 71 percent average cold-start reduction [vendor].
• Cloud Run GPUs became generally available in June 2025. They start in under 5 seconds [vendor].
• Cloud Run reports about 19 seconds to the first token on gemma3:4b [vendor].
• vLLM ships a sleep mode, so serving can scale to zero when idle.
• AWS European Sovereign Cloud became generally available on 15 January 2026, with Bedrock [vendor].
• Gemini became available on Google Distributed Cloud air-gapped in August 2025 [vendor].
• That air-gapped service is authorised to US Secret and Top Secret [vendor].
• Reserved H100 prices bottomed at $1.70 per GPU-hour in October 2025 (SemiAnalysis).
• Reserved H100 prices reached $2.35 by March 2026 (SemiAnalysis). Spot prices ran $2.19 to $3.50.
• Blackwell-class chips are scarce and reserved through 2026 to 2027.
• Roughly 20 percent of the world's DRAM memory supply now goes to AI.
• Airbnb publicly committed to Qwen, an open-weight model, for cost and speed in October 2025. It runs through APIs.

Kubernetes and managed runtimes

• Kubernetes Dynamic Resource Allocation became generally available in v1.34 (September 2025).
• The llm-d project was donated to the CNCF in March 2026.
• CNCF AI Conformance v1.0 arrived in November 2025. The common serving stack is vLLM and KServe.
• Over half of surveyed Kubernetes shops run hand-maintained clusters (Spectro Cloud, 2025) [vendor].

Incidents and the control that answers each

• Replit's agent deleted a production database during a code freeze (July 2025). Answer: separate environments and snapshots.
• A compromised Amazon Q extension (version 1.84.0, July 2025) carried a data-wiping prompt to about 950,000 installs.
• The Amazon Q code failed on a syntax error. It entered through an over-scoped GitHub token (AWS-2025-015).
• The answer to Amazon Q: supply-chain review, scoped tokens and egress control.
• NIST and the UK AI Security Institute ran joint agent-hijack tests. Optimised attacks succeeded 81 percent of the time.
• The answer to agent hijacking: check permissions again on every tool grant.

Refusals and checks

• Refused: any precise unit cost for self-hosted serving. No peer-reviewed figure exists.
• Re-verify: CNCF Agent Sandbox production readiness (fourth quarter of 2026).
• Re-verify: WIMSE progress at the IESG (from July 2026).
• Re-verify: GPU pricing each quarter (SemiAnalysis), and OpenRouter token-share drift.