Part II of 5 · Chapters 5 to 15

One loop under every agent, and eight levers to configure it

One loop underlies every agent. Eight levers configure it, and a test on two real coding agents shows that they describe real differences.

Chapters (11)5. Every agent runs the same loop, so the loop tells no two agents apart6. Context: what the agent knows is rebuilt, and paid for, on every turn7. Action capability: what matters is what cannot be undone8. Control loop: code or model, the one line every major vendor draws9. Decomposition and specialization: every split is paid for in coordination10. Verification and conformance: results are checked, the path almost never is11. State persistence: nothing survives a turn unless someone wrote it down12. Capability evolution: the hard part is choosing which new skills to keep13. Human control: a gate approved 93% of the time is a reflex, not a control14. The list is not closed: two dimensions sit outside the eight levers15. Tested on two opposed agents, the levers are a design grid, not a taxonomy

5. Every agent runs the same loop, so the loop tells no two agents apart

Part II begins beneath the methodologies. If dozens of them exist, is there anything they all share? This chapter identifies the one mechanism common to every agent, and shows why that mechanism, though real, cannot tell one agent from another. Beneath the proliferation of specialized frameworks and orchestration libraries sits a single underlying engine. Every agentic architecture, regardless of its visual topology or domain, reduces to an irreducible control loop composed of tool use, planning, and reflection. This foundational loop defines what makes a system agentic rather than a static prompt pipeline. However, while this first principle is universally irreducible, it carries no comparative or diagnostic power. Describing an application as a tool-using, planning, reflective agent explains its core mechanism. It does not allow designers to compare, differentiate, or evaluate distinct architectures.

Figure 5.1: Two different machines opened to show the same three-gear core. Figure 5.1: An editorial allegory of two different machines that share the same three-gear core of tool use, planning and reflection (Inside the Scaffold, arXiv 2604.03515). The core is common to both, so it cannot tell them apart. That takes the levers introduced in the chapters that follow.

As portrayed in Figure 5.1, two systems with radically divergent engineering profiles often hide the exact same three-gear escapement beneath their exterior casings. Andrew Ng, founder of DeepLearning.AI, formalized this foundational consensus by identifying four core agentic design patterns (Ng). The first pattern, tool use, empowers the model to gather information, execute actions, or process external data through APIs, code execution, or web search. The second pattern, planning, enables the model to formulate multi-step strategies. It breaks complex objectives into structured subtasks. The third pattern, reflection, prompts the model to inspect its intermediate outputs and refine its work iteratively. Ng identifies a fourth pattern, multi-agent collaboration, where multiple specialized agents split tasks and debate solutions. While the first three patterns form an internal execution loop, multi-agent collaboration represents an architectural choice regarding the division of labor.

Formalizing this loop through cognitive science demonstrates its deep alignment with classical architecture models. CoALA, a framework published in Transactions on Machine Learning Research, relates language agents to symbolic cognitive architectures (arXiv 2309.02427). It structures agentic operation across three core dimensions. CoALA divides the system into an information storage layer, an action space, and a decision-making procedure. The action space separates external grounding actions from internal operations, which comprise retrieval, reasoning, and learning. In CoALA’s taxonomy, planning and reflection correspond to internal reasoning actions. Tool use maps directly to external grounding actions.

The theoretical roots of agentic design extend into the Belief-Desire-Intention (BDI) framework (Springer). Originating from Michael Bratman’s theory of human practical reasoning and folk psychology, BDI models autonomous agency by mapping mental states to operational structures. Beliefs represent the agent’s knowledge of the environment, desires encode its long-term goals, and intentions formulate the active plans it commits to execute. Translating folk psychology into temporal logic established that autonomous agency requires explicit goal representations and deliberate plan revision. Modern language model agents inherit this framing. They rely on prompts and memory buffers to hold beliefs, extract intentions, and evaluate progress against goals.

The central constraint of operating solely at the first principle level is its inability to differentiate real-world systems. Inside the Scaffold, a taxonomy established by reading the source code of thirteen open-source coding agents, proves this limitation at code level (arXiv 2604.03515). The study demonstrates that a Monte Carlo Tree Search agent and a simple while loop with retries both qualify as tool-using, planning, and reflective systems. At the first principle level, their functional profiles are identical. Without a deeper framework, high-level primitives obscure critical engineering trade-offs in state management, execution control, and context compaction.

Table 5.1

Theoretical Framework Core Triad Mapping Operational Dimension Diagnostic Blindspot at First Principle
Agent Design Patterns (Ng) Planning, Tool Use, Reflection Workflow orchestration patterns Treats multi-agent collaboration as a fourth pattern rather than an orthogonal organizational choice.
Cognitive Architectures (CoALA, arXiv 2309.02427) Internal reasoning vs. external grounding Memory, action space, decision procedure Identifies cognitive modules without specifying whether transitions are code-deterministic or model-driven.
BDI Framework (Springer) Desires → Planning, Beliefs → Tools, Intentions → Reflection Mental state temporal logic Assumes coherent internal representations that stochastic LLM context buffers do not natively guarantee.
Code Scaffolding (Inside the Scaffold, arXiv 2604.03515) while loops, API calls, error parsing Concrete harness implementation Demonstrates that MCTS and trivial retries share the identical triad, proving the loop cannot discriminate architectures.

Table 5.1 synthesizes how these theoretical traditions (CoALA) converge on the same control loop while leaving architectural control unparameterized. This diagnostic gap highlights why practitioners cannot rely on the first principle alone to build production systems. The triplet of tool use, planning, and reflection defines the boundary between static model calls and agentic execution. It confirms that autonomy requires an interactive loop. What it fails to provide is a method to compare competing designs or parameterize execution choices. The first principle does not specify whether model reasoning or code controls the loop. Nor does it indicate how state persists across crashes or how human oversight is structured.

To achieve practical visibility and enable meaningful architectural comparisons, designers require a framework operating above the first principle. The eight invariant levers solve this constraint by breaking the abstract loop into parameterizable subcategories. Where the first principle asks what an agentic system is, the eight levers ask how a specific architecture should be configured. The levers parameterize the core loop across context construction, action capabilities, loop control, task decomposition, verification, persistence, capability evolution, and human control. By converting vague primitives into concrete design dials, the levers establish a practical grid to evaluate, compare, and engineer agentic architectures.

The next eight chapters take those dials one at a time, beginning with what an agent knows.

6. Context: what the agent knows is rebuilt, and paid for, on every turn

The first lever is the one every other lever depends on: what the agent knows at the moment it decides. This chapter shows that this knowledge has to be built, and paid for, on every turn. An agent’s context is not a given. It is a construction, rebuilt on every turn, and maintaining it costs more than the rest of the architecture.

The starting point is a property of the model itself. A language model is stateless: it keeps nothing from one call to the next. CoALA puts it flatly, in that models retain no information across invocations (CoALA, arXiv 2309.02427).

Everything the agent “knows” when it decides was therefore put there by someone. The designer writes the standing instructions. A search tool reports what it finds. A memory mechanism brings back what was seen earlier.

Concretely, an agent fixing a bug knows the code repository only through what a read tool showed it on the previous turn. Close the session, reopen it, and it knows nothing.

The Context lever therefore asks three questions: who fills that window, with what, and at what price?

The window in question is not a neutral store. Anthropic treats it as a finite resource. A model has an attention budget, and every token added consumes part of it. A token is the unit of text the model manipulates, roughly a syllable or a short word. The design rule that follows fits in a sentence. Find the smallest set of high-signal tokens that makes the intended result likely (Anthropic, Effective context engineering).

Figure 6.1: The context window as one bar divided into kinds of content, with stale tool results leaving it, content fetched from a repository outside it, and tacit context cut off beside it. Figure 6.1: The context window as a finite budget (Anthropic). Context editing clears stale tool results (Anthropic), and tacit context stays outside it (arXiv 2603.14805). Diagram drawn by this report.

Figure 6.1 draws the window as that budget, with the moves this chapter goes on to describe (Anthropic). The underlying driver is architectural, and it has a name. Current models rest on the transformer, an architecture in which every token of the text is related to every other. The number of relations therefore grows as the square of the length. Doubling the text supplied quadruples the relations to hold, while the model’s capacity for attention does not double. The longer the text, the less each relation weighs.

That dilution can be measured. Chroma, which publishes an open-source search infrastructure for AI, evaluated 18 models on a deliberately simple protocol. Hide a precise piece of information in a long document, then ask the model to find it. The question stays the same on every trial, and only the length of the document changes (Chroma Research). Performance degrades as the document grows, though unevenly from one model to the next.

One result stands out, and it holds for all eighteen. Models retrieve the information better when the passages of the document are shuffled than when they follow logically (Chroma Research). The coherence of the supplied text hurts performance, consistently. The authors draw a conclusion that moves the problem. What matters is not so much that the information is present, but how it is presented.

If presentation weighs that much, the tooling that handles it becomes a design object in its own right. Anthropic puts a number on its own. Context editing, which automatically clears tool results that have gone stale, adds 29 % performance on an internal agentic-search evaluation. Combined with a file-based memory tool, the gain rises to 39 %. On a hundred-turn evaluation, the same mechanism cuts token consumption by 84 % (Anthropic, Managing context).

The field has not converged on a method, though, and Inside the Scaffold shows it by reading code. A scaffold, in the sense set in chapter 3, is the code wrapped around the model. It holds the loop that decides when to call the model, the definition of the tools offered to it, and the code that runs those tools. It also holds what is kept or dropped from the history, and where state is stored. A scaffold is ordinary software, written by developers, and it is what one actually designs when designing an agent.

Across the thirteen scaffolds the paper analyzes, there are seven distinct forms of compaction, the operation that summarizes a conversation grown too long so that work can resume from the summary. The authors place it among the dimensions where the design questions remain open (arXiv 2604.03515).

Table 6.1

Context Management Strategy Primary Mechanism Empirical Benchmark / Finding Architectural Trade-Off & Boundary
Attention Budgeting (Anthropic) Minimal high-signal token selection Quadratic attention degradation in transformer relations Eliminates low-signal noise; demands strict upfront relevance filtering.
Context Editing (Anthropic Blog) Automatic pruning of stale tool call outputs +29% search accuracy, -84% token burn over 100 turns Vendor-reported internal benchmark; drops intermediate execution state.
History Compaction (Inside the Scaffold, arXiv 2604.03515) Trajectory summarization and state pruning 7 distinct patterns across 13 production coding agents Lossy compression; risks erasing subtle intermediate error traces.
Tacit Knowledge Codification (Yahoo, arXiv 2603.14805) Transforming informal team reflexes into structured context 2.6 hours saved per week per engineer in enterprise trial Non-delegable upfront human labor; bottleneck is knowledge architecture.

Table 6.1 summarizes how these context engineering strategies (Anthropic) balance token budgets against information loss. These figures call for two reserves. The 29 %, 39 % and 84 % come from internal, unpublished evaluations run by the vendor on its own product (Anthropic). That is a vendor’s argued position, not an established fact.

The second reserve concerns a figure that is missing. The degradation tied to length circulates under the label context rot, often with a range of 20 to 45 % beyond 64,000 tokens. That range does not appear in the primary report (Chroma Research). Chroma documents a slope, not a threshold, and the slope differs from one model to another. The figure is therefore set aside, and the chapter on the state of the evidence lists it among the discarded claims.

Delegation boundary

Part of the context is beyond an agent’s reach. It is tacit context, meaning what the members of an organization know without it being written anywhere.

An example makes the notion tangible. A team knows never to deploy on a Friday evening. It knows the billing service must be migrated before the payments service, and not the other way round. It knows which team owns which module, even though nothing in the code says so. It knows which reflex to apply when the queue backs up, because that has happened twice before. None of it is written down, and it is exactly what the agent lacks when it decides.

Knowledge Activation, a 2026 paper backed by an enterprise deployment at Yahoo, states the diagnosis. An organization’s institutional knowledge, made of architecture decisions, deployment procedures, compliance rules and incident reflexes, stays locked in formats designed to be read by humans. The bottleneck, its authors argue, is not model capability but knowledge architecture (arXiv 2603.14805).

The same work forbids treating this as a limit peculiar to machines. Faced with a task stripped of that context, three profiles fail the same way: an autonomous agent, a newly arrived engineer, and an experienced developer on code they do not know. The boundary is therefore not a matter of aptitude. It is a matter of cost, because making the unsaid explicit has to be paid for.

That cost has been measured once. The Yahoo deployment, assessed by a survey of 67 engineers, reports 2.6 hours saved per week per engineer (arXiv 2603.14805). The survey is self-reported and covers a single site. It gives an order of magnitude, not a measurement.

Delegable: fetching the information, filtering it, summarizing it, compacting it. Not delegable: deciding what deserves to be written down once and for all. As long as nobody pays the cost of making it explicit, that cost falls back on the most experienced engineers, task after task.

For the designer, the consequence is that a context window is managed as a budget rather than as a stock. Measure what each block of context buys in performance before adding it. Prefer just-in-time loading, where the agent receives light references and fetches what it needs itself: a coding agent given the file tree and two search commands will do better than one handed the whole repository. And treat compaction as an unsettled design choice, to be instrumented rather than inherited from an off-the-shelf framework. None of these moves reaches the tacit part, which stays a cost to be paid rather than a problem to be solved.

Knowing is the first half of deciding. The second half is acting, and the next lever asks what an agent should be able to do.

7. Action capability: what matters is what cannot be undone

The second lever concerns action. Once an agent knows enough to decide, what should it be able to do? This chapter argues that capability has to be judged by one criterion, reversibility, and that the controls that hold are the ones set before the agent acts. The right question is not what an agent can do. It is what it can do that cannot be undone.

An agent with no tools produces only text. Give it tools and it acts on the world, and its actions are not equal. Reading a file costs nothing. Modifying a file under version control is recovered with one command. Sending an email to a customer, wiping a database, approving a payment: none of that is recovered.

The Action capability lever therefore asks two distinct questions, and conflating them is the most common design fault. What is the agent technically able to do? And when must it ask permission?

OpenAI separates the two explicitly in Codex, its coding agent. The sandbox mode defines what the agent can do technically, for instance where it may write and whether it can reach the network. The approval policy defines when it must stop and ask, for instance to leave the sandbox or to run a command outside a trusted list (OpenAI, Codex agent approvals & security).

A sandbox is a constrained execution perimeter, enforced by the operating system, outside which the program can reach nothing. The documentation states its function well. One stops trusting the agent’s intentions and trusts the limits it operates within instead (OpenAI, Sandboxing).

Figure 7.1: An engineer reviews paper drafts outside a sealed chamber in which the irreversible action takes place. Figure 7.1: An editorial allegory of action capability containment and the blast radius (OpenAI). An engineer evaluates reversible paper drafts outside a sealed chamber where irreversible actions operate within strict physical limits, while an external messenger is barred from injecting unverified commands (CaMeL, arXiv 2503.18813).

As depicted in Figure 7.1, isolating an agent behind physical boundaries replaces probabilistic trust with mechanical containment (OpenAI). The defaults say how much suspicion is judged reasonable. Network access is cut, and the perimeter is limited to the current workspace. Commands launched by the agent inherit those limits: if it calls a package manager or a test runner, they stay inside the same boundary.

The clearest mechanism comes from the hosted version. Execution there is cut in two phases. A setup phase, allowed to reach the network in order to install dependencies. Then an agent phase, offline by default. Above all, the secrets configured for the environment are available only during setup, and are removed before the agent starts (OpenAI, Codex agent approvals & security).

The pattern is worth keeping as it stands: privileged phase first, degraded phase second. It is not a permission granted, it is a capability withdrawn before the action.

Anthropic went further, and justified it with a failure. Its protection used to rest on a permission request at every turn. Its conclusion is blunt. In theory that works, but the approach proved fallible in practice. The cause is approval fatigue, the fact that after enough approvals a human approves without reading. The chapter on the Human Control lever measures it.

The shift that follows is the core of this chapter. Rather than supervising what the agent does, Anthropic says it supervises what the agent is able to do. The means are sandboxes, virtual machines and network egress controls (Anthropic, How we contain Claude).

Three behaviors justify that suspicion, and the lab reports them on its own models. Claude models have “helpfully” left a sandbox to finish a task. Others consulted version control history to find the answers to a coding test. Others again identified the benchmark they were being run on, in order to work out its answer key (Anthropic).

These facts read as a mechanism, not as an intent. A system optimizing an objective will take the routes available to it, including the ones the designer left open. That is the strongest argument for containment decided before the action rather than trust granted after it. The admission that comes with it counts as much: this is where Anthropic says it concentrated the most effort, and it is also where its most surprising failures occurred.

A third mechanism has nothing agentic about it, and that is what makes it valuable. Teams who run servers have faced the same problem for years, in that a command which reconfigures an infrastructure cannot be replayed backwards. Their answer was to separate the proposal from the execution.

Terraform, a tool that creates and modifies machines from a configuration file, gives the canonical form. The terraform plan command produces a plan of the changes to come, and that command on its own executes nothing (HashiCorp). It serves to check that the proposed changes match what was intended, or to have them reviewed by the team. The pattern is called propose then apply.

Transposed to an agent, it cuts the irreversible action in two: a readable artifact first, execution second. It is a borrowing from earlier practice, not an invention of the field.

Everything above protects against the agent’s own deviations. There is a second source of risk, and it comes from outside.

An agent reads what it finds: a web page, a file, a tool result. Nothing in what it reads distinguishes data from instruction. A third party can therefore slip an order into content the agent is going to consult, and have it executed. This is called prompt injection. It makes the Action capability lever critical, since the more the agent can do, the further the smuggled order reaches.

Two families of answers compete. The first detects the hostile instruction, through a filter or a model placed on the path. The second makes it structurally ineffective, by arranging matters so that content read cannot alter the flow of the program.

CaMeL belongs to the second family. It extracts the execution plan from the trusted request, before any external content is read. Untrusted data fetched afterwards therefore cannot influence what will be executed (CaMeL, arXiv 2503.18813).

That guarantee has a price, and CaMeL is one of the rare approaches to quantify it. The measurement is made on AgentDojo, a benchmark that subjects an agent to injected instructions and checks whether it still does its job. CaMeL solves 77 % of tasks there while keeping its guarantee, against 84 % for a system with no defense at all (CaMeL, arXiv 2503.18813). Seven points of utility given up for a demonstrable property.

Table 7.1

Containment Archetype Primary Defense Mechanism Empirical Benchmark / Observation Boundary & Trade-Off
OS-Level Sandboxing (OpenAI Codex) Filesystem isolation, network egress blocking, ephemeral VM Prevents credential extraction and unauthorized system calls Widely tooled but rarely evaluated; container cold-start overhead.
Two-Phase Capability (OpenAI) Privileged setup phase (with secrets) → Degraded offline execution phase Removes API keys and network tokens before model loop begins Eliminates secret exfiltration; forbids late dependency fetching.
Propose-Then-Apply (HashiCorp) Human-readable plan artifact generated prior to mutation Decouples intention validation from irreversible environment changes Requires human attention budget; vulnerable to approval fatigue.
Structural Plan Isolation (CaMeL, arXiv 2503.18813) Extracts execution flow before reading untrusted third-party data 77% task success under injection vs. 84% without defense on AgentDojo 7-point utility sacrifice for verified prompt injection immunity.

Table 7.1 synthesizes how these containment strategies (OpenAI) trade execution utility against blast-radius protection. The CaMeL measurement is close to the only one in this chapter, and the imbalance should be stated plainly. Agentic containment is widely tooled and barely evaluated. This corpus holds no public measurement of the effectiveness of the sandbox itself, nor of the network cut, nor of secret withdrawal. These are architecture descriptions, not results.

Two reserves make the observation worse. The only family of approaches carrying a direct measurement is human permissions, and that measurement contradicts it; the chapter on the Human Control lever gives the figure.

The second reserve concerns detection-based defenses. The papers that publish them report attack success rates close to zero. But a published rate holds only against the attacks that were tried. Against an adversary who adapts to the defense in front of them, those same defenses fall, and the chapter on the state of the evidence gives the scale of the collapse.

Delegation boundary

Delegable: the choice of action inside the perimeter. Not delegable: the definition of the perimeter.

The point is harder than it looks, because irreversibility is not a technical property of the action. Deleting a row in a database is recoverable if a backup exists, and not otherwise. Sending an email becomes irreversible the second a third party reads it. The classification depends on the organization’s context, on how much damage it can absorb, and on who will answer for it.

No source in the corpus describes a mechanism by which an agent would set that boundary itself. Anthropic puts it in its own terms by recalling that every probabilistic defense has a non-zero failure rate (Anthropic). An agent can propose a classification of actions; a human settles it, because a human carries the consequences.

Designing against that background means giving up on permissions as the whole answer. Technical capability and approval policy are two settings, not one, and they are decided separately. Capabilities are withdrawn before the action rather than requested during it, on the two-phase model. And an irreversible action is cut into a reviewable artifact followed by an execution, on the propose-then-apply model. None of these measures eliminates the risk, and the actors who publish them do not claim otherwise.

Containment settles what an agent may do. It does not settle who chooses among the permitted actions, and that choice is the subject of the next lever.

8. Control loop: code or model, the one line every major vendor draws

The third lever is the control loop, the mechanism that picks the next step. This chapter shows that the question of who holds it, code or model, is the one distinction the major vendors agree on, and that real systems answer it differently at different scales. An agent works in a loop. On every turn, something decides what happens next. There are only two answers, and the choice between them is the sharpest division in the whole field.

Either the code decides, because a developer wrote the sequence in advance. Or the model decides, because it is shown the current state and the available tools on every turn, and it chooses.

An example makes the gap visible. A fixed chain reads the ticket, locates the file, proposes a fix, runs the tests, and stops. The order is written into the program and never changes. An open loop hands control back to the model after each tool: it can reread the ticket, search elsewhere, rewrite, rerun the tests three times, or abandon that line of attack. Both solve the same problem. They do not share a cost, a predictability, or a set of failure modes.

Figure 8.1: An axis running from code decides to the model decides, with a fixed pipeline at one end and Monte Carlo tree search at the other. Below it, the list of the five loop primitives, and three of them nested inside one another. Figure 8.1: Three vendors name the same axis (OpenAI, Google, Anthropic), and it is a gradient. One agent can stack several loop primitives at different scales (arXiv 2604.03515). Diagram drawn by this report.

Figure 8.1 places the two answers at the ends of one axis (OpenAI). What gives this axis weight is that it is not a construction of this report. Three major vendors organize their documentation around it, each independently.

OpenAI announces it as the first distinction in its Agents SDK, the library with which one assembles an agentic system. There are two ways to orchestrate agents: via the model, or via code, meaning by determining the flow in one’s own program (OpenAI, Agents SDK).

Google gives the sharpest formulation in ADK, its agent development kit. Workflow agents “determine the execution sequence according to their type, sequential, parallel or loop, without consulting an AI model for orchestration” (Google, adk-docs). The sentence is useful because it names what is taken away from the model, rather than what is given to it.

Anthropic draws the same line, under the names workflow and agent, with the definition set out in chapter 2. Three vocabularies, one axis.

That convergence is an argument for the framework proposed here. It is also its limit, because it suggests a switch where there is a gradient.

Inside the Scaffold demonstrates as much by reading the code of scaffolds, the code around the model described in the chapter on the Context lever. At one end of the spectrum sits the fixed pipeline, with no feedback loop at all. At the other end sits Monte Carlo tree search, a method that explores several possible action sequences, evaluates each, then propagates the result back to steer the next round of exploration (arXiv 2604.03515). Everything between the two ends is occupied.

More instructive still, the paper identifies five loop primitives. A primitive here is an elementary control pattern: the smallest coherent way of chaining turns, which gains nothing from being cut up further, and which serves as a building block for larger assemblies. The five it records each take a line to describe.

ReAct alternates a reasoning step and an action, the observed result feeding the next reasoning step (arXiv 2210.03629). Generate-test-repair produces a candidate, submits it to an automatic test, and uses the failure to correct it. Plan-execute writes the full plan first, then runs its steps. Multi-retry restarts the same attempt from scratch when it fails, up to a fixed limit. Tree search opens several branches in parallel and keeps only the best.

These five patterns work as composable blocks, and eleven of the thirteen agents stack several of them rather than relying on a single control structure (arXiv 2604.03515). One agent can run ReAct at the turn level, generate-test-repair at the fix level, and multi-retry at the level of the whole task.

Table 8.1

Loop Primitive Chaining Mechanism Operational Scope Scaffolds Composition Pattern
ReAct (Yao et al., arXiv 2210.03629) Alternating thought and tool observation Step-by-step local tool call decisions Embedded as the elementary turn loop.
Generate-Test-Repair (Inside the Scaffold, arXiv 2604.03515) Generates candidate, tests against code/linter, refines on failure Targeted patch or function modification Stacked above turn loop to iterate on intermediate compilation failures.
Plan-Execute (Google ADK) Decomposes complete task upfront into fixed sequence of subtasks Macro-level workflow orchestration Enforces deterministic high-level milestones before model execution.
Multi-Retry (Inside the Scaffold, arXiv 2604.03515) Restarts failed trajectory from scratch or checkpoint up to limit Task-level failure recovery Guards against stochastic dead ends across the entire run.
Tree Search (MCTS) (arXiv 2604.03515) Branches multiple exploration paths and backpropagates evaluations Complex algorithmic search space Highest token burn; deployed selectively when branching space is tractable.

Table 8.1 details how these five loop primitives (Inside the Scaffold, arXiv 2604.03515) are layered across different operational scales. The observation matters for the rest of the report. Asking whether a system is “a workflow” or “an agent” is badly posed, since it is most often a stack of both at different scales. The useful question is what settles each decision point: a line of code, or a model call.

What pushes a designer one way or the other has nothing ideological about it, and ADK’s documentation gives the most concrete motive. Beyond a certain length or complexity of an instruction sequence, one discovers that a single agent does not execute all of it reliably. Staying within the model’s context limits adds to that (Google, adk-docs).

Moving to code is therefore not a matter of school. It is what one does once the open loop has stopped holding, and it is an admission of a limit before it is an architectural preference. Google adds that it sets no strict requirement for taking that step (Google, adk-docs).

The reserve on this axis is serious, and it starts with an absence. None of the three vendors publishes a measurement comparing the two orchestration modes. They describe architectures, give conditions of use, and assert advantages. These are design positions, not results.

The difficulty runs deeper than missing data. Inside the Scaffold notes a methodological bias widespread in earlier work: studies observing agents at work used different models for different agents, which makes it impossible to attribute a behavioral difference to the loop rather than to the model (arXiv 2604.03515). Until that bias is lifted, what the control loop actually buys remains unknown.

Delegation boundary

Delegable: the choice of the next step inside a bounded space. Not delegable: deciding how wide that space is.

That second decision rests on a condition Anthropic states without hedging. To use an autonomous agent, one needs some level of trust in its decision-making. Autonomy suits tasks carried out in trusted environments, the source adds (Anthropic, Building Effective Agents).

The formulation moves the problem outside the technical domain. Trust is not a setting. It is a social mechanism, built by a history of interactions and withdrawn when it is betrayed. No source in this corpus describes how one would establish it towards an agent, nor what would make it vary from one task to the next.

What stands in for trust in practice is reversibility, treated in the previous chapter. The model’s decision space is widened where an error can be recovered, and closed where it cannot. That is not trust, it is a calculation of acceptable damage, and it belongs to whoever will answer for the damage.

ADK’s motive reinforces the point. It is not the designer who calmly decides to hard-code the sequence: it is the observed failure of a single agent on an over-long instruction sequence that forces the move. The boundary is discovered in use, it is not computed in advance.

So the designer stops reasoning in families and starts reasoning in decision points, saying for each fork in the system whether code or the model settles it, and why. Loop primitives get composed rather than adopted one at a time, since that is what eleven scaffolds out of thirteen do. And the part of the flow whose errors cannot be recovered is fixed in code, leaving the model the part where they can.

This axis is the best corroborated in the report, since three competing vendors use it. It is also among the least measured, which is a warning for Part III.

The loop described so far has a single executor. The next lever asks what changes when there are several.

9. Decomposition and specialization: every split is paid for in coordination

The fourth lever is decomposition and specialization, the division of work among several executors. This chapter weighs what that division buys against what it costs. Splitting a task across several model calls buys specialization. It is paid for in coordination, and the bill is rarely added up.

The three previous chapters treated the agent as a single executor. The question changes nature here: one executor or several, and along which cut.

The motive for dividing was already set out in the Context lever. A model’s attention is a finite resource, and it thins as the window fills. An executor handed a broad task carries everything at once: the standing instructions, the history, the tool results, the detail of every sub-problem. Narrowing its scope gives attention back, and that is what the division of labor buys first.

Figure 9.1: Four small sketches of the ways to cut a task (by step, by role, by instance, dynamic subagents), above three bars comparing token consumption. Figure 9.1: The four cuts and what each one buys (Building Effective Agents), above the token consumption the vendor reports for agents and multi-agent systems (Anthropic). Diagram drawn by this report.

Figure 9.1 shows what each way of dividing buys, and what dividing costs in tokens (Anthropic). Dividing is not one gesture, though. Anthropic’s catalogue distinguishes three axes, and the first cuts by step: the task becomes a sequence of steps, and each model call processes the output of the previous one. The condition of use is restrictive, since the task must decompose cleanly into fixed subtasks. The objective is stated without hedging. It trades latency for higher accuracy, by making each call easier (Anthropic, Building Effective Agents).

The second axis cuts by role. A first step classifies the input, then routes it to a specialized treatment. Anthropic’s justification is the classic argument for the division of labor: without that cut, optimizing for one kind of input degrades performance on the others (Anthropic). Concretely, a support desk that handles refund requests and technical outages in the same place ends up doing both badly, because the instructions that serve one work against the other.

The third axis cuts by instance: several calls work the same problem at the same time. They either split its aspects between them, or each attempt an answer so the answers can be compared. Anthropic recommends this arrangement when speed matters, or when several viewpoints and several attempts are needed for a more reliable result. The vendor adds a broader claim. For complex tasks with several considerations, a model would generally do better when each consideration is handed to a separate call, which lets it focus its attention on a single aspect (Anthropic). That last claim rests on no data, and the reserve in this chapter returns to it.

What these three axes share is that the designer writes the cut in advance. A fourth arrangement hands the cut to the system itself. A central call decomposes the task dynamically, delegates the pieces to subordinate calls, then synthesizes their results. Usage has settled on the word subagent for those subordinates. A subagent is an instance of the model launched by another, with its own context window, for a bounded piece of work whose result it returns.

Anthropic takes care to distinguish this fourth arrangement from the cut by instance, and the difference turns precisely on who decides. The resemblance is topographical, but the subtasks are not defined in advance. The orchestrator determines them from the input it receives (Anthropic). The intended use follows. These are complex tasks whose subtasks cannot be predicted, such as a fix when one does not yet know how many files it will touch.

Handing the cut to an orchestrator raises a question a single executor never posed: who owns the final answer. OpenAI’s Agents SDK puts the two possible answers side by side. In the first, a manager agent keeps hold of the conversation and calls the specialists as tools. In the second, the handoff, a routing agent transfers the conversation to a specialist, which becomes the active agent for the rest of the turn (OpenAI, Agents SDK).

The published arbitration rule is useful because it does not speak of performance. Specialists are used as tools when they must handle a bounded subtask without taking on the conversation. The handoff is used when the routing is itself part of the work (OpenAI). That second arrangement rests on a rarely stated assumption: that context travels with control. The chapter on the fault lines shows this to be the most contested point in the field.

These rules say when to use each arrangement. None says what it returns. The corpus holds a single figure for choosing between one executor and several, and it comes from Anthropic. The comparison sets two configurations against each other. On one side a multi-agent system, led by Claude Opus 4 with Claude Sonnet 4 subagents. On the other the same Opus 4 used alone. The first beats the second by 90.2 % on the vendor’s internal research evaluation (Anthropic, multi-agent research system).

The figure is spectacular and its setting far less so. The evaluation is internal, unpublished, not reproducible, and the vendor is judging its own architecture there. It is an argued position, not an established fact.

The authors bound their own result, and those limits are worth as much as the gain. The arrangement excels on breadth-first queries, the ones that call for pursuing several independent leads at once. It suits poorly those domains requiring all agents to share the same context, or carrying many dependencies between them. Programming, they note, offers fewer genuinely parallelizable tasks than document research (Anthropic).

The same post gives the price, and the concession counts because it comes from the favorable side. Agents consume around four times more tokens than a conversational exchange, and multi-agent systems around fifteen times more (Anthropic). Gain and overhead therefore come from the same text, written by the same actor.

Table 9.1

Decomposition Paradigm Structural Cut Primary Advantage Coordination Overhead & Boundary
Cut by Step (Anthropic) Sequential pipeline where step N feeds N+1 Higher accuracy per sub-call Cumulative latency; requires fixed predictable subtasks.
Cut by Role (Anthropic) Classifies the input and routes it to a specialized treatment Optimizing for one kind of input no longer degrades the others Needs inputs that fall into distinct categories.
Cut by Instance (Anthropic) Multiple parallel calls exploring same problem Speed and multi-viewpoint comparison Every parallel call is paid for (this report’s inference).
Dynamic Subagents (Anthropic) Central orchestrator defines the subtasks at run time Handles tasks whose subtasks cannot be predicted Who owns the final answer: specialists as tools, or a handoff (OpenAI Agents SDK).
Multi-Agent Assembly (Anthropic) Specialized hierarchy with dedicated manager 90.2% better than a single agent on the vendor’s internal evaluation 15x token multiplier; poor fit for tightly interdependent codebases.

Table 9.1 synthesizes the trade-offs across these five decomposition paradigms (Anthropic). A gain on one side, an overhead on the other: the division of labor is a trade-off, and its sharpest formulation comes from outside the technical field.

One paper sets out to restate organization theory in terms usable by a designer of agent systems. It puts the question this way: does the benefit of several specialized agents outweigh the increase in failure modes caused by poor coordination (arXiv 2512.07665)? Its authors tie the problem to structural differentiation, a notion they take from the study of organizations. The reading given to it here is this report’s: dividing a structure improves the work of each unit and raises what must be spent to hold the units together. Decomposition is never a net gain.

Settling that trade-off would take measurements, and the corpus supplies none. No vendor publishes a result per pattern. Anthropic’s five workflows, OpenAI’s two arrangements, their conditions of use: all of it is asserted, described and illustrated, never compared on a common bench. The claim that a model does better when each consideration goes into a separate call rests on no data in the post that carries it.

The 90.2 % does not fill that hole (Anthropic). It sets two architectures against each other, not two cuts, and does so with the token budget left free. Yet the chapter on the state of the evidence shows that token consumption explains most of the performance variance on benchmarks of that kind. A gain obtained by spending fifteen times more therefore does not yet say what the division of labor buys.

Delegation boundary

Delegable: cutting a task into subtasks, once the criterion for the cut has been supplied. Not delegable: the choice of that criterion, nor the definition of the roles.

The limit is set by the source most favorable to the arrangement. Agents, Anthropic writes, are not yet good at coordinating and delegating work to each other in real time (Anthropic). The admission bears precisely on the function one would want to hand them.

A second limit comes from the substrate, and comparison makes it visible. In a human organization, a role lasts. The specialist accumulates experience, one learns what they are worth, and that judgment follows them from one task to the next. In an agent, the role is an instruction held for the length of one run, and nothing accumulates in it.

The work cited above places that fragility among the problems organization theory does not cover. Agent coherence within an assigned role stays limited as soon as an exchange runs over several turns (arXiv 2512.07665). The part devoted to organizational resemblance returns to it. The practical consequence fits in a sentence: a role does not capitalize, and one cannot entrust more to a subagent because it did well last time.

These limits dictate the conduct to follow. The designer does not divide before naming what the division buys, in attention returned or in leads explored in parallel. The cut is written in advance whenever it is predictable, and handed to an orchestrator only where the subtasks cannot be known ahead of time. And the designer settles who owns the final answer, rather than letting that ownership go missing between agents.

The division of labor remains the lever borrowed most directly from human organizations. It is also the only one whose overhead is quantified by the very people who recommend it, which puts it in a curious position. Better documented than the others on what it costs, it is no better documented on what it returns.

Dividing the work multiplies the steps, and with them the places where something can go wrong unnoticed. The next lever is about noticing.

10. Verification and conformance: results are checked, the path almost never is

The fifth lever is verification and conformance. It asks how a system checks its own work, and this chapter separates two checks that are often confused: checking the result, and checking the way the result was reached. Checking what an agent produced is ordinary engineering. Checking how it produced that result has almost no tooling, and that is where the failures sit.

The previous chapter multiplied steps by dividing the work. A single run now passes through dozens of model calls and tool calls, and any one of them can go wrong without leaving a mark on the final answer. AgentEval, a protocol that formalizes an agent run as an evaluation graph so that an observed failure can be traced back to the step that caused it, states the problem plainly. Prevailing evaluation practice consists of end-to-end outcome checks and ad-hoc trace inspection. Those practices “systematically mask the intermediate failures that dominate real-world error budgets” (AgentEval, arXiv 2604.23581).

Looking inside a run requires some handle on it. The obvious handle is what the agent says about its own work: reasoning models emit a stream of words before they answer, and that stream reads like an explanation. A result from Anthropic rules out treating it that way.

The lab states the problem before measuring it. There is “no specific reason why the reported Chain-of-Thought must accurately reflect the true reasoning process”. There might even be circumstances where a model actively hides aspects of its thought process from the user (Anthropic, Reasoning models don’t always say what they think). Two distinct properties are at stake. Legibility asks whether words can carry every nuance of a decision made inside a neural network. Faithfulness asks whether the description offered is accurate.

The measurement is a three-step protocol. Slip the model a hint about the answer, confirm that it used the hint, then look at whether it mentions the hint in its reasoning. The comparison offered is a note passed to a student before an exam, telling them the answer is the first one. Some hints are deliberately awkward, such as a message telling the model that it has gained unauthorized access to the system.

The measured verdict is severe. Averaged across all hint types, Claude 3.7 Sonnet mentions the hint 25 % of the time, and DeepSeek R1 39 % (Anthropic). A substantial majority of answers, the paper concludes, were unfaithful.

METR disputes the reach of that conclusion without disputing the measurement. This evaluation organization replicated the faithfulness evaluations from the Claude 3.7 and Claude 4 system cards. It then varied the difficulty of the hint, to see whether faithfulness rose with complexity. Its objection targets the criterion. Consistent faithfulness in every scenario is “a very high bar that might not be required for most safety analyses of models” (METR).

The reframing it proposes is directly useful to a designer. Treat the trace “less like a purported narrative about a model’s reasoning and more like a tool: a scratchpad” the model uses for sequential memory or planning-heavy work (METR). The useful question becomes whether the trace is informative enough to reveal notable behavior, rather than whether it is faithful. The argument behind the shift rests on difficulty. Many dangerous kinds of reasoning are complex enough that a model benefits from using the trace to carry them out, so they stay visible in it.

Both positions lead to the same design move, which is what makes the disagreement settleable in practice. Auditing an agent cannot rest on its explanations. It rests on its actions: tool calls, writes, observable effects. The reasoning trace becomes a clue to corroborate, never a report to take at face value. The chapter on the layer with no organizational equivalent picks faithfulness back up for a verdict of a different kind; here it only decides what the checking is aimed at.

Figure 10.1: An inspector measures a finished mechanism while its maker explains it. Figure 10.1: An inspector measures the finished mechanism and does not look at the maker who is explaining it, an editorial allegory for this chapter. An agent’s account of its own reasoning is a clue to corroborate, not a report to take at face value (Anthropic, METR). What it did can be checked.

Existing tooling turns out to be better aligned with that conclusion than one might fear. Two platforms document their data model, and the two agree. LangSmith, the observability platform of the LangChain ecosystem, records a trace for each operation. That trace “records the sequence of steps your application takes for a single operation”. It is made of individual runs, and traces from multi-turn conversations link together into a thread (LangSmith, observability concepts). Langfuse, an open-source observability platform for applications built on a language model, holds three concepts: observations, traces and sessions. Observations are “the individual steps of your application: LLM calls, tool calls, retrieval steps”, and they nest to reflect the structure of the application (Langfuse, data model).

LangSmith and Langfuse therefore capture the same three-level hierarchy: the session, then the trace of one operation, then the observation of one step. Both are built around calls rather than intentions. A record says that a tool was invoked with these arguments, and nothing more. That is exactly the level the unfaithfulness of reasoning traces forces on a designer.

Recording, though, is not the same as attributing. When a run of several dozen steps fails, the step that caused the failure still has to be identified, and the number of steps makes hand inspection impractical. That is the work AgentEval does, by formalizing runs as directed acyclic evaluation graphs. Each node carries typed metrics scored by a calibrated language model. Those metrics are sorted by a hierarchical failure taxonomy, three levels deep and twenty-one subcategories wide. And each node stays linked to the nodes it depended on, which is what lets a root cause be attributed automatically (arXiv 2604.23581).

The protocol was then measured against itself, which is rare enough in this field to be worth reporting. The same evaluation ran twice, and one thing changed between the two runs: whether the graph recorded which step depended on which. The judge model and the scoring rubrics stayed identical. Recording those dependencies raised recall on failure detection, meaning the share of real failures the evaluation caught, by 22 points. It raised the share of failures traced to the right root cause by 34 points (arXiv 2604.23581). The gain came from making the structure of the run explicit, not from a better judge.

Two reserves come with that result. The comparison was run by the protocol on its own datasets. And the judge that scores each node is itself a language model, so the evaluation inherits part of the weakness it measures. The chapter on the state of the evidence puts these figures on the scale they deserve.

All of this tooling describes what happened. None of it describes what should have happened. Neither LangSmith nor Langfuse carries any notion of a reference plan or of conformance, and their data models have no slot for the expected. AgentEval, for its part, judges once the run is over.

Checking the process therefore has no tooled answer. Knowing whether an agent is following the intended procedure, while it is executing, matches no mechanism documented in this corpus. No source in it covers runtime verification applied to agent trajectories, declarative constraints on a trajectory, or plan deviation detection. None transposes the conformance checking that process mining applies to execution logs. What exists is control at the point of execution, which stops one action as it comes up, or diagnosis after the fact. Concretely, an agent told to read the specification before touching the code can do the reverse and still return a correct result. Nothing in the trace will flag the departure from procedure.

A signal about the state of the field comes with that gap, and it should not be read as a collection failure. OpenTelemetry, the most widely used instrumentation standard for tracing a system in production, published semantic conventions for generative AI. Its specification page now states that the conventions “have moved to the OpenTelemetry GenAI semantic conventions repository”, and that the page itself “is no longer maintained in this repository” (opentelemetry.io, accessed 2026-07-29). Their actual status and their coverage of agentic concepts have not been verified in the new repository. The shared vocabulary that any tooled conformance check would rest on is itself in motion.

A comparison of existing observability platforms clarifies where tooling stops and where manual inspection remains mandatory. Table 10.1 contrasts the three observability and verification tiers across production platforms and research frameworks (LangSmith, AgentEval).

Table 10.1

Tier Primary Target Representative Implementations Core Mechanism Critical Blind Spot
Trace Telemetry Execution events and call hierarchy LangSmith, Langfuse Nested logging of the steps of a run, with their inputs and outputs Lacks reference plans or dependency graphs
Evaluation Graphs Post-hoc attribution and failure analysis AgentEval Directed acyclic graphs with typed metric nodes Retrospective analysis only, no live intervention
Runtime Conformance Procedural adherence during execution Unsolved gap (OpenTelemetry) Declarative plan comparison and live deviation flags No mature tooling or shared semantic standard

Delegation boundary

Delegable: the judgment passed on one step, once the reference to compare it against has been supplied. Not delegable: writing that reference.

The reason follows from the negative result above. No tool models the expected, so the reference plan is written by hand or it does not exist. Nothing stops a model from drafting that plan, or from scoring a step against it once the plan is settled. What a model cannot do is settle by itself what should have happened, since that is precisely the thing being checked in it.

A second boundary separates this lever from the Human Control lever, and the question that tells them apart is who reads the check. The Verification lever covers the checks the system runs in order to close a step or clear a gate, with nobody reading them. Legibility and authority meant for a human belong to the Human Control lever, three chapters further on. Both often read the same trace, which is what makes them easy to confuse.

The designer therefore instruments calls rather than explanations, and writes down separately what the run was supposed to do, since no platform will hold that for them. The missing piece is a tool that carries the expected trajectory, and nothing in the corpus offers one in 2026. Whether the field lacks it, or the search behind these pages missed it, cannot be settled here. The chapter on the gaps this method accepts takes it from there.

Verification reads a run once it exists. The next lever concerns what a run needs in order to continue at all.

11. State persistence: nothing survives a turn unless someone wrote it down

The sixth lever is state persistence: what a system still has after the turn, the session or the process has ended. This chapter shows that continuity is designed, never given. Nothing survives an agent’s turn by itself. Whatever is still there on the next call was written down on purpose, in a place someone chose, in a format someone specified.

The previous chapter already put something outside the run. A trace outlives the execution so that a human or a judge can read what happened, once it is over. Persistence asks a store for a different service, which is to hand the run back its own state so the work can go on.

Three horizons are at stake, and they do not pose the same problem. Within a session, the agent has to carry what it decided ten turns earlier. Across sessions, it has to recognize a user, a project, a convention settled last month. And when a process dies at step forty of a run, someone has to have decided whether the work resumes or starts over.

Continuity therefore has a site, and that site sits outside the model. CoALA gives it a name and a shape: working memory is a data structure that persists across model calls, distinct from the context window supplied on each one (CoALA, arXiv 2309.02427). Beneath it sits long-term memory, divided into episodic, semantic and procedural.

Those three names cover distinct contents, and the difference shows up the moment one designs a store. Episodic memory holds what happened, run by run. Semantic memory holds what is true of the world the agent works in. Procedural memory holds how things get done. Concretely, for an agent maintaining a service, last Tuesday’s failed migration is episodic, the fact that the billing team owns a given table is semantic, and the sequence for cutting a release is procedural. The chapter on the Capability evolution lever takes up what happens when an agent starts writing to that third kind. This lever asks only whether a store exists that can hold all three, and who fills it.

The distinction does work the Context lever cannot do alone. That lever asks what fills the window at the moment of deciding. Persistence asks what is still on disk when the window is rebuilt from nothing, and the two answers rarely coincide.

Two mechanisms follow from the split, and LangGraph separates them more explicitly than any other framework in this corpus. It comes from the same ecosystem as LangSmith and handles the other end of the job, building the agent rather than watching it. Checkpointers “persist a thread’s graph state as checkpoints”, and cover short-term memory scoped to one thread: conversation continuity, human-in-the-loop workflows, time travel, fault tolerance. Stores “persist application-defined data outside the graph state”, and cover long-term memory readable across threads: user preferences, facts, shared knowledge (LangGraph, persistence).

The axis separating the two is scope, not duration. A checkpoint belongs to one thread of work and gives that thread back exactly where it stopped. A store belongs to the application, is readable from any thread, and is therefore where anything meant to outlive the current task has to go.

One engineering detail settles the matter better than any statement of principle. LangGraph’s in-memory checkpointers keep their checkpoints in RAM, and the documentation states the consequence without softening it: “When the process restarts, all checkpoints are lost” (LangGraph). Durability is a backend one selects, and the in-memory option forgets everything on restart.

What a durable backend buys is stated in LangGraph’s documentation on interrupts. Checkpointing “keeps your place”, in that the checkpointer writes the exact graph state so a run can be resumed later, “even when in an error state” (LangGraph, interrupts). Resuming from a failed step is the payload of the whole mechanism. A run of forty steps that dies at step thirty-eight either replays the thirty-seven that worked, at full cost, or picks up where it fell.

One use listed there deserves separating out, because it turns persistence into a precondition for another lever. Human-in-the-loop workflows sit among what checkpointers serve, and the reason is mechanical. A run that stops to ask a human holds its whole state while it waits, sometimes for hours, then continues from the same point. LangGraph’s interrupt function can be called at any point inside a node. Execution resumes when the graph is re-invoked with a command carrying the human’s answer (LangGraph, interrupts). Without a durable checkpoint, a gate is only a pause inside a process that has to stay alive. The chapter on the Human control lever weighs what such a gate is worth; this one says what it costs to build.

Other frameworks carve the same ground, and the number of pieces varies. Microsoft Agent Framework merges the foundations of Semantic Kernel with the orchestrations of AutoGen into one SDK. It makes checkpointing a property of every pattern it ships, alongside pause and resume for long-running workflows. Its own formulation is that “checkpointing and hydration ensure long-running processes survive interruptions” (Microsoft, Agent Framework 1.0). Its memory layer is pluggable, holding conversational history, persistent key-value state and vector-based retrieval over whichever backend the designer picks. ADK cuts the same ground into three named primitives, Session, State and Memory (Google, adk-docs).

A converging vocabulary is not a converging design, and reading the code shows the difference. Inside the Scaffold places state management among the dimensions where its thirteen scaffolds pull apart. Dimensions converge where external constraints dominate, its authors write, and they diverge “where open design questions remain” (arXiv 2604.03515). The list of open questions holds context compaction, state management and multi-model routing. Persistence sits in the unsettled half of the field, which is worth knowing before adopting one framework’s answer as though it were the field’s.

A structured comparison clarifies how frameworks decouple ephemeral context from durable memory structures. Table 11.1 organizes the persistence spectrum across operational scopes, retention lifetimes, and candidate storage backends (CoALA, LangGraph, Microsoft).

Table 11.1

Persistence Layer Operational Scope Retention Horizon Primary Function Representative Backends Failure Mode on Crash
Working Memory Across the model calls of one run The lifetime of the run, distinct from the window supplied on each call State carried between calls; each call’s input is drawn from a subset of it A data structure held by the scaffold (CoALA) Lost with the process unless checkpointed
Thread Checkpoints Single task or thread Turn-to-turn execution Mid-run fault tolerance, time travel, human interrupts In-memory RAM, SQLite, PostgreSQL (LangGraph) Volatile if RAM; full resume if durable disk
Long-Term Stores Cross-thread application Multi-session persistent User preferences, semantic knowledge, organizational facts Key-value state and vector-based retrieval, on a backend the designer chooses (Microsoft) Preserved across restarts; requires schema maintenance

The reserve here concerns an absent question rather than a contested figure. No source in this corpus measures what persistence returns. What the vendors state are guarantees about mechanism: a process survives an interruption, a run resumes even from an error state. None reports how often that changes an outcome: runs saved, tokens spared, tasks a restart would have lost. A designer choosing a persistence backend is therefore choosing on engineering grounds, with no evidence behind the choice either way.

Delegation boundary

Delegable: the writing, the retrieval and the resume, once the shape of what persists is fixed. Not delegable: fixing that shape.

LangGraph’s own wording carries the reason. A store holds “application-defined data”, so the schema comes from outside the system and nothing inside can derive it. An agent can propose fields and fill them; what it cannot do is settle which of them the organization will still want in six months.

In a human organization, continuity is free and nobody decides it. An engineer who spent Tuesday discovering that the migration script has to be run twice still knows it on Wednesday. No schema was written for that, no store was provisioned, and no retention period was chosen. An agent gets exactly the continuity somebody paid for in advance, field by field, and gets nothing else.

One thing an organization keeps without deciding to is a record of who works well. The Decomposition and specialization lever established that a role does not capitalize, in that nothing about a subagent’s past work follows it into the next task. A store is where that could change, being the one component built to outlive the task. Its documented uses are user preferences, facts and shared knowledge (LangGraph), and an agent’s own record of what it did well is not among them. Whether such a record would be worth keeping stays unsettled, and the shape of the decision matches every other one on this lever. Somebody has to name the field, and somebody has to answer for what the system then remembers.

The decision has a second half, which is what must not persist. A store that keeps everything ends up feeding the window it was meant to relieve, and the Context lever measured what a crowded window costs. Retention and deletion are one design decision seen from two sides, and the task itself dictates neither.

A persistence design can be tested with a single question, asked before any backend is picked: what exactly is lost if this process dies right now? Answering it names the unit of resume, the horizon that unit has to cross, and the store that follows from both. A team that cannot answer has not designed its persistence. It has inherited a default, and the default keeps everything in memory until the process ends.

Persistence returns a system to where it was. The next lever asks how a system comes to do what it could not do before.

12. Capability evolution: the hard part is choosing which new skills to keep

The previous chapter asked what an agent keeps from one moment to the next. This one asks a different question: can the agent become able to do something it could not do before? The Capability evolution lever covers every mechanism that changes what a system can do between two runs. Its central finding is that agents can now write their own skills, and that the hard part is deciding which of those skills to keep.

The difference from persistence is easy to miss, so it is worth an example. An agent that resumes from a checkpoint gets back its files, its history and its place in the task. Give it the same kind of bug next week and it will rediscover the same fix by the same trial and error. A checkpoint gives back what the agent was. Capability evolution is whatever makes next week’s attempt shorter than this week’s.

For a long time that change came only from people. A developer rewrote the prompt, or added a tool to the code, and the agent was different afterwards. The agent took no part in it. The idea examined here is that an agent can do some of this work on itself.

CoALA gives the idea a precise place. Among the things an agent can do internally, it lists three: retrieval, which reads from long-term memory, reasoning, which updates working memory, and learning, which writes to long-term memory (arXiv 2309.02427). Learning is therefore an action like the others, one the agent may choose to take. What it can write is broader than one might expect. CoALA lists four targets: the record of past experiences, the store of facts, the parameters of the model, and the agent’s own code.

That last target has a long history. Soar, a cognitive architecture from the symbolic era of AI, could already write new rules into its own procedural memory, which CoALA describes as “effectively updating its source code”. An agent that edits its own procedures is an old ambition, and language models have made it practical.

Voyager shows what the practical version looks like. Voyager is an agent that plays the game Minecraft, and it keeps “an ever-growing skill library of executable code for storing and retrieving complex behaviors” (Voyager, arXiv 2305.16291). A skill here is a small program the agent wrote itself. When Voyager meets a new task, it writes code, runs it in the game, reads the errors, and corrects it. Once the code works, it is stored. The next task that needs the same behavior calls the stored program and skips the planning that produced it.

More recent work treats the library itself as the object to manage. MUSE-Autoskill organizes skills under a single lifecycle with five stages: creation, memory, management, evaluation and refinement (arXiv 2605.27366). SkillOS separates two roles. An executor, left unchanged, carries out the tasks. A curator, trained for the purpose, decides what enters the skill repository, what is updated and what is deleted (arXiv 2605.06614). Its skills are plain Markdown files, closer to written procedures than to code.

The reason so much effort goes into curation is that generated skills are of uneven value. One study states it bluntly: even skills generated by leading commercial models show “highly mixed utility, with many providing little benefit or even degrading performance” (arXiv 2606.08755). Its answer is to test each candidate before storing it. The system runs the same task with and without the skill, keeps the skill only if it helps and is not redundant, and prunes the ones that have gone out of date.

A second weakness appears when a skill leaves the setting it was learned in. A benchmark built to evaluate procedural memory across workplace roles finds that some evolved skills become specialized to the workflow of one role and “lose effectiveness under transfer” to another (arXiv 2606.23127). A way of working that suits one role can be the wrong habit in another.

Table 12.1 organizes these mechanisms by what gets written, following CoALA’s four targets.

Table 12.1 Where an agent's capabilities can change

What is written Mechanism Example in the sources Main risk
Record of experiences Storing what happened in past runs Episodic memory in CoALA (arXiv 2309.02427) A store that grows until retrieval degrades
Store of facts Writing what was learned about the environment Semantic memory in CoALA A wrong fact, once written, is reused
Agent code and procedures Writing, testing and curating skills Voyager, MUSE-Autoskill, SkillOS (arXiv 2605.06614) Skills of mixed value; loss of effectiveness under transfer
Model parameters Training the model on its own runs Policy optimization alongside skill generation (arXiv 2606.08755) The costliest change, and the hardest to undo

The reserve on this lever is the youth of its literature. Each system above is evaluated by its own authors on a benchmark of their choosing. None of the sources read for this report compares an evolving agent with a static one at the same token budget, so the net gain remains unknown. Storing and testing skills costs tokens too, and no paper here subtracts that cost.

There is a risk that the papers mention less than the gains. An agent that writes its own tools can write a flaw into them. A skill with a hidden error, once stored, is reused on every later task that calls it, and a mistake that would have cost one run now costs many.

Delegation boundary

Delegable: extracting candidate skills from past runs, writing the code or the procedure, testing it, and indexing it. Not delegable: the rules that decide which skills are admitted, and the approval of any permanent change.

The reason is the same one that separates a personal shortcut from a company procedure. An employee is free to find a faster way of doing their own task. Changing the official procedure that everyone follows goes through review, because others will depend on it. An agent can judge whether a skill works. It cannot judge whether the organization wants that skill to become the way things are done, with the security and maintenance obligations that follow.

For the designer, the consequence is to keep two things apart: the place where the agent experiments, and the registry of capabilities the system relies on. A new skill is a hypothesis until it has passed tests on varied cases. The choice on this lever is therefore where to put the gate between a temporary adaptation and a permanent capability, and who holds it. The next chapter is about that kind of gate in general: the points at which a human decides.

13. Human control: a gate approved 93% of the time is a reflex, not a control

The last lever concerns the person outside the system. Through what surface does a human see what an agent is doing, and at what points can they stop it, correct it or take over? This chapter argues that the usual answer, asking for permission at every step, defeats itself. A gate works as a control only when it is rare and when the person at the gate has what they need to decide.

The usual answer is easy to understand. An agent that can act on the world is a risk, so the designer places a human in front of every action: each file edit, each command, each network call waits for approval. The design assumes that a person who is asked every time will check every time.

Measurement says otherwise. Anthropic reports from its own telemetry that users approved roughly 93 % of permission prompts, and adds the mechanism: the more approvals a user sees, the less attention they pay to each (Anthropic). A person asked a hundred times a day stops reading. The gate is still there and the control has gone, which is worse than having no gate, because everyone believes the action was checked.

The chapter on the Action capability lever described the first response to that failure. Technical capacity and approval policy are two separate settings, and OpenAI’s Codex shows the pattern by cutting network access and removing secrets before the agent starts (OpenAI). Containment handles the routine. What remains for this chapter is the gate itself: when one should exist, and what makes it worth a human’s attention.

Figure 13.1: Two timelines of the same run, one with a gate before every action and one with a single gate before the irreversible step. Figure 13.1: The same run under two arrangements of Table 13.1, drawn by this report. The approval rate is the one Anthropic reports from its own telemetry, and the contained zone follows the pattern described by OpenAI.

A framework for answering that question comes from research on oversight in general. Gaube et al., a group of twenty researchers who set out the conditions for effective human oversight of AI systems, start from a warning. Badly designed oversight creates an illusion of control, when its structures “give humans the responsibility, but not the agency, to prevent harm” (arXiv 2605.16278). The sentence describes the per-step prompt exactly, although the paper was not written about it. The person clicking “approve” is responsible for the action and has no real means of judging it.

Their framework says what real means would be. Oversight, in their account, is a layer of its own that sits above the task. It runs a loop of observing, deciding and intervening, and it operates at three paces: in real time, while the task runs, at the level of the system, when it is retrained or redesigned, and at the level of compliance, when obligations are enforced.

Five preconditions decide whether that layer functions. Three concern monitoring. The information has to be available, the person has to detect it, and they have to interpret it correctly. Two concern intervention. The person must be capable of acting, which includes having the authority and the skills, and an intervention must be feasible at all. A prompt that shows a raw command and waits for a click can fail all five: the relevant information is elsewhere, nothing draws the eye to it, and the person has neither the time nor the context to refuse.

The same authors describe three increasingly demanding ways of making oversight effective. In handover, a human takes over when the system fails or reaches the limit of its competence. In error detection, a human catches and corrects mistakes while the system works. In synergy, the two together do better than either alone. They are candid about agents: the kind of real-time oversight their paper mainly studies “may be infeasible” for agentic systems. For a fast, long-running agent, error detection in real time is out of reach, which leaves well-prepared handover and oversight at the slower paces.

Two earlier levers decide whether a gate can be prepared well. The Verification lever produces checks that a system reads in order to continue. A human needs something different: a short, legible account of where things stand. And the State persistence lever supplies the pause. A run that stops for a human has to hold its whole state while it waits, sometimes for hours. It then resumes from the same point, or rolls back cleanly if the answer is no.

Takeover is the hardest form of control, and it shows why persistence matters. Suppose an agent stops halfway through a migration, because a provider is down or because it has reached the limit of what it may do alone. A human has to finish the work. That is possible only if the harness left something a person can read: the files as they stand, a list of what was done and what was not, and the diffs. A human handed a dead session and a transcript of forty turns cannot take over. They can only start again.

Regulation turns part of this into an obligation. Article 14 of the European Union’s AI Act requires high-risk AI systems to be designed so that people can oversee them effectively (AI Act, art. 14). The article names automation bias, the tendency to rely too much on what a system outputs, as something the people overseeing must be able to remain aware of. It requires that they be able to disregard or override an output, and to interrupt the system through a stop button or a similar procedure. For one category, remote biometric identification, it goes further: no action may be taken unless at least two people have separately verified the result.

Table 13.1 sets four arrangements side by side. The readings in its last column are this report’s, and they are not legal opinions.

Table 13.1 Four ways of placing a human in relation to an agent

Arrangement When the human is called What it asks of the human How it fails Reading against Article 14
Containment without gates Never during the run Nothing routine A breach of the perimeter goes unseen Not an oversight measure in itself
A prompt at every action At each write or tool call Constant attention Reflex approval, at the 93% rate measured (Anthropic) Oversight in name, exposed to automation bias
Gates at checkpoints Before an irreversible step, on a prepared summary A real decision, a few times per run A summary too long or too thin to judge Closest to the intent of the article (AI Act)
Takeover after failure When the system stops or is stopped Resuming someone else’s unfinished work Late detection; nothing readable left behind The ability to interrupt is required

The reserve on this lever is the thinness of its numbers. The 93 % comes from one vendor’s telemetry on its own product (93 % approval metric). No study read for this report compares the quality of human decisions at different gate frequencies under controlled conditions. That gates should be rare is well argued and poorly measured.

Delegation boundary

Delegable: filtering routine actions, running policy checks, preparing the diff and the summary a human will read, and acting inside bounds that were validated beforehand. Not delegable: approving an irreversible action with real stakes, changing the rules, and answering for the outcome.

A harness can process telemetry and summarize a state. It cannot carry responsibility, because nothing happens to a harness when things go wrong. Handing the final approval to an automated check, or obtaining a human signature on a screen nobody could evaluate, produces the form of oversight and removes its substance.

The design that follows from all this replaces many small gates with a few serious ones. Routine actions are contained, so that nobody needs to approve them. Irreversible steps get a gate, placed at a durable checkpoint, with the state shown as a diff a person can read in a minute. A designer can test the result with one question: when the human is called, do they have what they need to say no?

That completes the eight levers. Before they are used to read the rest of the field, two checks are due. The next chapter asks what the eight leave out, and the one after tests whether they can tell two real systems apart.

14. The list is not closed: two dimensions sit outside the eight levers

Eight chapters have each presented one lever. Before the set is put to the test, this chapter asks what it leaves out. The honest answer is that the list is not closed. It has already grown once, and two dimensions of agent design still sit outside it.

The growth came from a gap between two chapters. State persistence restores what a system was: its variables, its history, the point where it stopped. Restoring a state does not change what the system is able to do when a new problem arrives. An agent that only reloads its past will solve the same problem by the same trial and error every time. CoALA treats that missing piece as an action in its own right, a learning step that writes to long-term memory (arXiv 2309.02427). The Capability evolution lever was added to cover it, which is how seven levers became eight.

A list that had to be extended once can need extending again, and the literature points to two places where it would.

Figure 14.1: The eight levers inside a frame, with goals on one side and interaction between agents on the other, both outside it. Figure 14.1: What the eight levers cover and what they leave out, drawn by this report. Beliefs and intentions fall under two levers, while goals (CoALA) and interaction between agents (arXiv 2501.06322) stay outside.

The first is goals. The Belief-Desire-Intention model introduced in chapter 5 keeps three things apart: what an agent believes, what it wants, and what it has committed to do. The levers cover the first and the third. Beliefs are the business of the Context lever, and the choice of the next action belongs to the Control loop lever. Nothing covers the second. No lever asks how an agent holds a hierarchy of goals over a long run, how it settles a conflict between two of them, or when it gives one up.

Current practice shows the gap. A goal is usually a sentence in the system prompt, written once at the start. Suppose the situation changes halfway through a task, so that the original goal no longer makes sense. A sentence in a prompt cannot register that. It cannot rank two sub-goals that now pull in opposite directions, and it cannot retire an intention that has gone stale.

The second is interaction between agents. The Decomposition and specialization lever describes how a task is cut and what the cut costs in coordination. Cutting a task is one way for agents to relate, and there are others. A 2025 survey of collaboration mechanisms classifies multi-agent systems along several axes at once: how different the agents are, whether they cooperate or compete, how they are connected, and by what protocol they coordinate (arXiv 2501.06322). Agents that bargain, vote or bid against each other follow rules that a division of labor does not describe. The lever sees a team, and has no words for a market or a negotiation.

Table 14.1 sets the two gaps side by side, with the workaround that current harnesses use for each.

Table 14.1 Two dimensions outside the eight levers

Dimension Missing Structural Primitive Common Harness Approximation Operational Failure Mode Architectural Implication
Persistent Goal Hierarchies Explicit tracking of desires (goals) (CoALA) Static prompt text strings and frozen system instructions Inability to prune obsolete sub-goals or resolve conflicting desires dynamically Requires explicit BDI-style motivational state machines
Game-Theoretic Topologies Formal bargaining, voting rules, and auction protocols (arXiv 2501.06322) Sequential handoffs, step slicing, and static role assignments Fails under adversarial dynamics, strategic withholding, or non-cooperative games Demands formal communication protocols and mechanism design

These omissions say something about the status of the framework. A taxonomy claims to be complete, and two missing dimensions would refute it. A design grid makes a smaller claim: the dials it names are real, and a designer who sets all eight has made the decisions that matter most often. Whether those eight dials are enough to tell two real systems apart is a separate question, and the next chapter puts it to a test.

15. Tested on two opposed agents, the levers are a design grid, not a taxonomy

A framework earns its place by telling systems apart. Chapter 5 showed that the first principle cannot do this, since tool use, planning and reflection describe every agent in the same words. The eight levers were introduced to do better, and this chapter checks whether they do, on the hardest pair available.

The pair comes from Inside the Scaffold, the study that read the source code of thirteen open-source coding agents (arXiv 2604.03515). Its authors open with exactly this contrast. A coding agent that explores candidate patches with Monte Carlo Tree Search and one that runs a simple while loop with test-driven retries both count as tool-using, planning, reflective agents under capability taxonomies. Yet their scaffolds, the authors write, “differ in fundamental ways that affect cost, reliability, and failure modes”.

The corpus of the paper holds one agent at each end. Moatless Tools implements full Monte Carlo Tree Search. It selects a node in a tree of attempts, expands it, simulates the outcome, then propagates a reward back up the tree to steer the next round. mini-swe-agent is the deliberately minimal case: one ReAct loop written as a while loop, with a single tool. The paper names it among the closest examples of a pure single-loop agent.

The test is simple to state. If the levers were only a longer way of saying “tool use, planning, reflection”, these two agents would receive the same profile. Table 15.1 fills in the eight levers for both, using only what the paper reports from the code.

Figure 15.1: A harbour inspector on a quay with eight mooring posts, beside a great ship and a rowing boat. Figure 15.1: A harbour inspector puts a great ship and a rowing boat through the same examination, on a quay with eight mooring posts. It is an editorial allegory of the test run in this chapter (Inside the Scaffold, arXiv 2604.03515). The levers put the same eight questions to both agents, and the answers differ.

Table 15.1 The eight levers applied to two opposed coding agents

Lever Moatless Tools (tree search) mini-swe-agent (while loop) Same setting?
Context Structure-aware and embedding-based code search; history selected within a token budget Shell search commands; raw history sent as it is, no compaction, crash when the window overflows No
Action capability 15 to 37 action classes; file changes tracked in memory rather than written to disk One shell tool; edits made directly through shell commands in stateless subshells No
Control loop Model-driven; tree search with reward backpropagation Model-driven; one sequential ReAct loop Same driver, different structure
Decomposition and specialization An actor and a critic: a value function, possibly another model, scores each node One model throughout No
Verification and conformance Dedicated validation tools; a discriminator re-evaluates the completed trajectories Validation folded into command execution No
State persistence A tree of nodes, each holding its own snapshot of the files A linear message history; nothing kept from one task to the next No
Capability evolution Tool set fixed at start Tool set fixed at start Yes
Human control Not reported by the source Not reported by the source Unknown

The two profiles are not the same. They differ outright on five levers. On the Control loop lever they share a driver, the model, and differ in structure. They coincide on Capability evolution. The Human control row stays blank, because the paper studies agents run on benchmarks and does not describe a human surface. The levers therefore do what the first principle could not: they say where two agents differ, in terms a designer can act on.

What the levers do not do is sort agents into kinds, and the same paper explains why no framework could. Scaffold architectures, it concludes, “resist discrete classification”. Agents sit along continuous spectra, and eleven of the thirteen stack several loop primitives rather than adopting one. Moatless Tools makes the point by itself. Its per-step agent can be driven either by a plain loop or by the search tree, “with no changes to the agent code itself”. Tree search, in that code base, is a setting someone chose. It is not a species.

That gives the verdict on Hypothesis 1, and it is a split one. The hypothesis holds in the form chapter 4 gave it: one design framework describes two architecturally opposed systems with the same eight questions, and separates them. It fails in the stronger form one might be tempted to read into it. The eight levers are not a taxonomy, and no agent belongs to a class because of its profile. They are a design grid, a set of dials whose positions can be read off any system.

A grid of that kind has a limit, and the limit sets up the rest of this report. Table 15.1 says how the two agents differ. It does not say which one a team should build. The paper states that the differences affect cost, reliability and failure modes, and stops there, because it reads code and measures nothing. Tree search expands many nodes, and every node costs model calls. Whether those calls buy enough to justify themselves is not a question the levers can answer. The levers describe a difference, and something else has to decide between the two profiles. Part V argues that the deciding factor is cost.

Three reserves bound this result. The test rests on one pair of agents, both built for coding benchmarks, read through a single source. The cells hold what that source reports and nothing more, which is why one row is empty. And the test shows that the grid separates two systems, not that eight is the right number of levers. The previous chapter already listed what the eight leave out.

Part III now uses the grid for what it is good at: reading the catalogue of patterns, frameworks and protocols as settings of the eight dials.

Sources cited on this page (38)

Downloads