Part IV · Applied concepts · Chapter 29

Agents and agentic flow

An agent is a model running in a loop: it looks at what it knows, decides on an action, usually a tool call, sees the result, and decides again, until it judges the task done. The word is used loosely, so this chapter pins it down, shows the loop one message at a time, and does the arithmetic that explains why agents fail more often than single calls.

Why it matters

Letting a model choose its own next step is powerful and expensive in reliability. Knowing where your system sits between "code decides every step" and "the model decides every step" tells you what can go wrong, what to cap, and what to test.

Workflows and agents

Two designs bracket every AI application:

A workflow. Code fixes the steps and their order; models fill in some of them. "Classify the message, then draft a reply with the matching template, then check the reply" is a workflow, even though three models may be involved. Predictable, testable, cheaper.
An agent. The model chooses the steps: which tool, in what order, how many times, and when to stop. Flexible enough for tasks whose steps cannot be known in advance, such as searching the web until a figure is found.

Most production systems are workflows with small agents inside them: code owns the overall flow, and a model gets freedom only within one bounded step. Start with the least autonomy that solves the problem, and add more only when a fixed flow demonstrably fails.

The loop

Every agent, however it is built, runs the same loop:

Observe. The context holds the task, the instructions, and everything learned so far.
Decide. The model writes either an action (a tool request) or a final answer.
Act. Code runs the action, with its own checks and limits, and appends the result.
Stop or repeat. Stop on a final answer, or when a limit is reached: turns, tool calls, money or time.

The pattern of interleaving short reasoning with actions and their observations was described in the paper "ReAct: Synergizing Reasoning and Acting in Language Models" (2022). Modern models are trained to work this way through tool calling (chapter 28), so the loop is usually a few lines of code around the model call.

Why long loops fail more often

If each step of a task goes right with probability p, independently, a run of n steps goes right every time with probability pⁿ. The numbers fall fast:

Each step right	Steps	Whole run right
0.95	10	0.95¹⁰ = 0.599
0.95	20	0.358
0.99	10	0.904

Turned around: for a 20-step run to succeed 90% of the time, each step must succeed with p = 0.9^1/20 = 0.9947. Real steps are not independent (one early mistake often poisons the rest) and an agent can sometimes notice and fix a mistake, so this is a rough guide, not a law. Its lesson holds: fewer steps, more reliable steps, and checks that catch an error before the next step builds on it.

Common patterns

Pattern	How it works	Good for
Chaining	Fixed steps, each model call using the previous output	Tasks that split cleanly: extract, then summarise
Routing	A classifier picks which path handles the input	Different kinds of question needing different experts or models
Fan-out, fan-in	Several calls run in parallel, then their results are merged	Independent sub-questions; voting
Orchestrator and workers	One model plans sub-tasks at run time and hands them to others	Tasks whose parts are not known in advance
Generate, check, revise	One step produces, another evaluates, the first revises from the feedback	Outputs with clear acceptance rules

Putting the pieces together

A sturdy design uses several of these patterns at once. Code owns the outer graph, with fixed steps and edges: clean the input, classify it, plan which helpers to run (at most three, say), run them in parallel, merge their answers, check, render. Routing there is plain code driven by the labels the classifier returned.

Each helper is a small, bounded agent: one model with its own allow-listed tools, at most 8 tool rounds, and a structured answer. At the limit one last call with no tools makes it answer from what it has read, and a repeated tool call gets its earlier result back with a note to answer now (chapter 28). The merge step joins or rewrites the helpers' answers (fan-in).

The check step is generate, check, revise: deterministic rules first, then a model check. A failed answer gets one retry, on a stronger model tier, with the original question followed by the block reasons; if that fails too, or the tier is already the top one, the user gets a plain fallback message such as "Please talk to a person about this one." Settling facts that matter, such as which of several figures to accept, is done by code, not by the agent (chapter 27).

Budgets, permissions and state

Cap everything that can grow. Model turns, searches, page fetches, money and wall-clock time. A good cap returns a message the model can act on ("the search limit for this run is reached"; at the spend limit, "stop and answer") rather than an exception that loses the work so far. A web-research agent might get 8 turns, 6 searches, 10 fetches and a spend limit.
Separate deciding from doing. The model proposes; code disposes. Irreversible actions (payments, sending, deleting) go to a person, or at least to a rule check, never straight from a model's request.
Least privilege. Give each agent only the tools its task needs. A search agent needs no write tools.
Explicit state. Keep the run's state in a structure your code owns (a graph's state object, a database row), not only in the model's context, so a run can be inspected, resumed or replayed.
Trace every step. When an agent goes wrong at step 7, you need steps 1 to 6 with their inputs and outputs (chapter 45).

Try it

An agent's run has 8 steps. Testing shows each step is right 97% of the time. Roughly how often does the whole run go right, and how many of 200 runs would you expect to contain at least one wrong step?

Answer

0.97⁸ = 0.784, so about 78% of runs are fully right and about 200 × 0.216 = 43 runs contain at least one wrong step, if the steps fail independently.

Build it

In Lab 5 you build this loop yourself for a bike shop's parts desk: four tools, a step limit you can watch stop it, and a pretend order that only runs after a person types y.

Self-check

What is the difference between a workflow and an agent?

In a workflow code fixes the steps; in an agent the model chooses its next step, including when to stop.

Why should a cap reply with a message instead of raising an exception?

The model can read the message and finish with what it has; an exception discards the whole run.

Each step of a 10-step task is right 95% of the time. About how often is the whole run right?

0.95¹⁰ = 0.599, about 60%, assuming independent steps.

In the design above, which parts are chosen by a model, and which by code?

Models choose the labels (the classifier), each helper's tool calls and the merged wording. Code chooses the order of steps, which helpers run, the retry and the fallback.