Agents in the Age of Resilient Systems: Where Does the Damage Stop?

An Agent Enters the System as a New Client

On December 19, 2024, Anthropic published Building effective agents and drew a line in it. In workflows, models and tools are orchestrated through predefined code paths. In agents, the model directs its own process and its own tool use.

The architectural reading of that line: an agent is not our code running inside our services. It is a client that comes in through the boundary and makes its own decisions. Between services, the party on the other side is deterministic. It reacts the same way to the same input, and when it fails, it mostly fails the same way. An agent given the same task twice can pick two different paths, and each path looks reasonable every time. So it has to be treated like user input: its authority is questioned at the boundary, and every call it produces has to pass the question “who is making this request, and on what basis?” The assumption “it’s our code, we can trust it” collapses here.

The classic resilience question is built around a service going down. What happens if the payment service stops responding? Michael Nygard’s Release It! (first edition 2007, second edition 2018) presents timeouts, circuit breakers and bulkheads as answers to that question. When a service goes down, an alarm rings and a graph turns red.

With agents comes a second question: what happens when a wrong but plausible-looking action is taken?

Let me set up a thought experiment, not an incident that happened. An agent is given the task “fix the customer’s duplicate invoice.” It inspects two invoices, sees that the amounts match, and cancels one. The call is valid, the authority is sufficient, the service is healthy, the response is 200. But the cancelled invoice belonged to a different order, and the matching amounts were a coincidence. No service went down, no timeout fired, no alarm rang. The system did the wrong job at full capacity.

This example widens the definition of resilience. The question “is the system up?” gets a companion: “is the system that is up doing the right work?” An outage used to come in through one door. A wrong but plausible action can come in through any door the agent can reach. The sections below cover how to narrow authority, where to stop the damage, and which new brakes can catch wrongness.

Authority Is Scoped to the Task, Control Flow Stays in the Skeleton

OWASP’s 2025 list for large language model applications includes Excessive Agency as LLM06. It covers giving an agent more functions, permissions and autonomy than it needs. Running the agent with the user’s session is the easiest way to make this mistake, because the token is already in hand and the integration takes a day.

To see the cost, look at the user’s permissions. A support employee can read customer records, start refunds and delete account notes. If the agent runs with this employee’s token, it carries all of that, even when the job in hand is “summarize the status of this request.” When the agent reaches for the wrong tool through a misunderstanding or an instruction buried in text, the only thing drawing the boundary is the user’s role. And nobody designed that role with an agent in mind.

The fix is to start authority at the task and end it at the task. When the agent begins work, it gets a short-lived identity for one job, over one set of records. A summarizing task comes with read permission, and the write tool is never shown to it. If you cap the identity’s lifetime at thirty minutes, a leaked token is useless thirty minutes later. Shortening the tool list is also part of the permission constraint, because an agent cannot call a tool that is not on the list.

The same logic applies to control flow. By the distinction I mentioned at the start, in a workflow you write the order of the steps, while in an agent the model rebuilds the order on every turn. Steps like payment and stock decrement have a known order, a known retry behavior and a known compensation. If you tie them to a deterministic skeleton such as a saga or a workflow engine, then at failure time you know which step you are in, what will be rolled back and what can be repeated. The saga idea goes back to the 1987 paper of the same name by Hector Garcia-Molina and Kenneth Salem, written to split long-running transactions into reversible steps. In a flow left to the model’s own plan, you can only reconstruct this knowledge by reading traces.

The agent works at specific points inside this skeleton. These are the places where the input is unstructured and the right answer cannot be written as a rule: which category an incoming request belongs to, or which of three candidate records matches. At each decision point the agent returns a narrow output. The skeleton validates that output, rejects it if it is not one of the allowed values, and chooses the next step itself. Between the agent saying “start the refund” and the system starting the refund, this validation stands.

You can ask this question about your own systems: in this flow, which step’s order has been left to the model, and why shouldn’t that step be code?

Damage Is Tiered by Write Type

Even when authority is scoped to the task, damage is possible if the task includes writes. You can measure the area where an agent can do harm in four tiers, by the type of write permission.

The first tier is read-only. The agent queries, reads and summarizes. If it gets something wrong, it produces a wrong answer, and the person who trusted that answer pays for it. The second tier is approved writes: the agent prepares the change as a draft, and it is applied once a human or a rule engine approves. The third tier is reversible writes. Records are not deleted, their state changes, the old value is kept, and there is a path that reverses the operation. The fourth tier is irreversible writes, such as a sent email or a charged payment. Here the agent is not given authority alone. At the very least there is an amount cap and a step cap, and most of the time approval is required. As the tier goes up, the cost of approval goes up too, and that is a deliberate choice: human time is spent where it is most expensive.

In the third and fourth tiers, idempotency becomes mandatory. The agent retries a call that timed out, and the model can generate the same plan twice and send the same refund request twice. The service receiving the call cannot tell the difference. The fix is an old distributed-systems lesson: every action carries an idempotency key generated by the system, not by the agent, and when the receiver sees the same key a second time, it returns the first result. If you derive the key from the task ID and the step number, a retry produces the same key.

Stale data is a separate risk. If the view the agent reads is outdated, a correctly working agent makes a wrong decision. If an order has been cancelled but the read replica has not caught up, the agent gives the customer shipping information or starts a second refund. Streaming the database change log with CDC ties the agent’s view to the source. The agent attaches the version of the last event it processed to its decision, and when it writes, the service checks whether that version is still current. An outbox protects the write side: the business record and the “this event will be published” record land in the same transaction, so a change triggered by the agent exists either in both the database and the stream or in neither. Debezium is one of the known ways to read this stream.

Service boundaries narrow the damage beyond these tiers as well. If the agent’s write authority is limited to a single service’s API, that service’s own rules become the last line of defense. If the order service rejects a refund with a negative amount, the agent cannot send one either. In services with blurry boundaries, an agent can write to several tables through a path nobody designed. The bulkhead pattern in Release It!, which confines a failure to one compartment, applies to agents too.

Assign every agent tool you have to one of these four tiers. The tool you cannot assign to any of them loses its permission.

Silent Wrongness and Loop Cost Call for New Brakes

Narrowing authority and tiering damage limit what the agent can do, but they do not help you notice that what it did was wrong. For that, you have to look at the brakes you already have.

A circuit breaker cuts the call when a dependency starts returning errors. A timeout does not let a call with no response wait forever. Both are Release It! patterns, and both work on the same assumption: what breaks announces itself as an error. An agent breaks that assumption.

A wrong but plausible-looking result comes back with HTTP 200. The agent writes the wrong refund amount to the customer, takes a record for a duplicate and deletes it, passes an incomplete summary to the next step as complete. No threshold is crossed, so the circuit does not open and no alarm rings. Because each step takes the previous step’s output as input, a small deviation grows a few steps later and its source gets hard to find.

Cost blows up by another route. If the agent decides to retry a failing tool with different arguments, every attempt is a successful call. It does not hit a timeout, and the error counter does not rise. The bill accumulates on every turn as tokens and tool calls. Classic brakes look at duration and error rate, and an agent stuck in a loop fits neither.

That is why two new limits are needed. One is the maximum number of steps that can be taken for a task, the other is the maximum amount that can be spent on it. When a limit is exceeded, the agent does not continue. It stops the task and leaves its state behind. The numbers have to come from measuring your own workload. I am not writing a number whose source I cannot show.

Catching silent wrongness takes a trail that can be read afterwards. An ADR records an architectural decision together with why it was made that way. The runtime counterpart is the decision log and the trace: which input the agent saw at each step, which tool it called with which arguments, which option it chose and why. Without that record, when a wrong result turns up, you answer “why did the agent do this?” with a guess.

Human approval needs to be designed in the same frame. If you put an approval on every step, nothing is left of the agent’s speed. If you put none, damage flows unchecked. Tying approval to two points works more robustly: when a set fraction of the budget has been spent, and when an irreversible write comes up. The first limits cost, the second limits damage. If the approver can see the agent’s trail at decision time, the approval becomes a real decision. If they cannot, they tick the box on the form and move on.

To start on your own system, pick one agent task, write down its step and spending ceilings, and define in a single sentence what happens when they are exceeded.