AI Agents for Real Work: Architecture, Guardrails and Failure Modes

A support agent we inherited from another team issued the same refund four times in eleven seconds. Not four different refunds. The same order, the same amount, four Stripe calls, because the tool returned a 504 after the charge had already been reversed, and the model did what models do when a tool “fails”: it tried again. Total damage was under 200 GBP and the finance team caught it the same afternoon. The fix took twenty minutes. The fact that nobody had thought about it for six weeks of development is the actual story.

AI agents stop being a demo the moment they touch a system that can lose money, leak data or email a customer. Everything that makes the demo impressive (free-form reasoning, dynamic tool selection, long loops) is exactly what makes production painful. The teams shipping agents that survive are not the ones with cleverer prompts. They’re the ones who moved all the important decisions out of the prompt entirely.

This is what we’ve learned building and rescuing agent systems on client work: the architecture that holds, the guardrails that actually fire, and the failure modes you will hit in week three.

What actually counts as an agent

An agent is a loop where the model decides what happens next. That’s the whole definition worth using. If your code decides the order of operations and the model only fills in text or picks from a fixed set of branches, you’ve built a pipeline with an LLM in it, and that’s usually a better system than an agent.

We apply a blunt test on client projects. If you can draw the flowchart and it has fewer than about six nodes with no cycles, build the flowchart. Deterministic orchestration gives you retries you control, errors you can trace, and a cost per run you can quote to a client before you build it. Autonomous agents earn their complexity when the task has an unbounded search shape: triage this inbox of 400 mixed-intent messages, reconcile these invoices against a ledger where the matching rules keep changing, debug why this build fails on CI but not locally.

The honest split from our last year of work: roughly two thirds of the things stakeholders asked for as agents shipped as workflows. They were cheaper, they were testable, and clients stopped asking why the output changed between Tuesday and Wednesday.

Agent architecture that survives production

A working agent architecture has four parts, and the interesting engineering is in three of them that nobody blogs about.

The controller owns the loop, not the model

The model proposes. Your controller disposes. It holds the budgets, the deduplication, the policy check and the transcript, and it is plain code you can unit test without a single API call.

async function runAgent(task, ctx) {
  const budget = { steps: 12, wallMs: 90_000, spendMinor: 500 };
  const started = Date.now();
  const seen = new Set();
  let steps = 0;

  while (steps++ < budget.steps) {
    if (Date.now() - started > budget.wallMs) return halt("wall_clock", ctx);
    if (ctx.spendMinor > budget.spendMinor) return halt("spend_cap", ctx);

    const step = await model.next(ctx.transcript);
    if (step.type === "final") return finish(step, ctx);

    // stableStringify sorts keys so {a:1,b:2} and {b:2,a:1} collide
    const fingerprint = ${step.tool}:${stableStringify(step.args)};
    if (seen.has(fingerprint)) return halt("repeat_call", ctx); // the classic spiral
    seen.add(fingerprint);

    const verdict = policy.check(step, ctx);
    if (verdict.action === "deny") {
      ctx.transcript.push(denyMessage(verdict)); // tell the model why, let it adapt
      continue;
    }
    if (verdict.action === "escalate") return pauseForHuman(step, verdict, ctx);

    ctx.transcript.push(await execute(step, ctx));
  }
  return halt("step_budget", ctx);
}

Two things in that code are worth calling out. First, the continue on deny: a denied call that terminates the run frustrates users, while a denied call that returns a readable reason lets the model route around the wall. Second, escalation pauses the run rather than blocking a thread. Agent runs that wait on human approval must be resumable from persisted state, because your approver is asleep.

Tools are the real API surface

Your tool layer is a public API consumed by an enthusiastic junior developer with no memory who will pass malformed arguments at 3am. Design accordingly. Narrow schemas, hard ceilings baked into the type, enums instead of free strings, and no tool that takes raw SQL.

import { z } from "zod";

export const refundOrder = {
  name: "refund_order",
  description: "Refund one order. Over 50.00 GBP requires human approval.",
  schema: z.object({
    orderId: z.string().regex(/^ord_[a-z0-9]{12}$/),
    amountMinor: z.number().int().positive().max(5000), // ceiling in the type itself
    reason: z.enum(["damaged", "notreceived", "duplicatecharge"])
  }),
  async run(args, ctx) {
    // Derived from the run, never from the model. Retries collapse onto one key.
    const key = ${ctx.runId}:refund:${args.orderId};
    if (await ctx.ledger.has(key)) return { status: "already_refunded", key };

    const res = await stripe.refunds.create(
      { payment_intent: args.orderId, amount: args.amountMinor },
      { idempotencyKey: key } // Stripe dedupes server side for 24h
    );
    await ctx.ledger.put(key, res.id);
    return { status: "ok", refundId: res.id };
  }
};

That idempotency key is the single highest-value line in the file. The four-refund incident from the opening happened because the previous build asked the model to “include a unique idempotency key” in the arguments. It did. A new one each time.

Context is a budget, not a bucket

Long transcripts degrade quality well before they hit the context limit. We see noticeably worse tool selection past roughly 30 to 40 steps of raw transcript even on models advertising huge windows. The fix is straightforward: keep a rolling summary, store full tool outputs in a key-value store, put a reference plus a 200 character preview in the transcript, and let the agent re-fetch by ID when it needs detail. That one change cut per-run token cost on a document triage agent by about 60 percent and made its behaviour noticeably more stable at step 20.

Guardrails that fire, and the ones that don’t

Instructions in a system prompt are not guardrails. They’re defaults. Treat anything written in prose as advisory and put the enforcement in three places instead.

• Schema validation before execution. Reject, don’t coerce. A model that sent amountMinor: "50.00" once will send something worse later.
• Authorisation per call, evaluated against the end user’s identity, not the service account. If a human couldn’t do it in the UI, the agent acting for them must not do it through a tool. We’ve seen agents handed an admin API key “temporarily” more times than anything else on this list.
• Blast radius caps. Per run, per user, per hour. One refund per order, five refunds per run, fifty per hour across the fleet, then the circuit opens and pages someone.

Output filtering matters too, but it’s the weakest layer. A second model checking the first model’s work catches obvious policy breaches and misses the subtle ones, and it doubles your latency. Use it for tone and PII redaction on customer-facing text, not as your primary control. If you’re putting agent-written text in front of the public, the disclosure questions in our piece on AI-assisted content ethics apply just as much to an agent’s email replies as to blog posts.

The failure modes you will actually hit

The repeat spiral. A tool returns an ambiguous error, the model retries, the error repeats. Caught by the call fingerprint set above. This is the most common production incident by a wide margin.

Silent partial success. The agent completes four of six steps, hits a wall, and writes a confident summary claiming all six. Fix it structurally: assemble the final answer from the tool result ledger in your code, not narrated by the model.

Confidently wrong arguments. Right tool, plausible but fabricated ID. Regex-constrain identifiers and make tools fail loudly on unknown IDs rather than returning empty results that the model interprets as “nothing to do”.

Prompt injection through retrieved content. This is the one that should keep you up. Simon Willison’s framing is the clearest available: the danger is the combination of access to private data, exposure to untrusted content, and an ability to communicate third-partyly. An agent that reads customer emails, queries your CRM and can send mail has all three, and there is no prompt that fixes it. Break one leg. Separate the summarising agent (reads untrusted text, no outbound tools) from the acting agent (outbound tools, only ever sees structured, validated data).

Cost blowouts from recursion. An agent that can spawn sub-agents needs a depth limit and a shared spend ledger, otherwise a branching task fans out until the bill tells you. Depth 2 covers almost everything we’ve shipped.

Observability, evals and the cost question

Log every step as a structured event: run ID, step index, tool name, hashed arguments, latency, token counts, policy verdict, outcome. Not the prose. The prose is for debugging a single run; the structured events are how you notice that search_orders is being called 4.1 times per run on average when it should be called once.

Build an eval set from real failures. Thirty to fifty recorded runs with known-correct outcomes, replayed on every prompt or model change, is enough to catch regressions that vibes-based testing misses entirely. Pin your model version in config. A silent provider-side update that shifts tool-calling behaviour is a genuine risk, and it’s one argument for self-hosting, with the trade-offs we covered in open source models in production.

On cost: measure per completed task, not per thousand tokens. A run that costs 0.30 GBP and resolves a ticket is cheap. A run that costs 0.04 GBP and escalates 70 percent of the time is expensive, because a human still pays the full cost of context switching into it.

Human handoff is a feature, not a fallback

Design the escalation path first. Which steps require approval, who approves, what they see, how long before it times out, and what happens to the half-finished run. An approval UI that shows the proposed tool call in plain language plus the three facts it was based on gets approved in seconds. One that shows a JSON blob gets ignored.

Default to approval for anything irreversible, anything over a money threshold, and anything customer-visible in the first month of production. Relax the thresholds once your logs justify it, with the numbers in front of you. The same discipline applies to the shallower end of this, which is why most of the chat widgets we assess fail the usefulness test we set out in AI chatbots on marketing sites: no handoff, no memory of the handoff, no record of what was promised.