A weekend project and a production platform run the same agent loop. What separates them is seven properties that the demo can ignore and the platform cannot. Each property maps directly to one or more layers of the architecture in Chapter 3 — so this chapter is, in effect, the rationale for every box on that diagram.

2.1 Why the demo lies to you

A demo agent runs once, on a happy path, watched by its author, on a single request, with the author's own credentials, and nobody counts the tokens. Every one of those conditions is false in production. The agent runs thousands of times, hits the unhappy paths, is unobserved, runs concurrently, acts on behalf of users with varying permissions, and every token is billed. The seven problems below are simply the seven ways reality differs from the demo.

2.2 The five that are unique to LLMs

Durability, observability, and scale are familiar from any distributed system. The other four are sharpened or wholly created by the language model, and they are what make agentic platforms a distinct discipline rather than "microservices with an LLM call."

Non-determinism inverts testing

In ordinary software, f(x) always returns the same value, so a test asserts equality. An LLM's f(x) is a sample from a distribution; two runs diverge even at temperature 0 due to floating-point and infrastructure variance. You therefore cannot test for correctness by equality. You test distributions of behavior with evaluation suites and graded rubrics (Ch. 19), and you monitor production with online evals because the model, the prompt, or the provider can shift under you at any time.

Cost is coupled to behavior

A conventional service's cost is roughly fixed per request. An agent's cost is emergent: it depends on how many loop iterations the model chooses, how much context it accumulates, and how many tools it calls. This couples your bill to the model's runtime decisions — which is why budget enforcement must live inside the loop, not in a monthly invoice review.

Latency forces architecture, not optimization

You cannot make a frontier model respond in 5 ms. Latency is therefore not a tuning problem but a structural one: you hide it with streaming, parallelize independent tool calls and sub-agents, cache aggressively (prompt caching, semantic caching), and choose smaller models for sub-tasks. The platform must make concurrency and caching first-class, which is why the model gateway (Ch. 6) and the durable engine's fan-out primitives (Ch. 21) are central rather than optional.

Security assumes the model is compromisable

The defining security fact of agentic systems: any text the model reads is a potential instruction. A web page, an email, a file, a tool's output — all can carry an indirect prompt injection that redirects the agent. Combine the ability to read untrusted content, access to private data, and the ability to act externally, and you have what is now called the lethal trifecta (Ch. 16). The platform must assume the reasoning core can be subverted and contain the blast radius with identity, least privilege, sandboxing, and output filtering — defenses that sit around the model because the model cannot reliably defend itself.

2.3 The cost of getting it wrong

These are not theoretical. The recurring production incidents in agentic systems map one-to-one onto the seven problems: the agent that looped until it spent the monthly budget in a night (4); the deploy that killed ten thousand in-flight runs with no recovery (1); the silent quality regression after a provider's model update that nobody caught for weeks (2,5); the agent that summarized an attacker's email and dutifully forwarded the user's inbox to an external address (6). A platform is, operationally, the set of mechanisms that make each of these incidents impossible by construction rather than by vigilance.

With the problems named, we can now lay out the full architecture and show exactly where each is solved.