Ch.3 · The Reference Architecture — The Agentic Platform

FND · Foundations Chapter 3

The Reference Architecture

Here is the whole platform on one page. Each band is a plane that solves one or more of the seven hard problems; each later chapter expands exactly one band. Read it top-down as the path a request travels, and right-to-left as the cross-cutting concerns that touch every plane.

Fig 3.1 · The master reference architecture. Vertical bands are the request-path planes, each expanded in a later chapter. The indigo dashed line marks the control plane driving the agent loop across reasoning, action, and memory. The right rail holds cross-cutting concerns — identity, policy, guardrails, observability, evaluation, metering — that touch every plane rather than sitting in the request path. The foundation supplies the stateful substrate everything depends on.

3.1 The planes and what each one guarantees

A plane is not just a grouping of tools; it is a contract. Each plane guarantees something specific to the planes above it, which is why the architecture composes rather than collapsing into a monolith. Stated as guarantees:

Table 3.1 — Each plane as a contract
Plane	Guarantee it provides	Problem solved	Chapter
CONTROL	A run, once started, completes exactly-once and survives crashes, deploys, and restarts.	Durability	4–5
REASONING	One uniform, observable, cost-tracked, fail-over-capable interface to every model.	Latency, cost	6–7
ACTION	Tools are discoverable, schema-typed, permissioned, and executed under isolation.	Safety	8–9
MEMORY	State persists correctly within and across runs and is retrievable by relevance.	Continuity	10–11
COORD	Agents can discover and delegate to one another over a typed protocol.	Decomposition	12
TRUST	Every action carries an identity, is authorized by policy, and passes guardrails.	Security, compliance	13–15
OPS	Every step is traced, every behavior is evaluable, every token is metered.	Observability, cost	16–18
FOUNDATION	Durable storage, events, cache, and compute beneath all planes.	(substrate)	19

Design decision · Control plane vs data plane

Borrowing from network and Kubernetes design, separate the control plane (decides what should happen: orchestration state, scheduling, policy, identity issuance) from the data plane (does the work at volume: model calls, tool execution, retrieval). The control plane is low-volume, strongly consistent, and durable; the data plane is high-volume, latency-sensitive, and horizontally scalable. Conflating them is the most common scaling mistake — it couples your slow, stateful coordinator to your fast, stateless workers, so neither can scale on its own axis.

3.2 The execution lifecycle

Tracing a single request through the planes makes the architecture concrete. Note where durability checkpoints land and where the cross-cutting rail taps in.

Fig 3.2 · The execution lifecycle. A request is admitted (authorized, budgeted), a durable run is started, and the agent enters the loop. Each turn is checkpointed so a crash resumes from the last persisted step rather than from zero. Throughout, the OPS and TRUST rails tap every step — tracing, guardrails, metering — without sitting in the critical path.

3.3 From logical planes to deployment topology

The planes are a logical decomposition; a platform must also map them to infrastructure. The control-plane/data-plane split above becomes a physical one: a small, strongly-consistent control tier on its own node pool; a large, stateless data tier that autoscales independently; model serving on GPU nodes (the provisioned-throughput floor of Ch. 21); the foundation stores beneath; and the cross-cutting mesh, telemetry, and audit spanning everything.

Fig 3.3 · Deployment topology. The logical planes map onto a physical split: a small control plane (orchestrator, scheduler, policy, identity) on its own node pool, strongly consistent and durable; a large, stateless data plane (agent runtime, gateway, tools/MCP, retrieval, sandboxes, egress) that autoscales independently; GPU serving for the provisioned-throughput floor (Ch. 21); and the foundation stores beneath. The mesh, telemetry collector, and audit log span every zone rather than living in any one.

The rest of the book is this diagram, decompressed. We begin at the bottom of the request path — the control plane — because durability is the property everything else assumes. An agent that cannot survive a restart is not a platform component; it is a demo with extra steps.

3.4 The model-serving plane: self-host, third-party, or hybrid

The foundation so far is stores and compute; the most consequential thing that compute exists to run is the model itself, and one of the earliest architectural decisions is how it is served. A GPU floor is not generic capacity — it exists, when it exists, to serve self-hosted open-weights models on an inference runtime (vLLM, SGLang, or TGI) that you operate. The alternative is to consume third-party model APIs and run no model infrastructure at all. Most platforms end up hybrid: frontier capability from a provider API, routine or sensitive traffic on self-hosted models. Crucially, this choice is not load-bearing for the rest of the architecture, because the model gateway (Ch. 6) is the seam that hides it — application code calls the gateway, and whether a request lands on your GPUs or a provider's is a routing decision (Ch. 6, 28), not an application change.

Table 3.2 — Serving the model: the foundational choice
Option	Control / residency	Cost shape	Ops burden	Best for
Third-party API	Provider-controlled	Per-token, no floor	None	Frontier capability, fast start, spiky load
Self-hosted	Yours · in-region	Fixed GPU floor	High (Ch. 21)	Residency, cost-at-scale, data sensitivity, control
Hybrid	Per-route	Mixed	Moderate	The common end state — route by task

The decision turns on residency and data sensitivity (self-hosting keeps prompts and data in your region and out of a provider's systems — Ch. 16, 27), cost at your volume (a GPU floor amortizes only above a break-even throughput — Ch. 21, 29), latency control, and how much frontier capability only a provider can supply. Whatever the mix, it is governed as part of the model lifecycle (Ch. 28) and reached through the gateway — never wired in.

· · ·