← manishpande.in Contents
Reference architecture · 2026 The Agentic Platform by Manish Pande
© 2026 Manish Pande Mumbai, India Set in Space Grotesk · Source Serif 4 · JetBrains Mono
SCALE · Multi-Tenancy Chapter 26

Multi-Tenancy

A platform, by definition, serves more than one customer. The moment it does, a new property dominates every design decision: isolation. One tenant must never see another's data, exhaust another's capacity, or be exposed to another's failure — and yet pooling resources across tenants is the only way the economics work. Multi-tenancy is the discipline of getting that trade-off right, separately, at every layer of the stack.

26.1 The isolation spectrum: silo, pool, bridge

Isolation is not one switch. For each resource you choose where to sit on a spectrum. At one end, the silo model gives every tenant dedicated infrastructure — strongest isolation and simplest blast-radius story, but linear cost and operational sprawl. At the other, the pool model shares one set of resources across all tenants, separated only logically — radically more efficient, but every shared component is now a potential cross-tenant leak and a shared failure domain. The bridge model mixes the two per resource: pool the stateless, cheap-to-share layers (compute, the gateway) and silo the dangerous or regulated ones (data, encryption keys), or tier it by plan so premium tenants get dedicated capacity. The right architecture is almost always a bridge, decided resource by resource rather than globally.

Shared control plane (tenant-aware)orchestrator · gateway · registry · routing Pooled stateless data planeagent / gateway / tool workers · every request carries tenant_id Tenant A data · RLS / schema-per-tenant BYOK key policy · quota Tenant B data · RLS / schema-per-tenant BYOK key policy · quota Tenant C data · RLS / schema-per-tenant BYOK key policy · quota tenant-scoped isolate data · keys · policy · quota per tenant; pool stateless compute — tenant is a dimension of every request
Fig 26.1 · A typical bridge architecture. The control plane and the stateless data plane are pooled and shared for efficiency, but every request carries a tenant_id that scopes it. The dangerous resources — each tenant's data, its encryption key (often customer-managed, BYOK), its policy, and its quota — are siloed. Pooling the cheap layers and siloing the regulated ones is the cost/isolation sweet spot.
Table 26.1 — Isolation models and their trade-offs
ModelDataComputeBlast radiusCost efficiencyFits
SiloDedicated per tenantDedicatedContainedLow (linear)Regulated / enterprise tenants paying for isolation
PoolShared, logically separatedSharedWideHighMany small tenants; cost-sensitive SaaS
BridgeSiloed (keys, store)PooledMixedHigh on computeThe pragmatic default — pool the cheap, silo the dangerous

26.2 Isolation is a decision at every layer

Tenancy is not a feature of one component; it is a property the whole stack must carry. Data — separate by a row-level tenant_id with database row-level security, a schema per tenant, or a database per tenant as isolation needs rise; vector and graph stores get per-tenant namespaces or collections; sensitive tenants get their own encryption key (customer-managed keys make a leak cryptographically impossible across tenants). Identity — the tenant is part of every principal, encoded in the SPIFFE ID or OAuth claims (Ch. 15), so authorization is tenant-aware by construction. Compute — sandboxes are per-tenant (Ch. 9, 24) and mesh namespaces segment traffic (Ch. 23). Policy — each tenant can carry its own OPA policy and configuration (Ch. 17). Cost and rate — per-tenant quotas and budgets, enforced as bulkheads (Ch. 20, 21). Observability — every span, metric, and log line is tagged with the tenant (Ch. 18), or you cannot attribute cost, debug a tenant, or honour a deletion. Miss the tenant dimension in any one of these and you have a leak or an un-attributable cost waiting to surface.

26.3 Noisy neighbours and fairness

Pooled capacity creates the noisy-neighbour problem in its sharpest agentic form: one tenant's runaway loop or aggressive fan-out can consume the shared inference floor (Ch. 21) and starve everyone else. Classic rate limiting is necessary but blunt. The platform needs fairness: per-tenant concurrency limits and token budgets so no tenant exceeds its share; bulkheads (Ch. 21) that isolate each tenant's resource pool so one tenant's saturation cannot drain the whole system; and priority tiers so a premium tenant's traffic is scheduled ahead of best-effort load when capacity is scarce. The durable engine's queue (Ch. 4) becomes the place fairness is enforced — admit and schedule per tenant, rather than first-come-first-served.

26.4 Tenant-aware configuration

The corollary of per-tenant policy and limits is that almost everything becomes configurable per tenant: prompts, enabled tools, model choice and routing, guardrail strictness, data-residency region, budgets, and feature flags. The clean design is a tenant registry in the control plane that resolves a tenant's full configuration at request time, so a single shared data plane behaves differently per tenant from one source of truth — rather than forking the deployment per customer. This is the same control-plane/data-plane separation of Chapter 3, now with the tenant as the key.

First principle · Isolate by default, pool by exception, make tenant a dimension of every request

Start from the assumption that resources are siloed and relax to pooling only where it is both safe and economically necessary. Every request, every row, every span, and every budget carries the tenant identifier from the edge inward — so isolation is enforced structurally at each layer, not remembered by convention.

Hazard · Cross-tenant leakage is the unforgivable failure

The catastrophic multi-tenant bug is one tenant seeing another's data. It hides in the shared layers: a retrieval query (Ch. 11) or memory lookup (Ch. 10) without a tenant filter, a cache key that omits the tenant, a log or trace that mixes tenants, or a model given context spanning two tenants. Filter by tenant at query time in every store, scope every cache and prompt to the tenant, and never let a single context window contain two tenants' data. One such leak can end the platform.

· · ·