The Deployment Substrate: Service Mesh, eBPF & Zero-Trust Networking
Identity (Ch. 15), guardrails (Ch. 16), audit (Ch. 17), egress control (Ch. 9), and the resiliency patterns of Chapter 21 share an awkward property: every service needs them, and a model-directed system spawns a lot of services. You can re-implement mutual TLS, identity-based authorization, retries, timeouts, and circuit breaking in each one — inconsistently, with audit gaps and a poor developer experience — or you can push them down into the network substrate once and let every workload inherit them. On Kubernetes that substrate is the service mesh, increasingly powered by eBPF. This chapter is about doing it without paying more latency, money, or operational risk than it saves.
Security and resilience controls that every service requires should live below the application, not inside it. A paved-road substrate that supplies mTLS, identity, authorization, retries, and telemetry uniformly gives better developer experience and — because the controls are consistent and centrally evidenced — better compliance than any per-team implementation can.
23.1 L4 and L7, and where each lives
Two layers carry traffic, and conflating them is the root of most mesh confusion. L4 is connection-level: TCP/IP, fast, identity by IP and port. In Kubernetes this is kube-proxy (iptables or IPVS) or, increasingly, eBPF in the kernel (Cilium can replace kube-proxy outright). L7 is request-level: it parses HTTP/gRPC and can route on headers, retry idempotent requests, break circuits, and detect outliers — the resiliency patterns of Chapter 21 are L7 behaviors and live in an L7 proxy, almost always Envoy. North-south traffic (into the cluster) is handled by the Gateway API; east-west traffic (service to service) is the mesh's job. The practical rule: do at L4 what only needs connections and identity, and reserve the cost of L7 parsing for where request-level control actually earns it.
23.2 Sidecar versus sidecarless
The classic mesh injects an Envoy sidecar into every pod; it intercepts all traffic and delivers full L7 features and mTLS. The cost is the sidecar tax: per-pod CPU and memory multiplied across thousands of pods, two extra proxy hops of latency on every call, and real operational friction around injection, startup ordering, and proxy upgrades. Linkerd lightens this with a purpose-built Rust micro-proxy rather than Envoy. The newer answer is sidecarless. Istio's ambient mode splits the mesh: a per-node ztunnel (written in Rust) handles L4 and mTLS for every pod on the node, and an Envoy waypoint is added per namespace only when L7 features are actually needed. Cilium takes the eBPF route — identity-aware L3/L4 policy enforced in the kernel with no proxy at all, a per-node Envoy only for L7, and Hubble for flow observability. Both cut the per-pod overhead dramatically and let you adopt L7 selectively.
ztunnel or eBPF) carries mTLS for all pods, and an L7 waypoint is inserted only for the services that need request-level features. Same security posture, a fraction of the overhead.23.3 SPIFFE as the identity spine
Zero trust means authorization is based on who the workload is, not where it sits on the network. That requires a portable cryptographic identity, which is precisely what SPIFFE/SPIRE issues — an SVID (an X.509 certificate or JWT) per workload, the same primitive Chapter 15 used for agent identity. The mesh consumes the SPIFFE ID for mTLS peer authentication and for authorization policy. The payoff is one identity threading the whole stack: the SPIFFE ID that authenticates a TLS connection at the network layer is the same identity the application authorizes against, and the same principal the gateway uses for on-behalf-of token exchange when calling a tool or MCP server (Ch. 15, Ch. 22). One identity, enforced consistently from packet to tool call.
23.4 Choosing — and the cost and DevX case
| Mesh | Data plane | Per-pod overhead | L7 features | Identity | Lean toward it when |
|---|---|---|---|---|---|
| Istio (sidecar) | Envoy per pod | high | richest | SPIFFE-based | You need full L7 everywhere and accept the cost |
| Istio (ambient) | ztunnel (L4) + waypoint (L7) | low | on demand | SPIFFE-based | You want Istio's L7 power but only where needed |
| Linkerd | Rust micro-proxy sidecar | moderate | focused | mTLS by default | You value simplicity and a light, opinionated mesh |
| Cilium | eBPF in-kernel + per-node Envoy | very low | via Envoy | SPIFFE-compatible | You want kernel-level L4 policy, performance, Hubble |
The economics are not a footnote. A sidecar's CPU and memory reservation, multiplied by thousands of pods, is a material line on the cluster bill; ambient and eBPF reclaim most of it. Topology- and locality-aware routing keeps east-west calls inside an availability zone, cutting cross-AZ data-transfer charges that quietly dominate networking spend. And the developer-experience dividend is real: application teams get mTLS, authorization, retries, and telemetry without writing any of it, while the compliance controls those features satisfy — encryption in transit, least privilege, audit — are evidenced once at the substrate instead of audited per service.
Pushing concerns down concentrates them. A proxy crash, a botched certificate rotation, or a bad policy push now has cluster-wide blast radius. Stage mesh config like application code (Ch. 21), monitor the data plane as a tier-one dependency, and remember that an L7 mesh secures transport and access — it is not a substitute for the application-level guardrails of Chapter 16.