Guardrails, Safety & the Lethal Trifecta
Identity and authorization decide what an agent is permitted to do. Guardrails decide what it actually does within those permissions — filtering harmful inputs and outputs, and defending against the adversary who will try to turn your helpful agent into their tool. The defining security insight of the agentic era has a name, and every architect must internalize it: the lethal trifecta.
16.1 Guardrails as a control layer
A guardrail is a check that sits around the model, inspecting what goes in and what comes out and intervening when policy is violated. Input guardrails screen for prompt-injection attempts, off-topic or disallowed requests, and PII that shouldn't be processed. Output guardrails screen generated text and — critically — proposed actions before they execute: blocking toxic or non-compliant content, validating that a tool call is permitted and well-formed, and catching attempts to leak data. Guardrails are deterministic policy wrapped around a probabilistic core (Chapter 2's founding move), and they belong in the request path, not in a dashboard reviewed after the fact.
16.2 The lethal trifecta
The central security hazard, articulated by Simon Willison, is that catastrophic risk emerges when an agent combines three capabilities at once. Each alone is benign; together they are exploitable.
The attack is concrete: a malicious instruction is hidden in a web page, email, or document the agent reads (untrusted content); that instruction tells the agent to gather sensitive information it can access (private data) and transmit it somewhere — an HTTP request, an email, a tool call (external communication). Because the model cannot reliably distinguish data it should process from instructions it should obey, it complies. This is indirect prompt injection, and there is no known way to make a model immune to it. The architectural response is therefore not "prompt the model to refuse" but to ensure the three capabilities are never simultaneously available for a sensitive operation — segregate trust domains, strip or sandbox untrusted content, and deny external egress on any path that has touched private data (echoing the sandbox egress rule of Ch. 9).
You cannot reliably instruct a model not to be injected. Security must be structural: for any action, ensure at most two of {untrusted input, private data, external communication} are present. Treat all retrieved and tool-returned content as untrusted data — never as commands (Ch. 8, 11).
16.3 The guardrail toolkit
| Guardrail | Where | Defends against |
|---|---|---|
| Prompt-injection / jailbreak detection | Input + on retrieved content | Hijacking the agent's instructions |
| Content moderation | Input + output | Toxic, illegal, or policy-violating text |
| PII detection & redaction | Input + output | Processing or leaking personal data |
| Action / tool-call validation | Before execution | Unauthorized, malformed, or destructive actions |
| Output schema & grounding checks | Output | Malformed structure; hallucinated, ungrounded claims |
| Topical / relevance bounds | Input + output | Off-purpose use, scope creep |
Frameworks such as NVIDIA NeMo Guardrails, Guardrails AI, and Llama Guard provide ready components; the gateway (Ch. 6) and policy engine (Ch. 17) are natural places to enforce them centrally so every agent inherits the same protections.
No single guardrail is sound against a determined adversary; classifiers can be evaded and validators have gaps. Treat guardrails as defense-in-depth combined with the structural protections of §16.2, least privilege (Ch. 15), and human approval for high-consequence actions (Ch. 5). Assume each layer will sometimes fail and design so that one failure isn't catastrophic.