← Manish Pande
Contents
Spine: the 12-stage AIDLC workflow Enablers: harness + loop engineering Window: Apr–Jun 2026, primary only Audit-ready: SOC 2-mapped

One workflow,from spec to self-healing.

A field manual for running the AI Development Lifecycle (AIDLC) end to end: a twelve-stage pipeline where humans steer with specs and gates, fleets of agents do the work in isolated sandboxes, and incidents feed fixes back to implementation. The workflow is the spine of this document.

Harness engineering and loop engineering are how you make each stage reliable — the deterministic scaffolding and the bounded feedback loops underneath the pipeline. They appear throughout as the "how," never as the headline.

Security and compliance run as a rail beneath all of it — the same gates that make agents trustworthy double as audit evidence, mapped to the SOC 2 Trust Services Criteria in §08 and packaged as a detachable auditor's runbook in §16.

12stages
spec → plan → design → implement → test → review → CI → deploy → observe → SRE → docs → security
THE SPINE
~3load-bearing gates
approve the plan · review the PR · promote to prod — the high-value condensation of ~8 human touchpoints (§02)
WHERE JUDGEMENT STAYS
[10]→[4]
agentic SRE feeds remediation back into implementation — a closed loop
SELF-HEALING
2enablers
harness engineering + loop engineering make the stages trustworthy
THE HOW · §09–10

// Model names, benchmarks and prices appear in §12 and are vendor-reported and harness-sensitive. Recency discipline and the full sourcing log are in §15.

01 · The spine · the end-to-end workflow

The AIDLC, end to end

This is the canonical pipeline. A human steering layer sets intent and holds the gates; an orchestration layer fans work out to parallel agents in isolated sandboxes; automated review and security gates stand between a green PR and production; and an agentic-SRE loop closes back onto implementation. Everything else in this manual hangs off these twelve stages.

Rendered as a control plane, work flows top-to-bottom; the violet bands are where a human must act; the green/red dashed bands are gates; the amber block is the orchestration engine; and the dotted arc is the self-healing loop.

How to read the rest of this manual

Sections 04–08 walk the workflow in order, with the real configs each stage needs. Sections 09–10 are the two enablers — harness engineering and loop engineering — that make stages 4–7 trustworthy. Section 11 is the orchestration layer (the fleet manager). The workflow is the destination; the enablers are the road.

02 · Operating model · who owns what

Humans steer. Agents execute. Gates decide.

The pipeline only works because authority is split cleanly. Humans own intent and accountability; agents own throughput; deterministic gates own the go/no-go. The bottleneck has moved from writing code to reviewing it — so the leverage is in specs, decomposition, and the gates.

Humans own

Intent & judgement

  • The spec and acceptance criteria (stage 1)
  • Plan approval (gate after stage 2)
  • Architecture invariants (stage 3)
  • PR review & merge accountability (gate 6b)
  • Promotion to prod for high-risk (gate 8)
Agents own

Throughput

  • Implementation in isolated sandboxes (stage 4)
  • Test generation (stage 5)
  • Self-review iteration (stage 6)
  • Fixing their own CI & remediation PRs (stages 7, 10)
Gates decide

Go / no-go

  • CI with a hard 2-round cap (stage 7)
  • AI review + SAST (gate 6b)
  • SLO-breach auto-rollback (stage 8)
  • Security/compliance throughout (stage 12)

Three load-bearing gates, condensed from about eight. Read end to end, the pipeline has roughly eight points where a human can act (the gate column in §03 tags them at stages 1, 2, 3, 6b, 8, 9, 10 and 12). In practice they concentrate into three high-value gates — approve the plan (after stage 2), review the PR (gate 6b), and promote to production (stage 8). This mirrors the prevailing industry framing that an agentic SDLC reduces roughly eight human interruptions to three checkpoints (Augment Code), raising reviewer leverage while preserving governance.

The human-gate spectrum — the count is calibrated, not fixed

How many gates bind is risk- and autonomy-dependent. At the autonomous end they collapse toward roughly one — exception-based escalation (OpenAI permits but doesn't require human PR review; humans set acceptance criteria and step in only when judgment is required). At the strict / regulated end the count climbs to five or more — spec-driven tools alone gate requirements, design and tasks separately before a single line is written. Stripe sits in between, with mandatory human review on every PR.

Sensitive code is an intensifier, not a separate gate. Auth, crypto, payments and other high-blast-radius changes don't add a stage — they tighten the gates already present, forcing a mandatory human reviewer at 6b and a human promotion decision at 8 even where those gates would otherwise be optional or automated. Set the thresholds explicitly and encode them — don't leave it to habit.

03 · The twelve stages · the master reference

Every stage: what the agent does, the gate, the enabler

One table to hold the whole workflow. The final column is the point of this manual: each stage is made reliable by a specific harness or loop mechanism (detailed in §09–10). The gate tags read HUMAN AGENT HARD.

#StageAgent does → gateRepresentative toolingWhat makes it reliable (the enabler)
1SpecifyNL → structured, testable spec (EARS) HUMANSpec Kit · Kiro · TesslHarness: repo as system of record; spec versioned in docs/
2Plan & decomposeSpec → non-overlapping tasks HUMAN: approve planexec-plans in-repo; plan modesHarness: non-overlapping decomposition (the scarce skill, §11)
3DesignADRs + boundaries HUMAN: own invariantsADRs; layer linters; structural testsHarness: machine-enforced architecture (§09)
4ImplementWrites all code in a sandbox AGENTClaude Code (Opus 4.8) · Codex (GPT-5.5) · Devin · CursorHarness: disposable sandbox + curated MCP; Loop: inner loop
5TestGenerates unit/integration/e2e AGENTagent-gen suites · Playwright/CDP · coverage lintsLoop: tests are the iteration contract (computational sensors)
6Self-reviewReviews own diff, iterates AGENTevaluator subagent · fresh contextLoop: bounded self-review; default-FAIL contract (§10)
7CIFixes own build, capped HARD 2-ROUND CAPGitHub Actions · GitLab CI · pre-push hooksLoop: hard iteration cap → escalate (§10)
6bAI review + SASTAutomated review & scan AGENTHUMANCodeRabbit · Greptile · Diamond · Snyk · Semgrep · CodeQLLoop: inferential sensors as a merge gate (§03,§10)
8DeployProgressive delivery HUMAN: high-risk promoArgo Rollouts · Flagger · feature flagsHarness: auto-rollback on SLO breach bounds blast radius
9ObserveEmits agent-legible telemetry HUMAN: define SLOsOpenTelemetry + Grafana · DatadogHarness: telemetry made legible to agents (LogQL/PromQL)
10Agentic SRETriage → RCA → fix PR HUMAN: escalation thresholdsincident.io · Rootly · Resolve AILoop: bounded remediation; closes [10]→[4] (§07)
11DocsKeeps docs/ fresh AGENTdoc-gardening agent · freshness lintersHarness: scheduled drift/GC agent; CI-validated
12SecurityRuns throughout HARD: auth/crypto/paymentsSAST · dep CVEs · MCP gateway · auditHarness: gateway as single policy-enforcement plane (§08)

The shape of the job. Read the gate column top to bottom: there are about eight points where a human can act, but the load-bearing ones concentrate into three — approve the plan, review the PR, and promote to production — with spec and design sign-off folded into planning, and sensitive-code review tightening the rest. That is the entire post-transition role: engineers move up the stack from authoring lines to authoring intent and judging output.

04 · Stages 1–3 · specify · plan · design

The front of the pipeline is where humans win or lose

Underspecified intent multiplies errors across a whole fleet, so stages 1–3 carry the heaviest human gates. Three tool categories matter here — and they are not interchangeable: a portable process scaffold, an agentic IDE, and a spec-as-source framework.

Spec Kit vs Kiro vs Tessl — three different roles

DimensionGitHub Spec KitAWS KiroTessl
What it isOpen-source CLI + commands/templates that scaffold spec-driven development inside your existing agentAgentic IDE (VS Code fork) + CLI where specs are the unit of workSpec-as-source framework and a registry of 10,000+ OSS "usage specs"
Artifactsconstitution.md, spec.md, plan.md, tasks.mdrequirements.md, design.md, tasks.md + steering/ + hooks/.spec.md (frontmatter + capabilities + [@test]) + usage specs
Flow/speckit.constitution → specify → clarify → plan → tasks → implementCreate Spec → requirements → design → tasks → execute; hooks on saveAgent writes specs → you approve → implement → verify; edits flow spec-first
Role in the pipelinePortable, agent-agnostic process for stages 1–2Enterprise spec-first IDE (esp. AWS) for stages 1–3Specs as durable source of truth + reliable OSS API usage
NOTNot an IDE, not a registryNot a registry, not just templatesNot a scaffold, not an IDE — a registry + spec-as-source framework
Why Tessl is genuinely different (a common conflation)

Tessl treats the specification as the primary artifact and lets code follow. Its Framework keeps specs (or AI-generated "vibe-specs") in the codebase as long-term memory, flows edits spec→code, and detects/reconciles drift; its Spec Registry ships 10,000+ usage specs for OSS libraries, installed like dependencies and optimized for agent readability (Tessl reports up to 3.3× improvement in correct API usage). It runs as a CLI for humans or an MCP server for agents — a different job from Spec Kit (a command scaffold) or Kiro (an IDE).

Spec KitPRIMARY · github/spec-kit · stable
# bootstrap
uvx --from git+https://github.com/\
github/spec-kit.git specify init APP

# inside your agent:
/speckit.constitution
/speckit.specify  <feature>
/speckit.clarify
/speckit.plan     <tech>
/speckit.tasks
/speckit.implement
KiroPRIMARY · kiro.dev/docs · stable
.kiro/
  specs/<feature>/
    requirements.md  # EARS
    design.md
    tasks.md
  steering/   # persistent
              # project knowledge
  hooks/      # on save:
              # lint · test · scan
TesslPRIMARY · docs.tessl.io · in-window
---
name: User Auth
targets: [../src/auth/*.py]
---
Users log in with email +
password.
  def login(email, pw)
      -> Session
[@test] ../tests/test_login.py
# run: tessl eval run .

CONFIG EARS — the structured-spec grammar (stage 1)

EARS keeps acceptance criteria testable. It is the connective tissue between the human's intent and the agent's contract, and is the native requirements format in Kiro.

requirements.md — EARS patternsPATTERN · EARS notation · stable
# Ubiquitous      The system shall encrypt all data at rest.
# Event-driven    WHEN a user submits invalid credentials, the system shall return 401.
# State-driven    WHILE a session is active, the system shall refresh the token every 15m.
# Unwanted       IF a payment webhook signature is invalid, THEN the system shall reject it.
# Optional       WHERE multi-factor auth is enabled, the system shall require a TOTP code.

Stage 3 · architecture the agent cannot violate

Design is enforced mechanically, not by hoping the agent remembers. Encode a forward-only dependency direction and a single seam for cross-cutting concerns as linters set to error; the rule is then a hard gate for every agent on every run.

AGENTS.md — the table-of-contents patternPATTERN · OpenAI harness build + GitHub agents.md analysis · stable
# MyProject — what this service does, in one line

## Commands
- build: `make build`   # lint rules are ERROR-level, not warn
- test:  `make test`

## Where things live (point outward; don't inline detail)
- Architecture & layer rules .... docs/architecture.md
- ADRs (why X over Y) .......... docs/adr/
- Testing strategy ............. docs/testing.md

## Boundaries
- Dependency direction: Types → Config → Repo → Service → Runtime → UI
- Cross-cutting concerns enter ONLY via the Providers interface
- Never edit: .env, infra/prod/**, .git/**
Keep the spec small, keep the rules hard

A monolithic instruction file rots; keep AGENTS.md ~100 lines and point into a structured docs/ tree (one frontier agent-first repo composes 88 of them across subcomponents). And follow the field's highest-leverage habit: "if you can articulate what you don't like about the code, write that down" — as a lint, a structural test, or a bespoke reviewer, set to error.

05 · Stages 4–7 · the orchestration engine

Inside the box: implement → sandbox → test → self-review → CI

This is where loop engineering earns its keep. Each agent runs a bounded inner loop in an isolated sandbox: generate → run sensors → self-review → fix, exiting on pass and escalating when capped. Nothing here is open-ended.

CONFIG The sandbox (stage 4) — disposable & isolated

Isolation is what lets an agent run with full permissions safely; pre-install dependencies at build time so a new agent starts ready to code (the "warm devbox" / cached-dev-container idea). Two real options:

.devcontainer/devcontainer.jsonPRIMARY · imbue.com (Sculptor) · in-window
{
  "name": "my-app",
  "image": "mcr.microsoft.com/devcontainers/python:3.12",
  // run once at build → agents start in seconds
  "onCreateCommand": "pip install -r requirements.txt",
  "forwardPorts": [3000, 8000]
}
e2b — microVM execPRIMARY · e2b.dev · in-window
# pip install e2b-code-interpreter
from e2b_code_interpreter import Sandbox
with Sandbox() as sbx:
    sbx.run_code("x = 1")
    r = sbx.run_code("x += 1; x")
    print(r.text)   # 2

CONFIG Hooks (stages 4–7) — turn the design into enforcement

Claude Code hooks make "the agent can't make that mistake again" literal: auto-format/lint every edit, deny destructive commands, inject branch context, and block stopping while tests fail. exit 2 is a blocking error fed back to the agent.

.claude/settings.jsonPRIMARY · code.claude.com/docs/en/hooks · stable schema
{
  "hooks": {
    "PostToolUse": [
      { "matcher": "Write|Edit|MultiEdit",
        "hooks": [{ "type": "command", "command": "npx prettier --write \"$CLAUDE_TOOL_INPUT_FILE_PATH\" && npx eslint --fix \"$CLAUDE_TOOL_INPUT_FILE_PATH\"" }] }
    ],
    "PreToolUse": [
      { "matcher": "Bash",
        "hooks": [{ "type": "command", "command": "echo \"$CLAUDE_TOOL_INPUT\" | grep -qE 'rm -rf|DROP TABLE' && exit 2 || exit 0" }] }
    ],
    "Stop": [
      { "hooks": [{ "type": "command", "command": "npm test || exit 2" }] }
    ]
  }
}

CONFIG The CI cap (stage 7) — green in two rounds, or escalate

The hard iteration cap is the single most important loop bound: it prevents infinite token-burning loops and routes genuinely hard problems to a human. (Documented at the strict end as a two-round cap.)

ci-cap.shPATTERN · hard 2-round cap → escalate · field practice
# the agent gets at most 2 attempts to make CI green, then a human is paged
MAX=2; n=0
until npm run ci; do
  n=$((n+1))
  [ "$n" -ge "$MAX" ] && { gh pr comment --body "CI red after $MAX rounds — escalating to a human."; exit 1; }
  claude -p "CI failed. Read the logs, fix the cause, do not disable tests."
done
echo "green in $n round(s)"
Loop bounds, in one place

Every inner loop in stages 4–7 carries four bounds: an iteration cap (e.g. --max-iterations or /goal), a budget cap, a kill switch (a watched AGENT_STOP file), and the CI round cap. Configure all four before you let a stage run unattended. Full mechanics in §10.

06 · Gate 6b + the human gate · review & delivery

Between a green PR and production

A green CI means the code runs, not that it's right. Two gates stand between the PR and prod: an automated gate (AI review + SAST + dependency CVEs) and a human gate whose strictness you set by blast radius. These are inferential sensors used as gates — rich but non-deterministic, so they gate rather than decide alone.

PR opened
Template-compliant, green CI

The inner loop (§05) has exited; the change arrives with tests and a description.

[6b] automated gate
AI review + SAST

CodeRabbit / Greptile / Diamond for review; Snyk / Semgrep / CodeQL for security; dependency CVEs.

human gate
Merge accountability

Strict (every PR) → autonomous (optional). Always for auth/crypto/payments/high-blast-radius.

CONFIG AI review gate (6b)

.coderabbit.yamlPRIMARY · CodeRabbit config schema · stable
reviews:
  profile: "assertive"
  request_changes_workflow: true   # block merge until review threads resolve
  path_instructions:
    - path: "src/payments/**"
      instructions: "High blast radius. Require a human approver; flag any auth or money-movement change."
  tools:
    semgrep: { enabled: true }
    github-checks: { enabled: true }
On AI-code risk — and why this manual won't quote a stale number

The design reason the 6b gate is load-bearing is that AI-authored code carries more correctness and security defects per unit than human code — which is exactly why layered review and SAST sit here. An earlier draft of this manual cited a specific December-2025 study for the magnitude; that figure predates the 60-day recency window and has been dropped (see §15). No in-window first-party replacement was found, so the manual states the principle without a stale statistic. Keep the gate regardless — it is cheap insurance against the failure mode.

07 · Stages 8–11 · deploy · observe · SRE · docs

Ship safely, watch closely, heal automatically

Past the gates, the workflow keeps containing risk: progressive delivery bounds the blast radius, telemetry is made legible to agents, an agentic-SRE loop turns incidents into fix PRs that re-enter implementation, and a doc-gardening agent keeps the system of record fresh.

Stage 8 · deploy

Progressive delivery

Canary or flagged rollout with automatic rollback the moment an SLO is breached. The human gate here is reserved for high-risk service promotions.

Stage 9 · observe

Agent-legible telemetry

OpenTelemetry into Grafana/Datadog, with logs and metrics queryable by agents (LogQL/PromQL) so the SRE loop can reason over them. Humans define the SLOs.

CONFIG Deploy with auto-rollback (stage 8)

rollout.yaml — Argo Rollouts canaryPRIMARY · Argo Rollouts spec · stable
apiVersion: argoproj.io/v1alpha1
kind: Rollout
spec:
  strategy:
    canary:
      steps:
        - setWeight: 10
        - pause: { duration: 5m }
        - analysis:                 # auto-rollback if the SLO query fails
            templates: [{ templateName: error-rate-slo }]
        - setWeight: 50
        - pause: { duration: 10m }

Stage 10 · the agentic-SRE loop that closes back onto [4]

This is the loop that makes the pipeline self-healing. On alert, an SRE agent triages, runs root-cause analysis over the agent-legible telemetry, performs a bounded remediation, and opens a fix PR — which re-enters the workflow at stage 4 and passes the same gates. A human stays on the loop via escalation thresholds.

Stage 10 · the principle

Bounded, reversible, approved

The agentic-SRE tools in the field converge on one rule: actions must be bounded, reversible, and require human approval at defined confidence/severity thresholds. Augment the on-call; don't hand over the pager.

Stage 11 · docs

Doc-gardening agent

A scheduled agent keeps docs/ — the system of record — fresh, with freshness validated in CI. This is harness maintenance: fighting entropy on a cadence so the next agent reads accurate context.

08 · Stage 12 · the rail that makes agents auditable

Security, compliance & audit, throughout

Stage 12 is not a step — it's a rail spanning the whole pipeline. In an agentic SDLC the same gates that make agents trustworthy double as your control evidence: the change-management trail an auditor samples is produced automatically, on every PR. The job is to make those gates emit immutable, attributable, replayable records by construction — so a SOC 2 audit becomes a query against artifacts you already have.

The two questions a SOC 2 audit asks

SOC 2 Type 1 · a point in time

Are the controls designed correctly?

Show the gates exist and are sound: branch protection requires review + SAST + green CI; the MCP gateway enforces an allowlist; sandboxes are isolated from prod; deploys roll back on SLO breach.

Evidence = configuration. Largely your settings and policy-as-code, captured at one date.

SOC 2 Type 2 · over a period (3–12 months)

Did they operate effectively?

Show the gates actually fired on every change across the window: each merge had a reviewer who wasn't the author, SAST ran on 100% of PRs, no agent bypassed the gateway, the audit log is complete and tamper-evident.

Evidence = operating records over time. This is where most orgs struggle — and where an agentic pipeline can excel.

The agentic-SDLC advantage

When humans hand-merge, change-management evidence is patchy and reconstructed at audit time. When every change is an agent-authored PR through a fixed gate, the evidence is uniform and machine-generated — provided you wire the gates to record who/what authored it, who reviewed it, which scans ran, and which scoped identity acted. The discipline that makes agents reliable is the same discipline that makes you audit-ready.

Mapping the pipeline to the Trust Services Criteria

Every AIDLC mechanism already maps to a SOC 2 Common Criterion. The columns that matter for the audit are the last two: what proves the control was designed (Type 1) and what proves it operated (Type 2).

AIDLC mechanism (stage)SOC 2 criterionType 1 — design evidenceType 2 — operating evidence
Plan approval gate (stage 2)CC3, CC8.1Documented plan-approval workflowApproval record linked to each change over the period
PR review + human gate (6b)CC8.1Branch protection requires a review100% of merges had a reviewer ≠ author
AI review + SAST + dep CVEs (6b, 7)CC7.1, CC3.2, CC4.1Scanners set as required status checksScan results retained per PR; findings triaged & closed
MCP gateway: RBAC, allowlist, OAuth (4)CC6.1, CC6.2, CC6.3Gateway policy + per-agent scoped identitiesAccess logs; zero shared tokens; periodic access review
Disposable sandbox, no prod/egress (4)CC6.6, CC6.7Sandbox isolation & egress configEvidence agents never touched prod data in the window
Progressive delivery + auto-rollback (8)CC8.1, A1.2Rollout policy with SLO analysisDeploy & rollback records per release
Observability + agentic SRE (9–10)CC7.2, CC7.3, CC7.4Monitoring + incident-response runbookIncident timeline, RCA, remediation PRs
Immutable audit log (12)CC2.1, CC4.1, CC7.2Append-only SIEM pipelineComplete, tamper-evident log across the period
Docs as system of record (1, 11)CC1.x, CC2.1Structured docs/ + ownershipDoc-freshness CI logs; change history

References follow the AICPA 2017 Trust Services Criteria (rev. 2022); Security (CC-series) is mandatory. Confidentiality (C1) and Privacy (P-series) attach the same way when in scope — map the data-handling stages (sandbox isolation, audit redaction) to them.

CC6 · agent identity is the new access-control surface

Agents are non-human identities, and SOC 2's access criteria (CC6.1–6.3) demand least privilege and attribution for every actor. The failure mode is a shared service token several agents reuse — it collapses attribution and over-grants access, failing CC6 on both design and operation. Issue per-agent, per-task scoped credentials, route every tool call through the gateway as the policy-enforcement point, and keep each call attributable to one identity. That is also what makes the periodic access review CC6.2/6.3 expect even possible.

CONFIG Change management as code (CC8.1)

The gate that satisfies CC8.1 should be enforced by the platform, not by convention. A branch ruleset makes review, status checks, and linear history mandatory — and is itself the Type 1 design artifact.

.github/rulesets/main.json — the required gatePATTERN · GitHub repository ruleset · stable
{
  "target": "branch", "enforcement": "active",
  "conditions": { "ref_name": { "include": ["refs/heads/main"] } },
  "rules": [
    { "type": "pull_request",
      "parameters": { "required_approving_review_count": 1,
                     "require_code_owner_review": true,
                     "dismiss_stale_reviews_on_push": true } },
    { "type": "required_status_checks",
      "parameters": { "required_checks": [
        { "context": "sast/semgrep" }, { "context": "deps/cve-scan" },
        { "context": "ci/tests" }, { "context": "compliance/evidence-gate" } ] } },
    { "type": "required_linear_history" },
    { "type": "non_fast_forward" }
  ]
}

CONFIG Make compliance a blocking check, not a hope

A small policy-as-code gate (the compliance/evidence-gate check above) refuses the merge unless the evidence a Type 2 auditor will sample actually exists. Encoding it means the control can't be silently skipped under deadline pressure.

policy/merge_gate.rego — OPAPATTERN · policy-as-code merge gate · field practice
package merge.evidence
default allow := false

allow if {
  input.review.approvals >= 1
  input.review.approver != input.pr.author        # CC8.1 — reviewer ≠ author
  input.scans.sast == "passed"                    # CC7.1
  input.scans.cve_high == 0                        # CC7.1
  input.actor.identity_scope == "per-task"        # CC6 — no shared tokens
  input.plan.ref != ""                            # CC3/CC8 — change traces to an approved plan
}

CONFIG The audit record — your Type 2 evidence unit

Type 2 lives or dies on the operating log. Emit one append-only record per gate event, attributable to a single identity, hash-chained, shipped to an immutable store. This is the artifact the auditor samples.

audit-event.json — one per gated action → SIEM (append-only)PATTERN · audit evidence record · stable
{
  "ts": "2026-06-15T14:03:11Z",
  "actor": { "identity": "agent://impl-7f3a", "scope": "task-4821", "human_owner": "u/eng-114" },
  "event": "pr.merged", "pr": 9182, "change_ref": "plan/exec-4821",
  "controls": { "reviewer": "u/eng-220", "sast": "passed", "cve_high": 0, "ci": "green" },
  "tool_calls": 37, "prev_hash": "e3b0c4…", "hash": "a17fde…"   // hash-chained = tamper-evident
}

CONFIG Least-privilege MCP for stage 4 (CC6)

mcp.json — disabled-by-default, per-task enablePRIMARY · MCP server config schema · stable
{
  "servers": {
    "github": { "command": "npx", "args": ["-y", "@modelcontextprotocol/server-github"],
                "env": { "GITHUB_TOKEN": "${scoped_repo_token}" } },
    "prod-db": { "command": "uvx", "args": ["db-mcp@1.4.2"],
                 "disabled": true, "comment": "never enabled in the agent sandbox" }
  }
}
Gateway pattern — the single policy-enforcement point

The MCP spec enforces neither authentication nor authorization, so a gateway is the single policy-enforcement plane for CC6: a per-consumer virtual key with an explicit tool allowlist, deny-by-default at both advertisement and execution, per-user OAuth 2.1, and an immutable per-call audit log to your SIEM. Documented gateway capabilities are vendor-reported; start in log-only mode, then enforce. Curate tightly — too many tools degrade the agent and slow the harness start.

Type 2 readiness — what you must be able to produce on demand

For any change sampled from the audit window: it traces to an approved plan; it was reviewed by someone other than its author; SAST and CVE scans ran and passed; the acting identity was task-scoped, not shared; and the whole chain sits in a complete, tamper-evident log. Hold those five for 100% of changes across 3–12 months and the agentic pipeline isn't a compliance risk — it's the strongest change-management evidence you've ever had.

09 · Enabler · the deterministic scaffolding

Harness engineering — the "how" under stages 1–4 & 8–12

A coding agent is a model wrapped in a system: Agent = Model + Harness, where the harness is everything except the model. It is where almost all of your engineering leverage lives, and it is what makes the front and back of the pipeline reliable. The operating heuristic of the field:

"Every time the agent makes a mistake, don't just hope it does better next time. Engineer the environment so it can't make that specific mistake the same way again."
— Mitchell Hashimoto, on harness engineering (Feb 2026)

Pictured as an exploded view, the agent is a small model core wrapped in concentric, mostly-deterministic rings — and every ring is something you author and version. Each one also maps onto the pipeline: context & tools feed stages 1–4, the sandbox is stage 4, the sensors are stages 5–6b, and orchestration & hooks span the whole engine.

The seven harness components — and the stage each one serves

Harness needMechanism → which stage it makes reliable
Repo as system of record"What it can't see doesn't exist."
Versioned docs/ + lean AGENTS.md → stages 1, 3, 11
Machine-enforced architecturerules as code, not as hope
Layer linters + structural tests, set to error → stage 3 (and every implement run)
Disposable sandboxescontained blast radius
Per-agent microVM / dev-container, no prod/egress → stage 4
Curated, minimal toolstoo many degrade the agent
Small default MCP subset behind a gateway → stages 4, 12
Feedback sensorssignal after each action
Computational + inferential sensors → stages 5, 6, 6b
Agent-legible telemetryso agents can reason over prod
OTel + LogQL/PromQL → stages 9, 10
Scheduled drift / GC agentsentropy is constant
Background refactor & doc-gardening passes → stage 11

Feedforward guides vs feedback sensors

The clearest current framing (Thoughtworks, Apr–May 2026) splits the harness into what you give the agent before it acts and what you measure after. The "after" signals divide into two kinds with very different economics — and they map directly onto pipeline stages.

10 · Enabler · the bounded feedback loops

Loop engineering — the "how" under stages 4–7 & 10

Loop engineering is designing the agent's iteration cycles so they converge on something verified and cannot run forever. The inner loop (§05) is one instance; the most reliable inferential sensor in the field is a fresh-context evaluator paired with a default-FAIL contract, so an agent can't claim a success it hasn't observed.

Planner
Under-specify on purpose

Sets goals + acceptance criteria, leaves implementation open — early over-specification cascades downstream.

Generator
Build against the contract

Writes code + tests; gathers evidence (screenshots, logs, DB state) by driving the app.

Evaluator · fresh context
PASS / NEEDS_WORK

No Write/Edit tools; judges the diff + evidence it never built. On NEEDS_WORK its findings become the next session's prompt.

CONFIG Fresh-context evaluator + default-FAIL (stage 6)

.claude/agents/evaluator.mdPRIMARY · anthropics/cwc-long-running-agents · in-window
---
name: evaluator
description: Skeptical reviewer. Reads the diff and the builder's evidence,
  returns PASS or NEEDS_WORK with specific findings. No Write/Edit tools.
tools: Read, Glob, Grep, Bash
---
You did NOT build this change. Begin your reply with the bare word
PASS or NEEDS_WORK on its own line. Judge ONLY against the spec and the
evidence that was actually Read.
test-results.json + verify-gate.shPRIMARY · anthropics/cwc-long-running-agents · in-window
// every acceptance criterion starts FALSE
{ "feature-1": { "passes": false }, "feature-2": { "passes": false } }

# PreToolUse verify-gate.sh blocks writing "passes": true until the agent
# has Read real evidence (a screenshot / console log) this session:
{ "decision": "block",
  "reason": "No evidence Read this session. Open the evidence file first, then retry." }

CONFIG Repeat-until-green, with a hard cap (stages 4–7)

ralph + /goalPRIMARY · anthropics/claude-code · in-window
# first-party plugin: loop until a completion promise or the cap
/plugin install ralph-wiggum@claude-plugins-official
/ralph-loop "<prompt>" --max-iterations N --completion-promise "DONE"

# built-in & version-stable — prefer this for "keep working until correct":
/goal all tests in test/auth pass and lint is clean

The discipline that ages well: re-audit the harness every model release

A harness encodes assumptions about what the model can't do; those assumptions go stale as models improve. On every major upgrade, ask "which components exist only to compensate for now-fixed limitations?" and delete them — the best harnesses get simpler over time.

11 · The orchestration layer · running the fleet

When stage 4 becomes many agents at once

The orchestration layer in the pipeline is a multi-agent workspace manager: it isolates each agent (worktree or container), tracks work that survives restarts, runs watchdogs, and merges through a verifying queue. It is the control plane for stages 4–7 at scale.

ToolTypeIsolationCoordination modelLicenseBest for
GitHub Agent HQcontrol planecloud + worktree"mission control" to assign/steer/track many agents in GitHub/VS CodecommercialGoverned multi-agent inside GitHub (rolling out through 2026)
Conductorlocal orchestratorworktree per taskhuman-in-loop dashboard; multi-model comparemacOS · free (BYOK)Mac devs wanting a clean dashboard beside the editor
Composio Agent Orchestratorautonomousworktree/branch/PReach agent fixes its own CI + review commentscommercialThe "orchestrator-shaped" autonomous step
Gastownunattended townworktree (git-backed)coordinator (Mayor) + watchdogs + bisecting merge queueOSS (MIT)Self-hosted autonomous fleets, 20–50 agents
Factoryenterprisecloud "Droid Computers"coordinator → specialized droids; MissionscommercialEnterprise, ticket-to-PR, multi-day Missions
Devin (Cognition)enterprisecloud + localagent-first; Agent Command CentercommercialEnterprise autonomy with an agent control center
Sculptor (Imbue)containersDocker per agentparallel agents + Pairing Mode (IDE sync)free (BYO LLM)When isolation & safety matter most
The ceiling every orchestrator hits — and why stage 2 matters most

Isolation buys parallelism, not coordination. The moment two agents must modify overlapping code, worktree/container isolation doesn't save you — you get several drift-parallel solutions and merge review becomes the bottleneck. The genuinely scarce skill is non-overlapping task decomposition (stage 2). No tool removes that responsibility; merge queues and coordinator agents only manage its consequences.

CONFIG Gastown — agent presets + a concurrency cap

gt config · schedulerPRIMARY · gastownhall/gastown · in-window (verify field names on main)
# define agent aliases (any CLI runtime) + a default
gt config agent set codex-low   "codex --thinking low"
gt config agent set claude-haiku "claude --model haiku --dangerously-skip-permissions"
gt config default-agent codex-low

# cap concurrent workers so you don't exhaust API rate limits
gt config set scheduler.max_polecats 5   # >0 enables deferred dispatch
gt sling gt-abc gastown                   # auto-defers when a cap is set

Read the open ones as a blueprint. Even if you never run Gastown, its architecture is the clearest open catalogue of what an unattended orchestrator needs: persistent work state, a coordinator, isolated worker storage, a verifying (bisecting) merge queue, health watchdogs, severity-routed escalation, capacity governance, and OpenTelemetry. Evaluate every commercial platform against that checklist.

12 · The stack · per-stage picks & current models

A concrete stack for the pipeline

One defensible pick and a strong alternative for each stage. Tool names are representative of the field, not endorsements — choose against your own constraints and the recency caveats in §15.

StagePrimary pickStrong alternative
1 SpecifyGitHub Spec KitKiro (IDE) · Tessl (spec-as-source)
2 PlanSpec Kit /tasks + exec-plans in repoClaude Code plan mode
3 DesignADRs + layer linters + structural testsSourcegraph / CodeScene context
4 ImplementClaude Code (Opus 4.8)Codex (GPT-5.5) · Cursor · Devin · Droid
4 SandboxWarm devbox pool / SculptorE2B · Daytona · Modal
4 ToolsMCP via governed gatewaycurated per-task tool subsets
5 TestAgent-gen suites + coverage lintsPlaywright/CDP · property tests
6b ReviewCodeRabbit (breadth) + Greptile (depth)Graphite Diamond · Claude Code Review
7 CIGitHub Actions + pre-push hooksGitLab CI · Depot
8 DeployArgo Rollouts + feature flagsFlagger · LaunchDarkly
9 ObserveOpenTelemetry + GrafanaDatadog · New Relic
10 SREincident.io AI SRERootly · Resolve AI
11 FleetGastown (OSS) / Factory (enterprise)Agent HQ · Conductor · Sculptor
12 SecuritySnyk + Semgrep + CodeQL + MCP gatewaySonarQube · Checkmarx

Stage-4 models — current lineup (vendor-reported, harness-sensitive)

ModelReleasedVendor headlinePrice /M (in/out)
Claude Opus 4.72026-04-1687.6% SWE-bench Verified$5 / $25
Claude Opus 4.82026-05-2888.6% SWE-bench Verified$5 / $25
Claude Fable 5 (GA)2026-06-0995.0% SWE-bench Verified$10 / $50
OpenAI GPT-5.52026-04-2382.7% Terminal-Bench 2.0 · powers Codex$5 / $30 †
Cursor Composer 2.52026-05-1879.8% SWE-bench Multilingual$0.50 / $2.50
Gemini 3.5 Flash2026-05-19powers Antigravity 2.0 agent harnessn/a

† GPT-5.5 price via secondary source — unconfirmed. SWE-bench Verified is near saturation and harness-sensitive: the same model scores differently under different harnesses, which is the whole reason §09–10 matter more than this table. Don't select a model on a leaderboard — evaluate it inside your harness on your own task set.

13 · Measurement & guardrails · prove value, not activity

Instrument the workflow, not the hype

Measure the pipeline's effect on outcomes you already track, plus the agentic signals that tell you the fleet is healthy. Resist vanity counters.

Layer 1 · DORA, reframed

Outcome metrics

  • Change failure rate — the most meaningful signal for AI code
  • MTTR — the most distortion-resistant
  • Deploy frequency & lead time (distort when much code is AI-gen)
Layer 2 · AI attribution

Is the AI code good?

  • AI code share & adoption
  • Code durability — does it survive or get reverted?
  • Time-in-review (quantifies the new bottleneck)
Layer 3 · agentic ops

Is the fleet healthy?

  • Task success rate & review-intervention rate
  • CI pass-on-first-try; retries-to-merge; escalation rate
  • Cost-per-task; SRE autonomous-resolution rate
Layer 4 · safety

Are we shipping risk?

  • AI-introduced vulnerability / logic-error rate
  • SAST findings per PR
  • % of incidents traceable to AI-authored changes
Anti-metric · do not reward this

"Tokenmaxxing" — treating raw token spend or lines generated as success — is a vanity metric flagged by DORA in 2026. It tells you how much AI is used, never whether the org is more effective. Tie every stage's metric back to an outcome.

14 · Adoption · sequence the build

Stand the workflow up in stages, with hard gates

Don't wire the whole pipeline at once. Build the harness floor before the fleet; orchestration on a weak harness amplifies dysfunction. Each step has an exit criterion mapped to the workflow.

0

Baseline

WEEK 1
  • Instrument cost/merged PR, time-to-merge, review velocity, defect-escape, compute/dev
  • The numbers that gate everything after
EXIT: you can attribute outcomes to harness vs model vs human
1

Harness floor

WEEKS 1–4
  • Repo as system of record; lean AGENTS.md (stages 1, 3)
  • Top-3 failures → error-level lints (stage 3)
  • Hooks + disposable sandboxes (stage 4)
EXIT: review time & defect-escape improve
2

Bound the loops

WEEKS 4–8
  • Inner loop with hard caps (stages 4–7)
  • Fresh-context evaluator + default-FAIL (stage 6)
  • AI review + SAST as merge gates (6b, 12)
EXIT: agents self-correct; no runaway loops
3

Close the loop & scale

WEEKS 8+
  • Spec tool by need (stage 1); progressive delivery (stage 8)
  • Agentic SRE feeding [10]→[4]; doc-gardening (stages 10–11)
  • Add an orchestrator (stage 11); audit the harness each model release
EXIT: the harness gets simpler over time, not just bigger
15 · Recency & sourcing · the discipline, shown

What's in window, what's quarantined

This manual applies a hard 60-day cut (≥ 2026-04-01) for any claim, statistic, benchmark, version, or "what a company does today," and uses primary engineering sources only. Foundational harness/loop posts that predate the window are used for stable architecture, never for time-sensitive metrics.

IN WINDOW — ≥ 2026-04-01 QUARANTINED — stable facts only, no live metrics VENDOR-REPORTED — self-reported, unaudited
SourceDateWindowUsed for
Anthropic — Scaling Managed Agents2026-04-08INHarness-decoupling thesis
anthropics/cwc-long-running-agents (repo)2026 · CwCINEvaluator, verify-gate, default-FAIL configs (§10)
anthropics/claude-code — ralph-wiggum / /goal2026INLoop primitives (§10)
Thoughtworks/Fowler — harness eng. + sensors2026-04-02 / 05-13INGuides/sensors taxonomy (§09)
Anthropic — Opus 4.7 / 4.8 launches + system card2026-04-16 / 05-28INModel facts (vendor)
OpenAI — GPT-5.5 launch · Codex docs2026-04-23+INModel + Codex facts (vendor)
Cursor / Google — Composer 2.5 · Antigravity 2.02026-05-18 / 19INModel + product facts (vendor)
github/spec-kit · kiro.dev · tessl.io2026 activeINSpec Kit / Kiro / Tessl roles + configs (§04)
gastownhall/gastown · gastown.dev2026 activeINOrchestrator config (§11)
imbue.com (Sculptor) · e2b.dev · Argo Rollouts2026 activeINSandbox / deploy configs (§05,§07)
Claude Code hooks · code.claude.comstable schemaINHook config (§05)
OpenAI — "Harness engineering"2026-02-11QUARANTINEDArchitecture only (AGENTS.md, layering) — NOT live metrics
Anthropic — "Harness Design for Long-Running Dev"2026-03-24QUARANTINEDPlanner/generator/evaluator shape only
Stripe — "Minions"2026-02QUARANTINEDBlueprints, 2-CI-cap, devboxes, Toolshed — NOT current PR/week
arXiv — "Securing the MCP" 2511.209202025-11QUARANTINEDStable controls taxonomy only (§08)
MCP-gateway vendor pages · sandbox comparisons2026INVendor-reported — capability claims flagged
Gaps & exclusions (stated plainly)

The earlier draft's December-2025 AI-code-defect study was dropped (pre-window); §06 states the principle without it. The headline throughput numbers — OpenAI's "1M lines / 3.5 PRs per engineer per day" and Stripe's "1,300+ PRs/week" — are pre-window and are deliberately not presented as current; only Stripe's stable architecture (two-CI cap, disposable devboxes, curated Toolshed) is used. Listicle/aggregator/vendor-comparison sites were not cited for any claim — only used to locate primaries, and flagged vendor-reported where no primary existed. Fast-moving repo field names (Gastown especially) should be verified against main before copy-paste.

16 · Appendix · hand this to your auditor or vCISO

The SOC 2 evidence runbook

A detachable one-pager that operationalizes §08. For each technical control the pipeline enforces, it names the criterion, the exact evidence artifact, where it's stored, who owns it, and how often it's collected. Type 1 wants the design artifacts (configuration); Type 2 wants the operating records across the audit window.

Control areaTSCEvidence artifact (what you hand over)System of record (where it lives)Owner · cadence
Change authorizationCC8.1Branch ruleset (required review + checks); PR approval with reviewer ≠ authorGitHub rulesets + PR / audit logPlatform · every merge
Plan / intent approvalCC3, CC8.1Approved exec-plan linked to the changedocs/exec-plans/Eng lead · per change
Secure-dev scanningCC7.1, CC4.1SAST + dependency-CVE results and triage, per PRCI logs + Semgrep / SnykAppSec · per PR
Logical access — agent identityCC6.1–6.3Gateway RBAC/allowlist policy; per-agent identity registry; access-review sign-offIdP + gateway config + SIEMSecurity · logs continuous, review quarterly
Boundary protectionCC6.6–6.7Sandbox isolation & egress (network policy)IaC repo + cloud configPlatform · config + runtime logs
Supply-chain integrityCC6.8, CC9Dependency pins, private registry, SBOM per buildRegistry + artifact storeAppSec · per build
Deployment controlCC8.1, A1.2Progressive-rollout policy; deploy + rollback recordsArgo / CD logs + release recordsSRE · per release
Monitoring & detectionCC7.2–7.3SLO/alert config; alert + triage recordsObservability platformSRE · continuous
Incident responseCC7.4–7.5IR runbook; incident timeline, RCA, remediation PRincident.io / Rootly + repoOn-call · per incident
Audit loggingCC2.1, CC4.1Append-only, hash-chained audit-event streamSIEM (WORM / immutable)Security · continuous
System docsCC1, CC2.1Structured docs/ + freshness CIrepo docs/Eng · continuous

One sampled change, one query

The point of wiring evidence into the gates: an auditor picks any merged change from the period, and a single query reconstructs its entire control chain from the hash-chained audit record plus the artifacts it points to.

auditor sampling walk — PR #9182PATTERN · reconstruct the control chain · illustrative
# 1) one query returns the gate record for the sampled change
$ aud get --event pr.merged --pr 9182
  actor      agent://impl-7f3a   (scope task-4821 · owner u/eng-114)
  reviewer   u/eng-220     sast passed     cve_high 0     ci green
  change_ref plan/exec-4821       hash a17fde… (chain verified)

# 2) cross-check the artifacts the record points to
GitHub PR 9182 ......... approver ≠ author ✓          (CC8.1)
CI run for the merge ... required checks present ✓    (CC7.1)
docs/exec-plans/4821 ... approved before work began ✓ (CC3)
SIEM chain 0…9182 ...... unbroken, append-only ✓      (CC4.1)

One record plus the artifacts it references closes the loop for that change. Repeat across the auditor's sample; if all five Type 2 conditions from §08 hold for every sampled change, the operating-effectiveness test passes.

Scope & boundaries — read before handing this over

This runbook covers the technical, automated controls the pipeline enforces — CC6 (access), CC7 (operations & monitoring), CC8 (change management), CC9 (risk mitigation), and A1 (availability) where in scope. Org-level controls live elsewhere: CC1 governance, CC2 role communication, HR onboarding/offboarding, vendor management, the formal risk-assessment cadence, and physical security belong in your GRC platform, not here. Criteria references follow the AICPA 2017 Trust Services Criteria (rev. 2022). This is an engineering implementation guide, not legal or audit advice — your auditor sets the final scope and sampling.