Review theater (AI reviewing AI)
The common claim
AI reviewing AI’s PR rubber-stamps the same blind spots — security, tenancy, money paths, race conditions.
What I actually do
Builder agent is not reviewer agent. Different context, different goal, different mandate.
I run multi-agent review with focused lenses — security and the rest. Reviewers also get the architecture spec so they can check fidelity, not just style.
In practice I do not see rubber stamps. Reviewers nitpick and find real issues. Far from theater.
Testing is part of the loop — not “AI writes, someone glances, merge.” Deeper test discussion is its own item later in this series.
Cross-vendor review might help reduce correlated blindness. I don’t do that in practice today — different agents and different mandates are the split I actually run.
Where #2 still bites
- Correlated blind spots on novel business logic, even with multiple agents.
- Vague or outdated specs turn “validated against architecture” into theater. Spec quality is the bottleneck.
- Severity calibration — agents love nits and can underweight “this can lose money.”
- If a human never samples hard paths, you’ve moved the rubber stamp to the merge button.
Hard-path rule: money, auth, migrations, public API, PII — a person reads those for real. “Lucky so far” is not a control. Make the checklist explicit.
Drafted with AI. Reviewed and edited by Ed Henderson.