Here is the prompt almost everyone starts with:
Act as a senior engineer and review this code for bugs
and security issues.
It returns something that looks excellent. Six or seven well-organised concerns, each phrased with confidence, each touching a real category of risk. Input validation. Error handling. Potential race conditions. Consider adding logging.
Then you go and check them, and most of them are not reachable in your codebase. The race condition is on a path that only ever has one writer. The validation concern is handled two frames up the stack. By the third review you are spending an hour to find nothing, and quietly stop running it.
This is the central problem with AI code review, and it is not a model capability problem. A frontier model can find genuinely subtle defects. The problem is that nothing in the prompt distinguishes a defect from a worry, so you get both, formatted identically, with equal confidence.
The rule that fixes it
One rule does most of the work:
No finding blocks without a concrete reachable failure.
Enforced as an output format on every review:
trigger → failure → cause (with file:line)
The reviewer must name the exact input or state sequence that reaches the problem, what observably goes wrong, and the line responsible. If it cannot produce that chain, the concern is advisory — worth reading once, never worth a day of investigation.
Compare the two forms:
| Not a finding | A finding |
|---|---|
| "This could be risky under concurrent load." | "When two refunds for the same customer are in flight and the first is abandoned, the attempt counter is never cleared, so the second refund starts at the escalated path and skips the safe route — retrier.py:88." |
The second one you can act on in ten minutes. The first costs you a day and usually ends in a shrug.
There is a stricter version of the rule that is worth adopting once you are comfortable: a blocking finding requires a failing test. If the reviewer cannot describe a test that fails before the fix and passes after it, the finding advises rather than blocks. This is aggressive, and it is the single most effective noise filter available.
Give the reviewer probes, not just a diff
The second-biggest lever, and the one people skip because it requires thinking before prompting.
A reviewer handed only a diff returns vibes — general observations, standard categories, a checklist rendered against your code. A reviewer handed named attack surfaces returns evidence, because it goes and traces the specific path you pointed at.
Defects hide in three places. Logic bugs inside a single function are the easy case; your tests and your own reading catch most of those. What survives to production lives here:
Seams — where components hand off
- Field X comes from an external response. Trace every type it can be, including null and empty, and what each does downstream.
- What does the caller assume about this return value that the new code no longer guarantees?
- Which exceptions can this path raise, and is any catch broad enough to hide a defect in the happy path?
- Does any test assert this write actually lands, or is the sink stubbed?
State — what persists between operations
- If step A succeeds and the process dies before step B, what does restart believe?
- This counter increments on failure — what resets it, and can it leak into an unrelated operation?
- Attempt 1 partially succeeded. Does attempt 2 do the remainder, or the whole thing again?
- Run this operation twice with identical input. Is the second run a no-op, and what makes it one?
Timing — what changed between the read and the write
- The snapshot is from T0 and the action fires at T2. What can change in between, and does the action re-derive or trust?
- Two actors reach this path at once — the automation and a manual operator action. What is the outcome?
- Is the iteration order deterministic? If it changed, would the outcome change?
Pick three to six that actually apply to your change. Resist listing ten — a reviewer with ten probes gives each one a sentence.
A prompt you can paste today
This runs four opposed reviewers in a single chat window. No tooling, no agent framework, works in any capable model.
You are running the four BLOCKING seats of a review board sequentially
over one change, returning all four verdicts in one report:
architecture-compliance, security, black-hat, red-team.
THE CHANGE: [paste diff, or describe precisely: files, before/after
behaviour]
CONTEXT: [2-4 sentences: why the change exists, what incident or
requirement motivated it, what the system does]
PROBE HARD (this list is where reviews are won):
1. [probe]
2. [probe]
3. [probe]
4. [probe]
DOCTRINE: a finding blocks ONLY with a concrete reachable failure —
state the exact inputs/state sequence and the defect line.
Plausible-but-unreachable concerns are advisory notes, clearly
separated.
Return: four titled verdict sections + a one-line chair summary.
The doctrine paragraph is not decoration. Remove it and the output degrades immediately back into a list of worries. It is the cheapest four lines in the prompt.
Why four reviewers rather than one
Because they hunt different things, and a single general reviewer averages across all of them badly.
| Seat | Attacks |
|---|---|
| Black Hat | The design's logic — hidden assumptions, quiet wrongness, second-order effects |
| Red Team | Its life in production — crash windows, restarts, concurrency, leaked state |
| Security | Real exploitable risk in this specific stack |
| Compliance | Silent deviation from the written architecture |
The Black Hat charter is the one to start with, and it is short enough to include in full:
You are the Black Hat on the [PRODUCT] review board (de Bono's black
hat: disciplined critical judgment, not cynicism). Your mandate: find
why this change is a mistake before it ships. Assume it is flawed and
prove it.
Review the change (start from the diff) and hunt for:
- HIDDEN ASSUMPTIONS. What must be true for this to work that isn't
guaranteed? (ordering, timezones, schema stability, single-writer
locks, memory headroom, clock skew, retry semantics.)
- FAILURE MODES. What breaks under scale, partial data, restart,
concurrency, or a late-starting dependency? [Add the specific
classes of bug your codebase has actually been bitten by.]
- CORRECTNESS. Look for the quiet wrongness: silent zero-fill hiding
missingness, a counter that survives a code path it shouldn't, an
estimate presented as a measurement.
- SECOND-ORDER EFFECTS. What downstream consumer does this quietly
break? What does it make harder to change later?
- THE WORST REALISTIC OUTCOME. State it plainly.
Be specific and adversarial; vague worry is not a finding. Every
finding needs a concrete trigger ("when X happens, Y breaks because Z").
Output: `Lens verdict:` BLOCK / CONDITIONAL / CLEAR, then findings
(severity, file:line, the trigger→failure→cause chain, and the fix or
guard required).
The bracketed line matters more than the rest. Replace it with the failure classes your system has actually been bitten by — go through your incident history and your postmortems. Ten minutes, once, and every review afterwards is sharper. That list is the difference between a generic reviewer and one that knows where your bodies are buried.
A worked example
A payments service adds a fast-void watchdog. When a card authorization must be voided, a background check now fires the void within seconds instead of waiting for the nightly batch cycle. On success it writes an audit row to the compliance journal.
Sixty lines. Tests pass. CI green. It reads as entirely reasonable.
Six probes went with the change. The one that mattered:
(6) Does the journal writer tolerate the lean row the watchdog passes?
Trace every field's type into the serializer.
Three of the four seats passed it. Compliance confirmed the placement reused the existing journal seam. Security found no new surface. Red Team probed the crash windows and double-void scenarios and documented why the batch cycle safely no-ops afterwards.
The Black Hat failed it, with one must-fix:
- Trigger
- Any successful watchdog void.
- Failure
- Compliance journal row silently dropped, on 100% of successful voids.
- Cause
- Audit row passes None for three numeric fields; serializer.py:142 casts unconditionally (float(None) → TypeError); swallowed by the broad except at watchdog.py:88.
The exact visibility the feature existed to provide would have shipped non-functional. Nothing crashes. Nothing logs at error level. The test suite is green because the test stubs the journal and never asserts the row lands. An auditor would have found it, months later, by asking why the journal had no watchdog rows.
One-line fix, merged the same day.
Worth noticing what did the work here. Not a cleverer model — the same model wrote the code. It was one seat whose entire job is to assume the change is broken, and a probe that told it to trace data types through the serializer rather than reason about the design.
Five ways this goes wrong
Confident theater
Beautiful verdicts, no finding that ever changes anything. Cause: personas without the doctrine rule. Fix: put the doctrine paragraph in every prompt and enforce trigger→failure→cause.
Death by advisory noise
Fifteen findings per review, so you stop reading them. Cause: accepting findings with no reachable failure sequence. Fix: reclassify anything without a trigger sequence as advisory and do not action it.
Vibes reviews
The reviewer tells you things you already knew. Cause: no probes. Fix: ten minutes, three to six probes, targeting seams, state and timing.
Abdication
"The AI reviewed it" appears in a pull request description. Cause: treating the reviewer as an authority rather than an adversary. Its job is to attack the change; your job is to judge the attack. A confident model has been wrong before and will be again.
Reviewing ideas instead of changes
Boards review diffs. "Review my architecture" produces a review of your architecture-shaped opinions. If there is no code yet, write the design down as a concrete proposal with named components and interfaces, and treat that document as the diff.
Start here
Take the last non-trivial change you shipped. Get the diff. Write three probes — one seam, one state, one timing. Paste the Black Hat charter, the diff, the probes, and the doctrine line. Then judge the attack yourself.
If it finds nothing, that is information about your probes, not about your code. Rewrite them to target state and timing rather than logic, and run it again.