Adopting AI Code Review

The most common adoption failure is a big-bang org-wide rollout with a hand-written prompt. This page describes the opposite: six steps, each a small, reversible config change, each with a measurable checkpoint. It is a distillation of the full practices document, which carries the evidence behind every claim here.

The staged path

1. Pilot one repo on tool defaults — zero config

VCR ships with a versioned defaults file: five review lenses (security, correctness, test-quality, and performance on by default; maintainability opt-in), 16 out-of-the-box pattern rules, a findings cap, and confidence thresholds. A repo with no .visdom.yaml gets exactly these defaults. The point is to observe baseline behavior on your code before you tune anything — tuning against a baseline you never measured is guesswork.

2. Make opt-in explicit

The first line of repo config is consent, not customization. The inverse also works: enabled: false skips the entire review before any layer instantiates — zero AI calls, zero findings, immediate exit. Teams that know they can turn the tool off in one line resist it less.

3. Treat review policy as code

.visdom.yaml lives in the repo, is versioned, and goes through code review like any other change. The merge semantics fit in one sentence: tool defaults merge with the repo file — the repo wins on scalars, rule lists concatenate, and inherited rules can be disabled by id. A platform team owns the defaults; each repo owns its deltas; git blame on the config answers "who decided this lens runs here, and when."

# .visdom.yaml
enabled: true           # consent first — set false to skip entirely, zero AI calls
lenses:                 # policy as code
  maintainability:
    enabled: true
    min_severity: medium
rules:
  - .visdom/rules/org.rules.yaml
standards:
  sources: ["docs/standards/*.md"]

4. Add org rules — pattern first, then llm

Both rule types share one YAML envelope, discriminated by type. Start with a pattern rule: it is mechanical, free, and has zero variance — the same input produces the same finding every run. Only after the team trusts that loop, add an llm rule for a policy that needs judgment — and give it explicit success criteria, failure criteria, and paired good/bad examples. An llm rule without failure criteria produces run-to-run variance. Scope org rules to org-specific policy and leave generic code quality to the built-in lenses; a rule that restates a lens produces doubled findings.

5. Inject standards you already have

standards.sources takes globs over existing repo documents — no forced duplication into a vendor format. Matched documents are injected into deep-review prompts, and findings cite them by path: in the controlled scenario, a broad-exception-catch finding quoted the org's own docs/standards/error-handling.md as the violated rule.

6. Watch the metrics before widening

The numbers to watch: findings per PR against a baseline, the noise rate developers actually report, and cost per PR. On a bulk run over 38 real PRs, the average was $0.078 per PR at roughly 22 seconds each. If the pilot's finding volume or noise rate looks worse than expected, fix the config on one repo — do not export the problem to fifty.

Controlling noise

The doctrine, stated once and applied everywhere: better to miss a LOW finding than erode trust with a false positive. Developer trust behaves like a threshold function — past a certain noise rate, developers stop reading the comments and eventually disable the tool. Noise compounds; trust does not recover. The constraint is expressed through four knobs, all in config rather than code:

The low-confidence bucket exists because deleting uncertain findings hides information while presenting them as facts destroys trust; labeling them "requires verification" is the honest middle. And tuning is not free in either direction — raising a confidence threshold cuts noise but costs recall, a trade you have to measure to see. Precision-over-recall means choosing your point on the curve deliberately, not pretending the curve isn't there.

What not to do

Four anti-patterns, each measured the expensive way during VCR's own iteration history:

• Longer prompts reduce precision. Expanding the deep-review prompt from 6 to 9 pattern categories produced fewer hits and more noise — every category you add is hallucination surface you now own.
• More context is not better context. Adding an import graph to the prompts produced zero extra hits and more speculative findings about modules the model could not see — a list of names without contents is an invitation to invent.
• A judge that never finds noise is flattering you. A keyword-based judge reported zero noise where an LLM advisor judge found 37 vague or hallucinated findings — treat a too-clean metric as a bug in the judge until proven otherwise.
• Free-form LLM output is not a join key. Deduplicating findings on a model-generated category field silently collapsed three distinct findings into one — key joins on fields your system assigns, never on fields a model generates.

Go deeper

• The full practices document: every claim above with its evidence trail, the measurement-honesty practices, and the layering economics — docs/practices.md.
• Configuration reference: the complete .visdom.yaml schema, merge semantics, and rule envelope — Configuration.
• Platform engineers guide: pilot operations and configuration decisions — Platform engineers.
• See it run: the real-repository example with artifacts — Demo.