Review AI-Generated Code by Blast Radius, Not Pull Request Size

An AI coding agent opens a 1,900-line pull request for a new onboarding flow. It looks frightening. The feature is off by default, makes only an additive compatible schema change, can be enabled for one internal account, and can be disabled without a redeploy. The same week, a person opens a 14-line pull request that changes how webhook signatures are checked. It looks tidy. If it is wrong, forged requests can update every customer's billing state as soon as the deployment finishes.

Which change deserves the longer human review?

Pull request size is useful evidence about comprehension cost: how much material a reviewer must understand. It is weak evidence about failure cost: how many users, records, privileges, or business promises a defect can affect before the team detects and contains it. AI coding agents make that distinction harder to ignore because they can produce complete, cross-layer features faster than a person can read every generated line.

This guide is for nontechnical founders and small product teams that use AI app builders or coding agents on a real product. The core judgment is: keep changes understandable, but allocate review and rollout controls by blast radius rather than line count alone. You will leave with plain-language definitions, a reusable change-risk card, a worked launch scenario, review rules for three risk levels, a progressive rollout ladder, and a 60-minute drill.

This framework does not make large changes inherently safe, replace qualified security review, or excuse generated code that nobody understands. It applies when a team can test, observe, limit, and reverse a release. Changes involving regulated decisions, irreversible data loss, safety-critical systems, or material financial authority need stronger domain controls.

Y Build's merge evidence pack explains what must be proven about an exact revision. This guide addresses the next, narrower decision: where to spend review attention and how much production exposure that evidence has earned. For destructive agent capabilities, use the separate recovery-boundary framework.

What the Rootly case actually shows

Rootly supplied the immediate prompt for this article. In May 2026, the incident-management company published why it removed its strict small-PR rule. Its account says AI agents increasingly produced whole features—migration, model, service, tests, and interface—while splitting that work into an artificial stack made reviewers reconstruct dependencies across multiple tabs. Rootly replaced the line-count proxy with risk labels, required rollback plans, production validation, and progressive rollout behind feature flags.

The interesting lesson is not “large pull requests are good.” It is that a process metric had stopped answering the team's real question. A small diff once correlated with a small unit of human thought and an easier revert. Once an agent can generate a coherent feature across layers, line count still predicts reading effort but no longer reliably predicts how safely the feature reaches users.

Rootly also says its reviewers concentrate on context, shared boundaries, migrations, security-sensitive behavior, and rollout paths. That is a sensible direction, but it is one company's operating report, not universal proof. Its team has incident-response expertise and internal review automation that a two-person startup may not have. A founder should copy the question—“what breaks if this is wrong?”—not copy Rootly's exact tolerance for large changes.

There is also a useful counterweight. Google's SRE guidance says small, self-contained release artifacts make rollback cheaper and help changes move through an automated delivery system. Its canarying chapter does not argue for arbitrary giant batches. It argues for partial, time-limited exposure whose signals can be compared with a control. The combined position is stronger than either slogan:

1. Make the logical change as coherent and comprehensible as practical.
2. Make the exposure unit smaller than the potential failure whenever possible.
3. Spend review time according to consequence, uncertainty, and recovery—not generated line count by itself.

Define the terms before assigning a risk label

Teams often use “small,” “safe,” “reversible,” and “behind a flag” as reassuring adjectives. They need operational definitions.

These terms prevent a common category error: a change can be easy to read and dangerous to release, or hard to read and safely contained. The review process needs to see both axes.

Keep two axes: comprehension cost and failure cost

Use a two-axis decision instead of replacing “small PR” with a vague “risk-based” label.

The upper-left cell is routine: a small copy adjustment or isolated component fix with a clear test. The upper-right cell is where line-count rules fail most visibly: a one-line permission wildcard, a short payment-routing change, or a small migration that locks a critical table.

The lower-left cell is common with generated code. A new internal report may touch API types, a view, tests, and fixtures, creating a large diff while remaining read-only and available to one internal account. Do not wave it through. Reduce comprehension cost with an architecture note, generated-file separation, contract tests, screenshots where relevant, and a guided review map. Then keep exposure narrow.

The lower-right cell is not a “review harder” problem. A sweeping identity rewrite that reaches every tenant, changes stored data, lacks version-specific signals, and cannot be disabled independently should be redesigned. Splitting it into ten pull requests may make the text easier to read while leaving one simultaneous, irreversible production transition. Conversely, placing all code in one pull request does not make the system coherent if it bundles unrelated outcomes.

Comprehension remains a safety control. It simply is not the same control as containment.

Complete the change-risk card before reading the diff

The person who requested the change should complete this card. For AI-generated work, do not ask the agent to invent the business motivation or acceptable loss. The agent can collect evidence, but a human owner must state why the change exists and what consequence the company accepts.

Score each dimension from 0 to 3. Use the highest credible consequence, not the best-case demo.

Then record the evidence, not just the number:

change: "Store company size during onboarding"
owner: "Mina"
business_reason: "Choose the correct onboarding path"
generated_scope: "form, API route, additive column, tests, analytics event"
scores:
  user_reach: 1
  authority_and_data: 2
  state_transition: 1
  reversibility: 1
  detection: 1
  coupling: 1
exposure_unit: "staff accounts, then 10 consenting new signups"
success_signal: "completion rate and valid company_size writes by release cohort"
stop_signal: "schema errors, >2 percentage-point completion drop, or wrong tenant write"
stop_action: "disable flag; preserve additive column; route to old onboarding"
recovery_owner: "Mina"
evidence_links: ["test run", "dashboard", "migration plan", "rollback drill"]
unknowns: ["one CRM sync consumer has not been exercised"]

Use simple decision rules:

• Low risk: no dimension scores 3; at most one scores 2; the change has a deterministic acceptance test and immediate disable path.
• Medium risk: two or more dimensions score 2, or one dimension scores 3 but exposure can be constrained to internal or named pilot users.
• High risk: two dimensions score 3; any unknown destructive transition; any broad change to authorization, secrets, payments, deletion, tenant isolation, or public communication.

Compare a large feature with a tiny security change

Consider a small B2B product using an AI coding agent to ship onboarding improvements.

Change A: the 1,900-line onboarding feature

The agent adds a company-size question, an additive nullable database column, a validation schema, a new onboarding branch, analytics, and tests. The feature is disabled by default. Staff accounts can opt in. Existing users never see it. The old route remains intact. The release dashboard separates candidate and control cohorts.

The diff has high comprehension cost. Reviewers should not pretend to understand it through a quick skim. They can make it reviewable by separating mechanical snapshots, providing a map of the six changed boundaries, running contract and end-to-end tests, inspecting the migration and tenant key manually, and reviewing the old-route fallback.

Its initial failure cost can still be low. Only staff are exposed. The migration is additive. The old application tolerates the new nullable column. The flag disables the behavior. A failed analytics event does not block onboarding. If the staff cohort passes, ten consenting new signups receive the feature while the team watches completion and write-validity signals.

Change A should not be approved because it is flagged. It should be approved only when the flag is verified in both happy and error paths, the migration is compatible with the old version, the observability distinguishes cohorts, and someone has actually performed the stop action.

Change B: the 14-line webhook verification edit

The second pull request changes a library call and accepts a new header format from the billing provider. It has one unit test. It is easy to read. It affects the endpoint that converts signed payment events into subscription state for every tenant.

This change scores high on authority, reach, coupling, and detection uncertainty. A defect can reject legitimate renewals or accept forged input. A global deployment exposes everyone immediately. Reverting the code may not repair subscription records already changed by accepted events.

Change B deserves independent review of the provider's signed-message contract, recorded fixtures from each supported event version, replay and timestamp tests, fail-closed behavior, tenant mapping checks, version-specific alerts, and a reconciliation query. If traffic cannot be safely mirrored or narrowed, the team should schedule a staffed release window and define a fast endpoint-disable or queueing mode.

The comparison produces the rule founders need: readability controls whether a change is reviewable; blast radius controls how much evidence, independence, and rollout protection it needs. Fourteen lines can demand more launch work than 1,900.

Assign review work by risk tier

The risk card should change what people do, not merely add a label.

Low risk: verify intent and automatic evidence

One owner can review the change. Confirm that the stated outcome matches the diff, tests cover the acceptance rule, unrelated files did not change, and the disable or revert path is real. A normal deployment is reasonable when user reach remains narrow and monitoring is already established.

Medium risk: inspect boundaries and prove containment

Require a second person who did not generate the change to review the highest-scoring dimensions. Inspect migrations, authorization checks, tenant filters, external calls, queues, scheduled jobs, configuration, and feature-flag behavior. Run the rollback or flag-disable action in staging. Start with an internal or named cohort and pause long enough for the chosen signals to become meaningful.

GitHub environments can enforce required reviewers, prevent self-review, restrict deployment branches, hold environment secrets until approval, and connect custom protection rules to observability or change-management systems, as described in the official deployments and environments documentation. Those mechanisms provide separation only if the initiator cannot bypass them and the approver sees the risk card and evidence.

High risk: redesign first, then require independent authority

Do not send an unbounded high-risk change into a heroic line-by-line review. First reduce reach, privilege, state coupling, or irreversibility. Separate destructive cleanup from additive migration. Add a read-only or shadow mode. Queue external effects. Introduce a feature flag whose off path is tested. Create a version-specific signal. If consequence remains material, require qualified security, data, legal, financial, or operational review as appropriate.

The AWS Well-Architected Framework recommends safe deployment strategies including feature flags, one-box or canary releases, immutable releases, traffic splitting, and blue/green deployments. The useful founder takeaway is not to adopt every technique. It is to match the release control to the failure mechanism.

Move the final gate from merge to controlled exposure

Merge review asks whether the proposal is acceptable to place in the codebase. Rollout asks whether the new behavior has earned more exposure. Treat them as separate decisions.

A practical ladder for an ordinary web feature is:

1. Build-only: compile, static checks, unit tests, dependency checks, and generated-file review.
2. Synthetic: run end-to-end fixtures with no production authority or customer data.
3. Internal: expose named staff accounts; verify the flag's on and off paths.
4. Named pilot: expose a few consenting customers whose expected workflows are understood.
5. Percentage cohort: increase exposure only when metrics can identify the candidate separately.
6. Broad release: move to 100% after the observation window covers the important delayed effects.
7. Cleanup: remove temporary compatibility paths and stale flags in a separate, reviewed change.

Google's SRE canary guidance illustrates this measurement problem: a small candidate cohort can have a high error rate while barely moving the service-wide average. Candidate and control signals must be separable. It also notes that exposure limits the amount of error budget placed at risk; canarying does not prove correctness, but it can reduce the cost of learning.

Cloudflare Workers' gradual deployments documentation provides a current implementation example: traffic can be split between versions, monitored by version, and rolled back. It also documents version skew—successive requests or service bindings can reach different versions—and offers version affinity and overrides. Progressive delivery creates its own failure modes, so the rollout plan must test mixed-version contracts rather than assume percentages equal isolation.

Treat flags and rollbacks as claims to test

“Behind a flag” is not equivalent to “safe.” A flag reduces exposure only when all consequential paths check it, the off state preserves the old behavior, operators can change it during an incident, and the change has no unavoidable side effect before the check.

Feature flags are especially weak around one-way state. If the new code writes a format the old code cannot read, switching the interface off does not restore compatibility. If a migration drops a column, the flag cannot recreate it. If the feature sends email or submits a payment, off prevents future actions but not past ones.

Use an expand-and-contract pattern for shared schemas: add a compatible new structure, deploy code that can tolerate old and new states, backfill with observability, switch reads after verification, and remove the old structure later. Prisma's official expand-and-contract migration guide shows the pattern and recommends production-copy testing and backups. The exact commands vary by database; the principle is to separate a reversible introduction from destructive cleanup.

Cloudflare's rollback documentation makes the boundary explicit for its platform: code can return to a prior version, but connected resources are not rolled back, and data-structure changes can make old code fail. Every rollback claim in a risk card should therefore answer two questions:

• What code or configuration returns to the known-good version?
• What durable effects remain and how are they reconciled, restored, or accepted?

Measure delivery outcomes instead of rewarding small diffs

Teams optimize what leaders celebrate. If the founder praises pull request count and small diffs, people and agents will split work until the metric looks healthy. That can create dependent stacks, repeated review setup, and false confidence without changing production consequence.

Track a small set of outcomes instead:

• Percentage of releases requiring immediate intervention.
• Time from a stop signal to restored safe service.
• Percentage of unplanned deployments used to repair a prior release.
• Percentage of medium- and high-risk changes with tested stop actions.
• Percentage of rollouts whose candidate signals are separable from control.
• Number of incidents where the actual blast radius exceeded the risk card.

Keep diff size as a diagnostic measure. A growing change may indicate unclear scope, missing boundaries, or review overload. Do not turn it into a safety score. Ask whether large generated changes correlate with more missed defects in your own release history, and adjust the review map accordingly.

Avoid six failure modes that produce false confidence

“The AI reviewer gave it five out of five”

An automated reviewer can find patterns and summarize a diff. It may share the generator's blind spots, lack business context, or miss an undocumented consumer. Treat it as evidence collection, not independent acceptance.

“CI is green”

CI proves the checks that exist passed in the tested environment. It does not prove the tests represent customer behavior, production permissions, mixed versions, real data shape, vendor failure, or delayed jobs. Tie each high risk-card score to a specific piece of evidence.

“We split the big PR into five”

Splitting helps only when each piece is coherent, compatible, independently testable, and safe to deploy in sequence. A stack that can be understood only as a whole may multiply review overhead while preserving one large release event.

“The change is only configuration”

Configuration can expand public access, route all traffic, disable verification, change retention, or expose a secret. Judge effect, not file extension or line count.

“We can always revert”

Revert is a source-control operation. Recovery may include data repair, customer communication, credential revocation, payment reconciliation, cache purge, or forward fixes. Name the residual effects.

“The canary passed, so the release is safe”

A canary may exclude the rare account type, delayed renewal, large dataset, region, or permission combination that triggers the defect. Progress only when the cohort and observation window exercise the mechanism at risk. Some failures require shadow traffic, synthetic probes, contract tests, or a dedicated migration rehearsal instead of percentage rollout.

Run a 60-minute blast-radius drill

Choose one pending AI-generated change in staging. Do not deliberately damage production or use customer data.

The drill passes when another person can explain the consequence, find the candidate signal, execute the stop action, and verify the remaining state. It fails if safety depends on the original agent session or on the author being available.

Know where the framework applies and where it stops

This method fits ordinary SaaS features, internal tools, content workflows, and AI-built web products where releases can be segmented and observed. A solo founder can use it with a spreadsheet, deployment log, feature flag, and one trusted reviewer. The point is disciplined judgment, not enterprise ceremony.

Do not use the matrix to justify broad autonomy in medical devices, industrial control, critical infrastructure, regulated eligibility, high-value financial movement, or other safety-critical systems. A low label from a general checklist cannot override legal duties, validated engineering processes, or qualified domain review.

Some products cannot canary by user percentage. A schema migration touches shared storage. A mobile binary remains installed for months. A privacy-policy change affects a legal promise. An encryption-key rotation may be global. In those cases, reduce blast radius through compatibility, shadow validation, staged resources, dual reads or writes, offline rehearsal, backups, or a separate approval boundary—not by pretending a 1% flag exists.

Small pull requests also remain valuable when the work has natural independent boundaries. A focused bug fix is easier to review and revert than a bundle of unrelated fixes. The goal is not to normalize unreadable output. It is to stop mistaking a neat diff for a safe production change.

Use the founder launch gate

Before approving the next AI-generated change, require clear answers:

• What user or business outcome justifies this change?
• Which parts are generated, mechanical, or moved, and which paths require human judgment?
• Is the logical change coherent and understandable without reconstructing an artificial stack?
• What is the maximum credible blast radius before detection and containment?
• Does the risk card score reach, authority, state, reversibility, detection, and coupling with evidence?
• Which short lines carry disproportionate consequence?
• Which large sections increase comprehension cost without increasing failure cost?
• Can the exposure begin with synthetic, internal, or named pilot users?
• Can candidate outcomes be distinguished from the control by version or cohort?
• What exact signal stops the rollout, who owns it, and how quickly can they act?
• Has the stop action been performed before customer exposure?
• What data, messages, payments, caches, or external effects remain after code rollback?
• Are mixed-version and delayed-job paths compatible?
• Does an independent reviewer inspect the highest-risk boundary rather than merely skim every line?
• Is the change held or redesigned when consequence is high and containment is weak?

Keep the code comprehensible. Make the exposure smaller. Put the deepest review where the product can hurt users, not where the diff happens to be longest.

References

• Rootly, Why we got rid of our small-PR rule, the first-party operating account that prompted this analysis.
• Google SRE Workbook, Canarying Releases, on partial deployment, control comparison, version-specific signals, rollback, and error-budget exposure.
• GitHub Docs, Deployments and Environments, on required reviewers, self-review prevention, branch restrictions, environment secrets, and custom deployment protection.
• AWS Well-Architected Framework, Employ Safe Deployment Strategies, on feature flags, canaries, traffic splitting, blue/green deployment, monitoring, and post-deployment tests.
• Cloudflare Workers Docs, Gradual Deployments, on traffic splitting, version observability, version skew, affinity, and overrides.
• Cloudflare Workers Docs, Rollbacks, on restoring code versions and the limits created by resource and data changes.
• Prisma Documentation, Expand-and-Contract Migrations, on compatible schema evolution, production-copy testing, backups, and migration monitoring.
• DORA, Software Delivery Performance Metrics, on deployment throughput, change fail rate, failed deployment recovery time, and deployment rework rate.
• Martin Fowler, Feature Toggles, a detailed taxonomy of flag lifetimes, decision points, canary cohorts, and operational tradeoffs.