AI Doubled the Pull Requests. Did It Improve the Product?

Linear has published a large new view of how AI is changing software teams. In its paid customer base, pull requests opened per workspace were 111% above a June 2024 baseline by June 2026. Workspaces with a coding agent connected went from 21 to 65 weekly pull requests over the same two-year span, while workspaces without one moved from 8 to 10. AI also authored just under half of the issues created in Linear.

Those are meaningful activity signals. They are not evidence that AI doubled product value. Linear says so directly: it counts pull requests opened, not merged, and an opened pull request says nothing about the value of the change. The agent-connected teams were already higher-output before coding agents existed, so the two groups are not a clean treatment and control.

For a nontechnical founder, this is the important moment. AI can make planning artifacts and code changes appear faster than your team can decide whether they were necessary, correct, safe, adopted, or profitable. If the dashboard celebrates issues written and pull requests opened, the organization will optimize the new supply of work instead of the outcomes that work was meant to create.

This guide gives small product teams a practical alternative: a six-layer measurement stack from adoption to economic value, a four-week measurement card, a worked onboarding scenario, rules for choosing comparison windows, failure modes, and a scale/hold/stop decision. You do not need an analytics department. You do need to stop calling activity “productivity” before the evidence reaches the user.

What Linear Measured, and What It Did Not

The Linear AI usage report is valuable because it observes work across AI conversations, agent sessions, issue activity, comments, and pull requests instead of relying only on a survey or a model vendor's token count. Its adoption charts include a fixed cohort of 127,000 paid users active in both January and June 2026. Its pull-request trend covers 47,900 paid workspaces in June 2026.

The methodology also sets clear boundaries. It covers Linear's paid workspaces, misses AI activity outside Linear, defines an “AI-active” user as someone with at least one AI interaction in a 28-day window, and defines an “agent team” as a workspace with a coding agent connected. Some charts keep fixed cohorts; others do not. The report measures opened pull requests connected to Linear, not merged changes or production results.

That makes the report a strong description of adoption and activity inside one product ecosystem. It is not a causal estimate of what a coding agent would do to your team. The report's own observation that agent teams already had higher output matters: a team that chooses an agent may differ in staffing, architecture, process maturity, product stage, or willingness to automate.

Read the 111% as a capacity warning and an opportunity. Software teams can now generate much more candidate work. The scarce resource may have moved downstream to specification, review, validation, release, support, or deciding what deserves to exist.

Define the Terms Before You Build the Dashboard

Teams often use five words as if they were interchangeable. They are not.

Adoption means a person or team used an AI capability under a stated rule, such as at least one agent session in 28 days.
Activity means the system produced or modified an artifact: an issue, design, test, pull request, support reply, or deployment candidate.
Accepted change means an accountable owner approved a specific result after its required checks. “Accepted” must not mean the agent marked its own task complete.
Delivery means the accepted change reached the intended environment and cohort with measured reliability.
User outcome means a target user's behavior or experience changed in the intended direction.
Economic value means the outcome changed revenue, cost, risk exposure, or strategic learning after the full cost of generation, review, correction, operation, and incidents.

The distinction is the foundation of the framework. An agent can raise adoption without creating useful activity. It can create more activity without increasing accepted changes. It can increase accepted changes while overloading release and support. It can improve delivery speed without moving the user outcome. It can improve the outcome while costing more than the benefit.

This is why one productivity number is usually unsafe. The original SPACE framework from Microsoft Research, GitHub, and academic collaborators separates satisfaction and well-being, performance, activity, communication and collaboration, and efficiency and flow. Its practical lesson is not that every team needs a vast scorecard. It is that activity is one dimension, not a substitute for the rest.

Use a Six-Layer Evidence Stack

Measure the path from AI use to product value as a sequence. Each layer answers a different question and has a different owner.

Layer	Question	Example measure	Useful decision
1. Adoption	Is the intended team using the workflow?	Eligible users with a qualifying session	Improve access, training, or task fit
2. Activity	What candidate work is being produced?	Issues, prototypes, or PRs opened	Check whether supply moved
3. Acceptance	What survives accountable review?	Accepted changes / submitted changes	Improve task selection or evidence
4. Reliable delivery	What reaches users without avoidable rework?	Lead time, failed changes, deployment rework	Fix the delivery system
5. User outcome	Did the target behavior improve?	Activation, completion, retention, resolution	Keep, revise, or remove the feature
6. Economics	Was the outcome worth the full cost?	Cost per accepted outcome	Scale, constrain, or stop the workflow

Do not require every metric to rise. More agent activity may be healthy while acceptance falls during a deliberate exploration week. Deployment frequency may stay flat while the team ships fewer, more valuable changes. Support contacts may rise because adoption grew, even while contacts per activated user fall.

The stack is diagnostic. If activity triples but acceptance does not, inspect task definition, evidence, and reviewer capacity. If acceptance rises but reliable delivery worsens, improve tests, release boundaries, and rollback. If delivery improves but the user outcome is flat, question the roadmap and product hypothesis. If the outcome improves but economics deteriorate, reduce inference, review, support, or incident cost before scaling.

Treat Pull-Request Volume as a Flow Signal, Not a Goal

Pull-request volume is not useless. A sharp increase can reveal a real change in the development system. It can show that agents are taking previously deferred tasks, that work is being split differently, or that the review queue is about to become the bottleneck.

It becomes dangerous when leaders attach status, compensation, or a success declaration to the count. One agent can split a change into many pull requests or combine unrelated changes into one. A team can open candidates that never merge. A merged change can sit behind a flag. A deployed change can be rolled back. A successful deployment can still solve nothing for the user.

Recent research reinforces the need to inspect the work behind the count. A study of 25,264 agentic pull requests across 2,361 popular GitHub repositories found adoption and activity were highly uneven: the median repository produced only one or two agentic pull requests in the three-month observation period, and single-human oversight dominated. That is a useful description of early open-source practice, not a universal productivity benchmark.

Another study comparing 24,014 merged agentic pull requests with 5,081 merged human pull requests found structural differences in commits, files, and deleted lines, while description-to-diff alignment was only slightly higher for agentic work. Even a merged pull request with a good description still answers only whether a code contribution entered a repository. It does not establish product impact.

Use volume to ask operational questions: Is the queue growing? Is review time changing? Are candidates concentrated in one risky area? Are humans spending more time correcting, waiting, or coordinating? Never let it answer “Is AI working?” by itself.

Measure Accepted Change Before Speed

For a small team, accepted change rate is the most useful bridge between activity and delivery:

accepted change rate = changes that pass the defined acceptance gate / changes submitted for that gate

Define both sides before the trial. A submission might be a pull request linked to one outcome, a no-code workflow revision, a support automation rule, or a generated onboarding variant. Acceptance should require the evidence appropriate to its consequence: owner review, test results, policy check, preview, or a bounded user trial.

Also record rejection reason. Use a short controlled list:

wrong problem or duplicate work;
incomplete or incorrect result;
missing evidence;
excessive scope or risk;
review capacity unavailable;
product priority changed.

This prevents a falling acceptance rate from being blamed automatically on model quality. If half of the rejected work was never strategically useful, the task-selection system failed. If correct changes wait for days because one founder reviews everything, the bottleneck is human capacity. If work passes review but returns as incidents or urgent fixes, the gate is not predicting production quality.

A 2025 controlled GitHub study is a good reminder to keep scope explicit. 202 experienced Python developers completed one bounded web-server exercise, and blind reviewers used a shared rubric. The design can support a claim about that task and population. It cannot prove that every agent produces higher-quality changes in a mature product with customer data, legacy dependencies, and ambiguous requirements.

Add Delivery Health and Downstream Burden

Once accepted work moves toward production, measure the delivery system separately. DORA now describes five software-delivery performance metrics: change lead time, deployment frequency, failed deployment recovery time, change fail rate, and deployment rework rate. The last two are particularly important when AI increases candidate volume.

Do not optimize one metric in isolation. Faster lead time with more failed changes may move work from development into incidents. Higher deployment frequency can be healthy when changes are independently releasable and observable; it can be noise when a team repeatedly patches avoidable defects. Rework should include unplanned deployments that correct a user-visible problem, not only a revert with a particular label.

Track downstream burden in ordinary language that a founder can review weekly:

reviewer minutes per accepted change;
median wait before first accountable review;
percentage returned for material rework;
production fixes linked to the change within 7 or 14 days;
support contacts caused by the change;
incident and rollback minutes;
unresolved operational work added to the next cycle.

A current preprint on open-source Copilot adoption reports a cautionary pattern: core developers reviewed 6.5% more code while their original-code productivity fell 19%, alongside more rework after adoption. It is one observational research setting, not a forecast for your company. Its value is the mechanism it makes visible: apparent output can transfer hidden work to scarce experts.

Connect Every Trial to One User Outcome

An AI productivity trial without a user outcome becomes a tool-adoption program. Start with a behavior the product exists to change.

For onboarding, the outcome might be “a new workspace reaches its first successful result within 24 hours.” For support, it might be “the user's issue is resolved without reopening within seven days.” For a marketplace, it might be “a qualified listing reaches its first buyer conversation.” For an internal app, it might be “the operator completes the job with fewer corrections and no policy exception.”

Choose one primary outcome and one guardrail. Examples:

Product job	Primary outcome	Guardrail
Improve onboarding	First-value completion rate	Refunds or help requests per cohort
Reduce support backlog	Durable resolution rate	Incorrect-action escalations
Increase checkout conversion	Completed purchase rate	Payment errors and disputes
Accelerate internal reporting	Accepted report by deadline	Material correction rate

The primary outcome prevents the team from declaring success because it shipped more. The guardrail prevents it from buying the outcome through harm elsewhere. Define the eligible population, event, window, exclusions, data owner, and minimum sample before looking at the result.

The 2025 DORA study offers the same systems warning at a larger scale. Based on nearly 5,000 survey responses and more than 100 hours of qualitative data, DORA reported positive relationships between AI adoption, delivery throughput, and product performance, but a negative relationship with delivery stability. DORA's stated conclusion is that AI amplifies the surrounding system. That is more actionable than either “AI makes teams faster” or “AI makes teams worse.”

A Worked Scenario: The Onboarding Team That Tripled Activity

Consider a four-person team building a project-planning app. Its AI builder can create interface changes, analytics events, lifecycle messages, and tests. The team wants to improve the percentage of new workspaces that create a project, invite one collaborator, and complete one item within 24 hours.

During the four weeks before the trial, the team opens 12 onboarding changes, accepts 8, deploys 7, and reaches a 31% first-value completion rate. Review takes 80 minutes per accepted change. One deployed change needs urgent rework. The team spends $1,600 in labor and tooling attributable to the onboarding work.

During the four-week AI trial, it opens 36 changes. If leadership stops there, the dashboard reports 200% growth. But the rest of the stack says:

15 changes pass the acceptance gate;
12 reach the trial cohort;
reviewer time rises to 125 minutes per accepted change;
3 changes require urgent rework;
first-value completion moves from 31% to 35%;
help requests per activated workspace rise from 0.18 to 0.27;
attributable labor, inference, review, and support cost reaches $3,900.

This is neither an obvious win nor an obvious failure. The team created more accepted experiments and the primary outcome improved, but review, rework, support burden, and cost worsened. The next decision should not be “roll AI out everywhere.” It should be “keep the bounded workflow, reduce candidate supply, and require each candidate to name the predicted onboarding failure it addresses.”

The team then runs a second four-week cycle. It caps submissions at 16, rejects work without user evidence before generation, and requires a preview plus event validation. If acceptance remains 15, urgent rework falls to 1, reviewer time returns toward baseline, and completion holds at 35%, the system has learned to convert AI capacity into useful throughput. If the user outcome returns to baseline, the first result was not yet stable enough to scale.

Copy This Four-Week Measurement Card

Use one card for one workflow. Do not combine support automation, coding, research, and marketing generation into a single “AI productivity” experiment.

yaml

trial:
  workflow: onboarding improvement changes
  owner: product lead
  comparison_window: 4 weeks before vs 4 weeks during
  eligible_work: changes linked to the onboarding outcome
  primary_outcome:
    event: first successful project within 24 hours
    population: new eligible workspaces
    baseline: 31_percent
  guardrail:
    event: onboarding help requests per activated workspace
    baseline: 0.18
  layers:
    adoption: eligible_people_with_qualifying_ai_session
    activity: submitted_changes
    acceptance: accepted_changes / submitted_changes
    delivery: deployed_changes_without_14_day_rework
    user_value: first_value_completion_rate
    economics: total_trial_cost / incremental_successful_workspaces
  burden:
    reviewer_minutes_per_accepted_change: track
    urgent_rework: track
    support_contacts: track
  decision_date: 2026-09-16
  decision_options: [scale, continue_bounded, redesign, stop]

Record raw counts next to rates. “Acceptance fell from 70% to 50%” means something different at 10 submissions than at 1,000. Keep the pre-trial definition stable. If an event or cohort changes, start a new measurement version instead of silently rewriting the baseline.

Include full cost: subscriptions, inference, integration, prompt and workflow maintenance, human specification, review, corrections, support, incident response, and opportunity cost for the bottleneck role. Do not assign imaginary precision to opportunity cost; a documented estimate range is better than omitting it.

Calculate cost against the incremental outcome, not every successful user in the trial cohort. If 350 of 1,000 users succeed during the trial and the comparable baseline predicts 310, the provisional denominator is 40 incremental successes, not 350. If the measured lift is zero, negative, or too uncertain to distinguish from ordinary variation, report economic value as “not established.” Do not create a favorable ratio by dividing cost by activity or by users who would probably have succeeded anyway.

Choose a Comparison That Can Support the Decision

A small team rarely has enough traffic for a perfect causal study. It can still avoid obvious self-deception.

First, compare the same workflow and outcome. Do not compare AI-assisted onboarding work with last quarter's payment migration. Second, use fixed windows long enough to include review and production consequences. Third, keep a cohort definition that can be reconstructed. Fourth, record concurrent changes such as a launch, pricing shift, staffing change, or seasonal traffic.

When risk and traffic allow, use a randomized or staggered rollout. When they do not, compare matched task types or alternate weeks, and label the conclusion observational. A randomized trial by METR found experienced open-source developers working in familiar repositories took longer with early-2025 AI tools. METR later changed its experimental design as tools and usage evolved. The lesson is not that AI currently slows every expert. It is that effect size depends on the tool, date, user, task, repository, and study design.

Write conclusions at the strength the comparison supports:

Fact: 36 eligible changes were submitted during the trial.
Observation: acceptance rose in the trial window while review burden also rose.
Inference: task filtering probably contributed to the later reduction in rework.
Unknown: whether the workflow caused the user-outcome change.
Decision: continue with one cohort while collecting another window.

This wording is not academic decoration. It stops an internal dashboard from becoming a marketing claim that the evidence cannot defend.

Decide With a Matrix, Not a Victory Metric

At the end of the window, use acceptance, user outcome, and burden together.

Accepted changes	User outcome	Burden and guardrails	Decision
Up	Up	Stable or better	Scale one boundary at a time
Up	Flat	Stable	Redesign the work selection or product hypothesis
Up	Up	Materially worse	Continue bounded; fix review, rework, support, or risk
Flat/down	Up	Stable	Investigate whether fewer, better changes caused the gain
Up	Down	Any	Stop expansion and inspect harmful or distracting work
Any	Unclear	Any	Extend measurement; do not claim impact

“Scale” must name the next boundary: one more workflow, one larger cohort, or one additional action type. It should not mean buying more seats across the company because a four-week onboarding trial worked.

“Stop” does not mean AI has no value. It means this workflow, with this task definition and control system, has not earned more exposure. You can preserve the evidence and test a narrower task later.

Common Failure Modes and Misreadings

Calling connection adoption. A workspace with an enabled integration is not proof that the intended people use it for the intended job. Require a qualifying event and report eligible users. Calling activity output. An issue or pull request is a candidate artifact. Report whether it was accepted, deployed, and retained. Letting the agent grade itself. A completion message is not acceptance. Bind acceptance to an accountable person or an independent deterministic check appropriate to the risk. Counting only successful work. Include abandoned, duplicated, reverted, and superseded candidates. Otherwise the denominator disappears. Ignoring displaced labor. If a founder, senior engineer, or support lead absorbs more review and correction, the system may move work rather than remove it. Changing definitions after seeing the result. Version events, cohorts, exclusion rules, and windows. A prettier number produced by a new definition is a new metric. Turning a vendor cohort into your forecast. Linear, GitHub, DORA, METR, and open-source studies observe different populations, tools, time periods, and outcomes. Use them to identify mechanisms and design local tests, not to manufacture your ROI. Rewarding the metric. If people or agents are told to maximize issues, pull requests, or deployments, they can raise the number without raising value. Use activity as a diagnostic signal and judge the program on downstream evidence.

Where This Framework Fits, and Where It Does Not

Use this framework when a small team is adding AI to repeatable product work: code changes, no-code app revisions, support workflows, data reports, content operations, onboarding experiments, or internal automations. It works best when the team can define a candidate artifact, an accountable acceptance decision, a deployment or use event, and one observable outcome.

Do not use it as an individual employee ranking system. Work differs in difficulty, risk, collaboration, and invisible value. Per-person pull-request or acceptance counts invite gaming and punish people who review, simplify, prevent incidents, or solve ambiguous problems.

It is also insufficient for high-stakes medical, legal, financial, safety, or regulated decisions. Those workflows need domain validation, qualified oversight, applicable compliance, incident duties, and evidence specific to the consequence. A six-layer product dashboard does not authorize a risky use.

Finally, the framework cannot prove causality when the comparison cannot. Its job is to make the uncertainty visible and improve the next decision, not to turn every small-team rollout into a research paper.

A 48-Hour Setup for a Small Team

Hours 0–4: choose one workflow. Name the job, owner, eligible people, accepted artifact, current bottleneck, primary user outcome, and guardrail. Hours 4–8: freeze definitions. Write the six layers, raw counts, rates, cohort, event window, exclusions, and comparison period. Save a versioned card. Hours 8–16: connect existing evidence. Use the issue tracker, repository, deployment platform, product analytics, support system, and invoices you already have. Manual weekly entry is acceptable for a four-person team; an unreviewed “AI productivity” dashboard is not automatically better. Hours 16–24: establish the baseline. Reconstruct at least one comparable prior window. Mark missing evidence and concurrent changes. Do not invent a baseline because one field was not tracked. Hours 24–32: set the operating limits. Cap candidate work, define who accepts it, require rejection reasons, choose the rollout cohort, and state the conditions that pause the trial. Hours 32–40: test the measurement path. Follow one sample from AI interaction through submission, acceptance, deployment, product event, support impact, and cost. Fix broken identifiers before the real window. Hours 40–48: schedule the decision. Put a date on the calendar and pre-commit to scale, continue bounded, redesign, or stop. Assign one person to write facts, observations, inferences, unknowns, and the decision.

AI may have doubled the supply of candidate work. That is a real change. The founder's job is to build the evidence path that tells the team which candidates became reliable user value, at what cost, and without hiding the burden elsewhere. Measure that path, and higher activity can become leverage. Skip it, and the fastest-moving metric may simply be the queue.

References