Engineering-productivity dashboards tend to show the same five pull-request metrics: Time to First Review, Time to Merge, Comments per PR, Review Cycles, and AI Resolution Rate. Each gets a card and a trend line.
I wanted to know which of them can answer one narrow question: "is review the slow part of shipping a change?" My own repositories are no help here. I work alone, and my busiest repo merged four PRs in the last 90 days. So I pointed a script at two public projects with real review traffic:
- pytest-dev/pytest: 107 merged PRs from humans (166 merged, 59 from bots), 46 authors
- pydantic/pydantic: 196 merged PRs from humans (221 merged, 25 from bots), 40 authors
The window is PRs merged between 2026-07-21 and 2026-10-19 (UTC). All numbers below come from the GitHub REST API, and you can rerun the script to check them.
The script
# review_metrics.py
import json, os, sys
from datetime import datetime, timedelta, timezone
from statistics import mean, median
from github import Auth, Github
REPO = sys.argv[1] if len(sys.argv) > 1 else "pytest-dev/pytest"
END = datetime(2026, 10, 19, tzinfo=timezone.utc)
START = END - timedelta(days=90)
g = Github(auth=Auth.Token(os.environ["GITHUB_TOKEN"]))
repo = g.get_repo(REPO)
rows = []
for pr in repo.get_pulls(state="closed", sort="updated", direction="desc"):
if pr.updated_at < START:
break # sorted by updated: nothing older can have merged in the window
if not pr.merged_at or not (START <= pr.merged_at < END):
continue
if pr.user.type == "Bot" or pr.user.login.endswith("[bot]"):
continue
author = pr.user.login
reviews = [r for r in pr.get_reviews()
if r.user and r.user.login != author and r.submitted_at]
first = min((r.submitted_at for r in reviews), default=None)
approvals = [r.submitted_at for r in reviews if r.state == "APPROVED"]
hours = lambda a, b: (b - a).total_seconds() / 3600
rows.append({
"pr": pr.number,
"author": author,
"created_at": pr.created_at.isoformat(),
"merged_at": pr.merged_at.isoformat(),
"first_review_h": hours(pr.created_at, first) if first else None,
"merge_h": hours(pr.created_at, pr.merged_at),
"last_approval_to_merge_h": hours(max(approvals), pr.merged_at) if approvals else None,
"review_comments": pr.review_comments,
"cycles": sum(1 for r in reviews if r.state in ("CHANGES_REQUESTED", "APPROVED")),
"changed_lines": pr.additions + pr.deletions,
})
json.dump(rows, open(f"{REPO.replace('/', '__')}.json", "w"), indent=1)
def summary(key):
v = [r[key] for r in rows if r[key] is not None]
return f"n={len(v):3d} median={median(v):7.1f} mean={mean(v):7.1f}"
print(f"{REPO} merged {START:%Y-%m-%d}..{END:%Y-%m-%d} human PRs: {len(rows)}")
print(f"PRs with no review from anyone but the author: {sum(r['first_review_h'] is None for r in rows)}")
for k in ("first_review_h", "merge_h", "last_approval_to_merge_h", "review_comments", "cycles"):
print(f"{k:26s} {summary(k)}")
Three definitions matter. A "review" is any submitted GitHub review (approve, request changes, or comment) from someone other than the PR author. review_comments counts inline code comments only, not the conversation tab. A "cycle" is one approve or request-changes event.
Raw output:
pytest-dev/pytest merged 2026-07-21..2026-10-19 human PRs: 107
PRs with no review from anyone but the author: 0
first_review_h n=107 median= 9.2 mean= 284.6
merge_h n=107 median= 143.4 mean= 806.0
last_approval_to_merge_h n=107 median= 1.8 mean= 128.3
review_comments n=107 median= 0.0 mean= 3.3
cycles n=107 median= 1.0 mean= 1.5
pydantic/pydantic merged 2026-07-21..2026-10-19 human PRs: 196
PRs with no review from anyone but the author: 100
first_review_h n= 96 median= 0.2 mean= 38.2
merge_h n=196 median= 1.6 mean= 63.4
last_approval_to_merge_h n= 41 median= 0.3 mean= 28.7
review_comments n=196 median= 0.0 mean= 0.8
cycles n=196 median= 0.0 mean= 0.2
Two things in that output are worth more than any single metric.
Problem 1: the average describes five PRs
pytest's mean Time to First Review is 284.6 hours. The median is 9.2 hours. A dashboard that shows the mean says "nearly 12 days"; nearly two thirds of the PRs got one within 24 hours.
The gap comes from a handful of old PRs. Five of the 107 were opened between January and early July 2026 and only merged inside the window. Their first-review waits (5,568h, 5,415h, 4,261h, 3,305h, 1,650h) add up to 66% of all first-review hours in the sample. Drop those five and the mean falls to 100.5 hours.
The spread is wide even without them:
| pytest, Time to First Review | Hours |
|---|---|
| 25th percentile | 0.6 |
| median | 9.2 |
| 75th percentile | 95.4 |
| 90th percentile | 507.5 |
43 of 107 PRs (40%) got a first review within 4 hours, 69 (64%) within 24 hours, and 20 waited more than a week. "Share of PRs reviewed within N hours" plus the 90th percentile says far more than an average.
Problem 2: the metric quietly skips half the PRs
In pydantic, 100 of 196 human PRs were merged without any GitHub review from someone other than the author. 75 of those 100 came from a single maintainer. Unreviewed PRs merged in a median of 0.3 hours, reviewed ones in 7.5 hours.
Time to First Review cannot be computed for a PR that never got a review, so the 0.2-hour median above describes only 96 of the 196 PRs. Whether merging without a review is fine depends on the project's rules, and I am not judging pydantic's. The dashboard problem is that the number looks complete when it covers 49% of merges. Put the coverage next to the metric: "median 0.2h, 96 of 196 PRs reviewed".
The two metrics that point at review
Time to First Review, reported as a median and percentiles with coverage, measures the stretch where a PR waits and no reviewer has acted. That wait is a review-process delay by definition.
Published research gives a reference point. Google's 2018 case study of its own code review, based on logs of about nine million reviewed changes, reports a median wait for initial feedback of under an hour for small changes and about 5 hours for very large ones, with an overall median under 4 hours for the whole review process (Sadowski et al., ICSE-SEIP 2018). That is a large company with dedicated tooling, so treat it as a reference point rather than a target for a volunteer project.
The lever with the strongest evidence is explicit assignment. Meta suspected that diffs assigned only to a team were slowed by the bystander effect ("someone else will probably review this") and tested it. In an A/B test on 12.5k authors in spring 2023, it randomly assigned one of the top three recommended reviewers on team-only diffs. TimeInReview fell 11.6%, with no significant change in the time reviewers spent on each diff (Improving Code Reviewer Recommendation, ACM TOSEM 2026, arXiv:2312.17169). You do not need Meta's recommender to try the idea. GitHub can request code owners automatically, and team review requests can be routed to individual members by round robin or load balance.
Review Cycles measures whether review converges. In pytest, 68 of 107 PRs merged after one approve-or-request-changes event, 26 after two, and 13 after three to five. Cycles barely tracked PR size (Spearman 0.19 against changed lines), so a high count is less about big diffs and more about how a review gets run, for example comments arriving in several rounds. That makes it a review-process signal you can act on.
The three that answer other questions
Time to Merge mixes three intervals: waiting for a first review, the back-and-forth until the last approval, and the wait from last approval to merge. I split every pytest PR into those three segments and looked at which one was longest:
| Longest segment of Time to Merge (pytest) | PRs |
|---|---|
| Waiting for first review | 44 |
| Approval to merge | 34 |
| First review to last approval | 29 |
Summed over all PRs, the split was 35% waiting, 49% back-and-forth, 16% after approval. One number covers three different problems, and the approval-to-merge part (CI, merge queues, release timing) is outside review entirely. Time to Merge is still a fine throughput metric. It just cannot tell you review is the bottleneck.
Comments per PR mostly tracks size. In pytest, inline comment count correlated with changed lines at Spearman 0.53, and 67 of 107 PRs (63%) had zero inline comments. In pydantic, 135 of 196 had none. A month-over-month rise in the average usually reflects a shift in PR mix. What it can flag is a sustained zero on large PRs, which is worth a look.
AI Resolution Rate cannot be computed from the GitHub API alone, because GitHub does not record whether a bot comment was "resolved" in the sense the metric needs. You need data from the AI review tool itself, so I did not measure it here. One structural caution applies anyway: the rate depends on what the bot chooses to flag, so changing the tool or its rules moves the number without any change in how your team responds. Compare it only within one tool and one configuration.
What I would put on the dashboard
- Time to First Review: median, 90th percentile, share reviewed within 4h and 24h, and how many merged PRs had a review at all
- Review Cycles: distribution (1 / 2 / 3+) alongside the mean
- Time to Merge, Comments per PR, AI Resolution Rate: on a separate throughput or tooling page
Rerun the script on your own repo before you trust any of these. The two projects here behaved so differently (pytest reviews every PR, pydantic merged half without one) that a single benchmark would have misled both.
The longer version, with the three-layer review model (hooks, AI review, human review), the CodeRabbit configuration, and a GitHub Actions workflow that writes a weekly metrics report, is in my book Systematizing AI Code Review. The English edition is on Kindle.
Sources:
- pytest-dev/pytest and pydantic/pydantic, GitHub REST API, PRs merged 2026-07-21 to 2026-10-19
- Sadowski, Söderberg, Church, Sipko, Bacchelli. Modern Code Review: A Case Study at Google. ICSE-SEIP 2018
- Improving Code Reviewer Recommendation: Accuracy, Latency, Workload, and Bystanders. ACM TOSEM 35(1), 2026
- GitHub Docs: About code owners, Managing code review settings for your team
Top comments (0)