We Scored 28 Famous Open Source PRs for Deploy Risk

Andrew — Mon, 13 Apr 2026 02:18:54 +0000

TL;DR
The React Hooks PR that changed every React application on earth? Three words in the commit message. One feature flag removed. It scored 91 out of 100 for deploy risk. The Svelte 5 release scored 99. A 65-line TypeScript change scored 79 and silently broke type inference in codebases worldwide. We ran 28 landmark open source pull requests through Koalr's deploy risk model. Here is what we found — and why it matters for the PRs your team ships every week.

The problem with code review
Modern code review answers one question well: is this code correct?

It answers a different question poorly: how likely is this to cause a production incident?

Those are not the same question. A PR can be clean, well-written, and thoroughly reviewed — and still wreck production because it touches a critical path nobody flagged, because the reviewer had twelve other PRs open, or because it is the fourth consecutive revert of a feature that never landed cleanly.

Most teams have no objective signal for the second question. They have green checkmarks.

What deploy risk scoring is
Koalr scores every pull request from 0 to 100 before it merges. The score is built from 36 signals:

Blast radius signals

How many files changed
What services those files belong to
Whether shared libraries or interfaces were modified
CODEOWNERS compliance — did the right people review the right files

Change quality signals

File churn — how recently and how often these files have been modified
Change entropy — how spread across the codebase the diff is
Lines added vs deleted ratio
Test coverage of changed files

Context signals

Reviewer load — how many open PRs each reviewer currently has
Author's recent incident rate
Time since last deploy to the same service
Revert history on the changed file set

History signals

Consecutive reverts of the same feature
Recent incident correlation with this file set
PR age — how long the branch has been open
A score of 0–39 is Low. 40–69 is Medium. 70–89 is High. 90–100 is Critical.

The score does not replace review. It gives reviewers a number to orient around before they start reading.

The experiment
We pulled 28 of the most consequential pull requests in open source history and ran them through the model. These are PRs the industry knows by name — the ones that shipped features used by millions of developers, or broke them.

Here is what the model said.

The obvious ones scored as expected

Svelte 5 release https://github.com/sveltejs/svelte/pull/13701 — score 99

The full runes rewrite merged to main. Thousands of files changed, the entire reactivity model replaced, years of migration work consolidated into one merge. Of course it scored critical. High blast radius, enormous file count, fundamental architecture change. The model does what you would expect.

TypeScript modules conversion https://github.com/microsoft/TypeScript/pull/51387 — score 98

Microsoft's conversion of the entire TypeScript compiler codebase from namespaces to ES modules. It touched every source file in the compiler, changed the build system, and dropped dependencies. If any PR in history deserved a mandatory all-hands review before merge, it was this one.

The surprising ones — small diffs, enormous blast radius
This is where it gets interesting.

React PR #14679 "Enable hooks!" https://github.com/facebook/react/pull/14679 — score 91

The commit message is three words. The diff is the removal of a single feature flag. You could read the entire change in thirty seconds.

It scored 91.

Why? Because the model does not count lines — it looks at what the changed code controls. A feature flag in a framework used by tens of millions of applications is not a small change. It is a detonation switch. The blast radius is every React application on earth. The model flagged it correctly.

Signals fired:
  blast_radius_score: 0.97
  feature_flag_detected: true
  downstream_consumers: critical
  reviewer_load: 0.2 (core team — low load)

Final score: 91 / Critical

Node.js PR #41749 "lib: add fetch" https://github.com/nodejs/node/pull/41749 — score 82

One file changed: the bootstrap script that runs inside every Node.js process. Adding the global fetch API touched the most critical execution path in the runtime.

Single-file PR. High score. The file changed is what matters, not how many files changed.

TypeScript PR #57465 "Infer type predicates from function bodies" https://github.com/microsoft/TypeScript/pull/57465 — score 79

65 lines of new code. One function modified.

Those 65 lines changed type inference behavior across the entire checker, producing new type errors in codebases that had compiled cleanly for years. A reviewer looks at 65 lines, sees clean code, approves it. The model sees that those 65 lines live inside the type checker core and have cross-cutting effects on every downstream consumer.

This is the failure mode standard review misses every time.

The revert pattern

Next.js PR #45196 https://github.com/vercel/next.js/pull/45196 — score 88

Title: "Revert 'Revert 'Revert 'Revert 'Initial metadata support''"''

PR body: "Hopefully last time."

Four consecutive reverts of the same feature. The model has a specific signal for this: repeated churn on the same file set with revert commits in recent history. It is one of the strongest predictors of another rollback. The PR scored 88 before anyone read a single line of the diff.

The one that surprised us most

The Jest-to-Vitest migration in tRPC — PR #3688 https://github.com/trpc/trpc/pull/3688 — scored 67. Medium risk.

At first glance, that sounds about right for a test runner swap. But look at what actually changed: every single test file in the repository, plus the root configuration, plus the CI pipeline. The surface area was enormous.

The score was “only” 67 because the risk model correctly identified that none of the changed files were production code paths — only test infrastructure. A test runner change cannot break a production deployment directly. What it can do is make future regressions invisible, which is a subtler and harder-to-measure risk.

The model is honest about what it can and cannot see. Broken test infrastructure does not score as a deploy risk — it scores as a coverage risk. Different signal, different response.

The score table
Here are eight of the 28 PRs we scored, with the risk level and the primary reason for the score:

What this means for your PRs
The open source examples are useful because they are public and well-documented. But none of those teams needed a risk model — the React core team was reviewing the hooks PR. It still would have scored 91.

The real value is the ordinary PR your team ships on a Thursday afternoon, reviewed by one person in fifteen minutes, that quietly introduces a breaking change nobody caught. That team does not have the React core team. They have two engineers, a Monday morning deadline, and a PR that looks fine.

That is who Koalr is built for.

Try it
The live risk demo at koalr.com/live-risk-demo scores any public GitHub PR in seconds. No account, no install. Paste a URL, get a score.

If you want to score your own team's PRs — every PR, automatically, as part of your GitHub workflow — there is a free trial at app.koalr.com/signup .

We Scored 28 Famous Open Source PRs for Deploy Risk — Here's What We Found

Andrew — Mon, 13 Apr 2026 01:35:54 +0000

The React Hooks PR that changed every React application on earth? Three words in the commit message. One feature flag removed. It scored 91 out of 100 for deploy risk. The Svelte 5 release scored 99. A 65-line TypeScript change scored 79 and silently broke type inference in codebases worldwide.

We ran 28 landmark open source pull requests through Koalr's deploy risk model. Here is what we found — and why it matters for the PRs your team ships every week.

The problem with code review

Modern code review answers one question well: is this code correct?

It answers a different question poorly: how likely is this to cause a production incident?

Most teams have no objective signal for the second question. They have green checkmarks.

What deploy risk scoring is

Koalr scores every pull request from 0 to 100 before it merges, built from 36 signals across four categories:

Blast radius — files changed, services affected, CODEOWNERS compliance, shared library modifications

Change quality — file churn, change entropy, lines added vs deleted, test coverage of changed files

Context — reviewer load, author's recent incident rate, time since last deploy, revert history on the changed file set

History — consecutive reverts of the same feature, recent incident correlation, PR age

0–39 → Low
40–69 → Medium
70–89 → High
90–100 → Critical

The score does not replace review. It gives reviewers a number to orient around before they start reading.

The obvious ones scored as expected

Svelte 5 release — score 99

The full runes rewrite merged to main. Thousands of files changed, the entire reactivity model replaced, years of migration work consolidated into one merge. High blast radius, enormous file count, fundamental architecture change. The model does what you would expect.

TypeScript modules conversion — score 98

Microsoft's conversion of the entire TypeScript compiler from namespaces to ES modules. Touched every source file in the compiler, changed the build system, dropped dependencies. If any PR in history deserved a mandatory all-hands review before merge, it was this one.

The surprising ones — small diffs, enormous blast radius

This is where it gets interesting.

React PR #14679 "Enable hooks!" — score 91