I Analyzed 10 Major Open-Source Repos, and Every Single One Had Significant Drift

#devops #opensource #vibecoding #programming

Google's own DORA research shows a paradox: AI tools increase developer throughput while decreasing delivery stability.

I wanted to understand why. So I built an open-source CLI that detects development process drift, and ran it against 10 of the most widely-used repos in the industry: React, Next.js, VS Code, AWS CDK, Google Cloud Python, Supabase, LangChain, Stripe Node, Cloudflare Workers SDK, and Plaid.

Every single one had significant drift signals across multiple dimensions. Here's what I found.

CI Builds Are Silently Exploding

Every project showed elevated CI/build times. Not failures — the builds still pass. They just take dramatically longer than baseline, and nobody notices because there's no alarm for "your CI is 100x slower than it was 3 months ago."

Project	Build Duration vs Baseline
Google Cloud Python	1,552x
Stripe Node	1,361x
VS Code	946x
LangChain	889x
Supabase	611x
AWS CDK	112x
Next.js	104x
Cloudflare Workers	75x
React	13x
Plaid	3.2x

The median spike across all 10 repos is ~100x baseline. These aren't flaky tests or broken builds — they're passing builds that silently consume 100x more compute than they used to.

File Change Explosions Reveal Architectural Drift

When a single commit touches thousands of files across unrelated directories, something structural has shifted. This metric — file dispersion combined with files changed — was elevated in all 10 projects.

Project	Files Changed vs Baseline
AWS CDK	14,464x
VS Code	13,593x
Google Cloud Python	9,113x
Next.js	215x
React	208x
Stripe Node	150x
LangChain	87x
Supabase	67x
Plaid	65x
Cloudflare Workers	52x

The root causes vary — Google Cloud Python has Librarian bots mass-updating image dependencies, Next.js has Turbopack switching chunk hashes from hex to base40, Plaid has OpenAPI code generation. But the pattern is universal: large automated changes that no human reviews file by file.

Co-Change Novelty Drops to Zero

This is the most subtle and arguably most important signal. "Co-change novelty" measures whether new combinations of files are changing together, or whether the same files keep getting modified as a group.

When novelty drops, it means development has become pattern-locked — the same templates, the same file groups, the same automated workflows touching the same paths. It's the fingerprint of bot-driven or template-driven development.

9 out of 10 repos showed depressed co-change novelty:

Project	Co-change Novelty vs Baseline
AWS CDK	587x below
Supabase	587x below
VS Code	547x below
Next.js	509x below
Google Cloud Python	439x below
React	109x below
Cloudflare Workers	105x below
LangChain	88x below
Plaid	29.8x above (OpenAPI generates novel pairings)

Release Cadence Is All Over the Map

Deployment frequency — one of the four DORA metrics — showed extreme variance:

Project	Release Cadence Deviation
Google Cloud Python	2,548,259x
Cloudflare Workers	694,167x
React	69,548x
Supabase	229x
LangChain	220x
VS Code	104x
Next.js	68x
AWS CDK	13x
Stripe Node	11x
Plaid	N/A

These numbers look absurd, but they reflect real gaps between releases — periods where code accumulates without shipping, then gets batch-released. That accumulation is where drift compounds.

Cross-Family Correlations: The Hidden Connections

The most interesting findings come from correlating signals across different dimensions. These patterns are invisible if you only monitor one signal at a time:

CI duration correlates with files touched — found in all 10 repos. More scattered commits = longer builds. Obvious in retrospect, but nobody tracks it.
Release cadence correlates with code dispersion — found in 8/10 repos. When releases slow down, code changes spread across unrelated areas.
Dependency growth correlates with build time — VS Code (283x dependency growth + 946x build growth), Cloudflare (30x + 75x).

What This Means

No single commit in any of these projects looks wrong. Tests pass. Reviews approve. The code is correct.

But the aggregate pattern drifts. Files spread. Builds slow. Releases stall. Dependencies grow. And because each change is individually fine, nobody raises an alarm.

This is the drift problem. And it affects every project at scale — even the ones built by the best engineering organizations in the world.

It Pinpoints the Exact Commit

EE doesn't just say "your builds drifted." It identifies the specific commit where the deviation started — with a clickable link to the PR. For example:

React's file explosion? Triggered by b16b768f — [compiler] Feature flag cleanup (#35825)
React's co-change novelty drop? Started at 6853d7ab — [Perf Tracks] Prevent crash when accessing $$typeof

Every finding in the report links back to the trigger commit, so you're not hunting through git log trying to figure out when things changed. You see the drift, click Commit, and understand the root cause.

The Tool

I built Evolution Engine to detect this automatically. It's open source, runs locally (your code never leaves your machine), and works on any git repo:

pip install evolution-engine
evo analyze .

The analysis is pure statistics — no AI APIs called. When you want deeper investigation, EE generates a structured prompt you paste into your own AI tool.