DEV Community: Codequal

What 38 months of commits did to LangChain's architecture — measured

Codequal — Mon, 20 Jul 2026 01:09:43 +0000

For most of its history, LangChain shipped a new release roughly every 30 minutes. By the end of the window I analyzed, releases were coming once every six days, and a CI build that historically took about 10 seconds was taking 55 minutes — 889 times the repo's own baseline, and still 27 standard deviations above normal at the last measurement.

None of this is judged against an external standard, a linter rule, or anyone's opinion of what a "healthy" repo looks like. It's LangChain measured against LangChain: 38 months of its own commit history (January 27, 2023 to March 14, 2026), with every number below tied to a specific commit you can look up.

I picked LangChain deliberately. It's one of the few major codebases whose entire git history sits inside the AI-assisted era — it was born the same quarter the world started pair-programming with LLMs. To be clear about what that means: I cannot tell you which of its commits were written by humans and which by AI tools. Nobody can, from the outside. What I can measure is what happened to the repository's structural rhythms over those 38 months — how big commits got, how builds behaved, how releases moved, which files changed together. Here is what the data says.

1. Builds: 10 seconds → 55 minutes, and still elevated

The headline number, because it's the one that hasn't gone away.

LangChain's typical build duration across its history: 9.5 seconds. Observed at the end of the window: 3,306 seconds — about 55 minutes. That is 889x the repo's own baseline, and unlike most deviations in this analysis, it did not settle back down: at the last measurement it was still 27.6 standard deviations above normal.

Some of this is the honest cost of growth — LangChain became a large monorepo with many packages, and monorepos pay CI taxes. But the shape matters: the analysis found build-time growth statistically entangled with two other signals (commit size and file-pairing novelty — see below), escalated because independent measurements kept converging on the same story. Slow builds weren't an isolated infrastructure problem. They moved together with how the code itself was changing.

2. Releases: every 30 minutes → every 6 days

Historical norm between releases: 0.5 hours. Observed: 145.8 hours — a bit over six days. That's 220x the baseline.

If you followed LangChain in 2023, you remember the joke that it released faster than you could pip install it. The data confirms the joke, then shows it ending. One honest caveat, and it's important: unlike the build-time finding, this deviation later returned to baseline — the release rhythm recovered. Whether the slowdown was a bottleneck, a deliberate maturation of the release process, or a stabilization push is a question the maintainers could answer in one sentence and I can't. The measurement only says: for a while, the repo's release heartbeat ran 220x slower than its own normal, and the change correlated with commits touching more files than usual.

3. Commit size: 2 files → 131 files

Typical files changed per commit, across 38 months of history: 2. Observed: 131 — 87x the baseline.

The trigger commit is public and you can check it yourself: 4e9a1eb2 — "chore: delete CLI (#34855)", 131 files in one commit. Deleting a whole CLI is a legitimate thing to do; the point is not that this commit was bad. The point is that a repo whose median commit touches two files developed a recent window where hundred-file commits stopped being exceptional. Large commits are where review quality goes to die quietly — not because reviewers are lazy, but because no human meaningfully reviews 131 files.

4. Novelty of file pairings: 0.99 → 0.00

This is the strangest one, and I want to present it carefully, because the honest reading is less dramatic than the number.

"Co-change novelty" asks: of the file combinations changed together in a commit, what fraction has never been seen together before? LangChain's historical norm is 0.99 — nearly every commit combined files in some new way, which is what a rapidly evolving codebase looks like. In the recent window it hit 0.0000: commits touching only file combinations the repo had seen many times before.

The trigger commit is cdf140e7 — "release(core): 1.2.17 (#35527)", a two-file version bump. So a big driver here is release automation: bots bumping the same two files over and over. That's the honest explanation, and I'm giving it to you rather than hiding it. But it's worth sitting with what it means in aggregate: an increasing share of this codebase's commit stream is repetition — the same well-worn file pairs, again and again. If that sounds familiar, it should: GitClear's research on AI-era code found the same directional story in a different metric — copy-pasted and duplicated code rising while refactoring (moved code) falls. Two independent measurement approaches, same rhyme: more repetition, less novel restructuring. I'd call that a trend worth watching rather than a verdict.

5. The signals converge

Any one of these numbers alone is a curiosity. What made me write this up is the convergence: the analysis matched 23 known cross-signal patterns (from a library of 44 validated across 200+ open-source repos), and several were escalated specifically because independent signals kept moving together — file-pairing novelty with build time, commit size with release frequency, change focus with both. When one metric drifts, it's noise. When five drift in a correlated way, the repository's development rhythm has genuinely changed.

Methodology, honestly

Because this kind of analysis is easy to do badly, here's exactly what was and wasn't done:

Baselines are per-repo. Every "typical" value is computed from LangChain's own history using robust statistics (median-based measures — MAD and IQR — which resist being skewed by outliers). No comparisons to other repos, no universal thresholds.
The deviation window is compared against a baseline of 20 prior commits. That's short by design — it answers "is this abnormal for this repo right now," not "is this a bad codebase."
Some deviations later returned to baseline (release cadence; commit size at its trigger). Build duration did not — it was still 27.6σ elevated at last measurement. I've marked which is which above.
Units: build duration in seconds; release cadence in hours between releases.
Every finding names its trigger commit by hash and PR number. Please check them — that's why they're included.
All data is public. The analysis ran locally against the public git history, GitHub Actions, and GitHub Releases data.
What this can't tell you: which commits were AI-written, whether any individual change was a mistake, or anything about the correctness of individual lines. It measures structural rhythm, not code quality line-by-line.

The full interactive report — every metric, every trigger commit, all 23 matched patterns — is here: codequal.dev/reports/langchain.

Measured with Evolution Engine, a drift detector I built.

I'm Rostislav Alpin, owner of CodeQual. I spent 20 years in QA before building measurement tools, so my bias is exactly what you'd expect: I trust instruments over vibes, and I distrust my own instruments until someone else has kicked them.

Which is the real reason this post exists. One repo is an anecdote. If you want to know what your repository's baseline looks like — and whether the last few months broke it:

Want this run on your repo? Reply.

I Analyzed 10 Major Open-Source Repos, and Every Single One Had Significant Drift

Codequal — Tue, 24 Mar 2026 00:57:06 +0000

Google's own DORA research shows a paradox: AI tools increase developer throughput while decreasing delivery stability.

I wanted to understand why. So I built an open-source CLI that detects development process drift, and ran it against 10 of the most widely-used repos in the industry: React, Next.js, VS Code, AWS CDK, Google Cloud Python, Supabase, LangChain, Stripe Node, Cloudflare Workers SDK, and Plaid.

Every single one had significant drift signals across multiple dimensions. Here's what I found.

CI Builds Are Silently Exploding

Every project showed elevated CI/build times. Not failures — the builds still pass. They just take dramatically longer than baseline, and nobody notices because there's no alarm for "your CI is 100x slower than it was 3 months ago."

Project	Build Duration vs Baseline
Google Cloud Python	1,552x
Stripe Node	1,361x
VS Code	946x
LangChain	889x
Supabase	611x
AWS CDK	112x
Next.js	104x
Cloudflare Workers	75x
React	13x
Plaid	3.2x

The median spike across all 10 repos is ~100x baseline. These aren't flaky tests or broken builds — they're passing builds that silently consume 100x more compute than they used to.

File Change Explosions Reveal Architectural Drift

When a single commit touches thousands of files across unrelated directories, something structural has shifted. This metric — file dispersion combined with files changed — was elevated in all 10 projects.

Project	Files Changed vs Baseline
AWS CDK	14,464x
VS Code	13,593x
Google Cloud Python	9,113x
Next.js	215x
React	208x
Stripe Node	150x
LangChain	87x
Supabase	67x
Plaid	65x
Cloudflare Workers	52x

The root causes vary — Google Cloud Python has Librarian bots mass-updating image dependencies, Next.js has Turbopack switching chunk hashes from hex to base40, Plaid has OpenAPI code generation. But the pattern is universal: large automated changes that no human reviews file by file.

Co-Change Novelty Drops to Zero

This is the most subtle and arguably most important signal. "Co-change novelty" measures whether new combinations of files are changing together, or whether the same files keep getting modified as a group.

When novelty drops, it means development has become pattern-locked — the same templates, the same file groups, the same automated workflows touching the same paths. It's the fingerprint of bot-driven or template-driven development.

9 out of 10 repos showed depressed co-change novelty:

Project	Co-change Novelty vs Baseline
AWS CDK	587x below
Supabase	587x below
VS Code	547x below
Next.js	509x below
Google Cloud Python	439x below
React	109x below
Cloudflare Workers	105x below
LangChain	88x below
Plaid	29.8x above (OpenAPI generates novel pairings)

Release Cadence Is All Over the Map

Deployment frequency — one of the four DORA metrics — showed extreme variance:

Project	Release Cadence Deviation
Google Cloud Python	2,548,259x
Cloudflare Workers	694,167x
React	69,548x
Supabase	229x
LangChain	220x
VS Code	104x
Next.js	68x
AWS CDK	13x
Stripe Node	11x
Plaid	N/A

These numbers look absurd, but they reflect real gaps between releases — periods where code accumulates without shipping, then gets batch-released. That accumulation is where drift compounds.

Cross-Family Correlations: The Hidden Connections

The most interesting findings come from correlating signals across different dimensions. These patterns are invisible if you only monitor one signal at a time:

CI duration correlates with files touched — found in all 10 repos. More scattered commits = longer builds. Obvious in retrospect, but nobody tracks it.
Release cadence correlates with code dispersion — found in 8/10 repos. When releases slow down, code changes spread across unrelated areas.
Dependency growth correlates with build time — VS Code (283x dependency growth + 946x build growth), Cloudflare (30x + 75x).

What This Means

No single commit in any of these projects looks wrong. Tests pass. Reviews approve. The code is correct.

But the aggregate pattern drifts. Files spread. Builds slow. Releases stall. Dependencies grow. And because each change is individually fine, nobody raises an alarm.

This is the drift problem. And it affects every project at scale — even the ones built by the best engineering organizations in the world.

It Pinpoints the Exact Commit

EE doesn't just say "your builds drifted." It identifies the specific commit where the deviation started — with a clickable link to the PR. For example:

React's file explosion? Triggered by b16b768f — [compiler] Feature flag cleanup (#35825)
React's co-change novelty drop? Started at 6853d7ab — [Perf Tracks] Prevent crash when accessing $$typeof

Every finding in the report links back to the trigger commit, so you're not hunting through git log trying to figure out when things changed. You see the drift, click Commit, and understand the root cause.

The Tool

I built Evolution Engine to detect this automatically. It's open source, runs locally (your code never leaves your machine), and works on any git repo:

pip install evolution-engine
evo analyze .

The analysis is pure statistics — no AI APIs called. When you want deeper investigation, EE generates a structured prompt you paste into your own AI tool.

GitHub | PyPI | Website

What 10 Open Source Repos Reveal About Development Drift in the AI Era

Codequal — Tue, 17 Mar 2026 23:58:57 +0000

Every major tech company is pushing AI-generated code. Microsoft says 30% (targeting 80%). Google says 30%. Uber reports 65-72%. Amazon mandated 80% AI tool usage. Shopify made AI "mandatory."

But Google's own DORA research shows a paradox: AI increases throughput while decreasing delivery stability. Teams ship faster, but the code breaks more often.

I wanted to understand why. So I built Evolution Engine, an open source CLI that detects development process drift — when patterns in commit history, CI builds, deployments, and dependency signals shift in ways that often precede production issues. Then I ran it on 10 major open source repos across cloud infrastructure, frontend frameworks, AI tooling, and developer platforms.

No AI APIs are called during analysis. All pattern detection is deterministic and statistical. The tool runs entirely locally — your code never leaves your machine.

Here's what I found.

The scale

Across 10 repos, the tool analyzed over 130,000 commits, generating 250,000+ events across git, CI, deployment, and dependency signal families. It matched patterns from a knowledge base calibrated across 200+ open source repositories.

Metric	Result
Repos analyzed	10
Total commits	130,000+
Total events	250,000+
Signal families	4 (git, CI, deployment, dependency)
Repos with significant drift	10 out of 10
Average drift signals per repo	6.6
Average correlation patterns per repo	24.3

Every single repo had significant drift signals. Every one.

Finding 1: CI build times spike dramatically — and nobody notices

The most consistent pattern across all 10 repos: CI build duration spikes that dwarf historical baselines.

Repo type	Normal CI time	Spike	Deviation
Cloud SDK (monorepo)	~45 seconds	6+ hours	1,552x
AI framework	~95 seconds	55 minutes	889x
Cloud infrastructure toolkit	~26 seconds	70+ minutes	111x
Edge platform SDK	~33 seconds	60+ minutes	74x
Commerce framework	~64 seconds	8+ minutes	43x
Code editor	~41 seconds	23+ minutes	34x
Frontend framework	~45 seconds	6 minutes	13x
Fullstack framework	~6 minutes	32 minutes	5x

These aren't gradual slowdowns — they're sudden spikes, often tied to a single commit or dependency change. The problem? Most teams don't track CI duration as a process signal. They notice when builds fail, but a 34x slowdown that still passes? That drifts silently.

8 out of 10 repos had CI spikes exceeding 10x their baseline. The median spike was 53x.

Finding 2: Release cadence gaps correlate with code spread

When a repo's release cadence suddenly lengthens, it's almost always accompanied by increased code dispersion — changes spread across unrelated parts of the codebase.

Repo type	Normal cadence	Gap	Slowdown
Cloud SDK (monorepo)	~2.9 hours	22 days	182x
Commerce framework	~1.5 days	37 days	24x
Cloud infrastructure toolkit	~21 hours	16.5 days	18x
Fullstack framework	~6 days	96 days	16x
Logging library	~13 days	200 days	15x
Frontend framework	~28 days	113 days	4x

This correlation showed up as a known pattern in 8 out of 10 repos. When engineers touch more unrelated files per commit and releases slow down, something structural has shifted — often a large refactoring, a dependency migration, or (increasingly) an AI-assisted batch change that touches more files than a human would.

Finding 3: Co-change novelty drops to zero

"Co-change novelty" measures how often files that change together in a commit have changed together before. A score of 1.0 means entirely novel pairings. A score of 0.0 means the exact same files are changing together repeatedly.

In 9 out of 10 repos, we found commits where co-change novelty dropped to zero — indicating repetitive, pattern-locked changes rather than organic development. This is a hallmark of:

Automated dependency bumps (bots touching the same lockfiles repeatedly)
Code generation tools producing similar diffs
AI-assisted changes that follow templates rather than addressing unique problems

The interesting question: is this a problem? Sometimes repetitive changes are exactly right (automated security patches). But when novelty drops to zero and CI times spike and release cadence gaps appear, the correlation suggests something has gone wrong.

Finding 4: Merge-back commits create statistical blind spots

Three repos had single commits touching 10,000-21,000+ files. These are merge-back commits in monorepos — technically expected, but they create extreme statistical outliers that mask real drift signals underneath.

If your drift detection (or any metrics tool) doesn't account for these outliers, the signal-to-noise ratio collapses. A legitimate 34x CI spike looks insignificant next to a 14,000x files_touched outlier.

Finding 5: Cross-family correlations reveal systemic patterns

The most interesting findings weren't individual metrics — they were correlations between signal families:

CI duration <-> files touched: When commit size increases, build times increase non-linearly. This correlation appeared in all 10 repos.
Deployment cadence <-> code dispersion: When releases slow down, changes spread wider. Found in 8/10 repos.
Dependency changes <-> change locality: When dependencies change, subsequent code changes tend to be less focused. Found in 7/10 repos.

These cross-family patterns are invisible if you only monitor one signal family (just CI, or just git). You need the full picture.

What this means for AI-assisted development

Google's DORA research found that AI increases throughput but decreases stability. Our findings suggest why:

AI generates larger commits — more files touched per change, increasing CI load
AI follows templates — co-change novelty drops, creating repetitive patterns
AI doesn't respect cadence — large batch changes break release rhythm
The drift is gradual — no single commit looks wrong, but the aggregate pattern shifts

The fix isn't to stop using AI tools. It's to monitor the process signals they affect. The same way you'd monitor application performance after a deployment, you should monitor development process patterns after adopting AI coding tools.

What's next

This is the first in a series. In upcoming posts, I'll publish detailed case studies of individual repos (with permission from maintainers where applicable) and dive deeper into specific patterns — like how dependency drift predicts deployment instability, and what "healthy" drift patterns look like versus problematic ones.

Try it yourself

Evolution Engine is open source. Install it and run on any repo:

pip install evolution-engine
evo analyze /path/to/your/repo

The tool generates an interactive HTML report with all findings, plus an investigation prompt you can paste into any AI assistant for root cause analysis — so your AI tools can help diagnose the drift patterns they create.

All analysis is local and statistical. No code leaves your machine. No AI APIs are called.

GitHub: github.com/alpsla/evolution-engine
Website: codequal.dev

I built this. Evolution Engine is open source — dual-licensed: CLI and adapters are MIT, core engine is BSL 1.1 (converts to MIT in 2029). Happy to answer questions in the comments.

AI Writes Your Code. Who Watches for Drift?

Codequal — Tue, 10 Mar 2026 13:31:22 +0000

The problem

AI coding tools are incredible — until they quietly drift off course.

You're using Cursor, Copilot, or Claude Code. The code looks fine. Tests pass. But over a few commits, subtle shifts accumulate: more files touched per commit than usual, dependency trees growing unexpectedly, CI times creeping up.

By the time you notice, the damage has compounded across dozens of commits.

What Evolution Engine does

Evolution Engine is a local-first CLI that monitors your SDLC signals and flags statistical anomalies:

Git patterns — file dispersion, change locality, co-change novelty
CI pipelines — duration trends, failure patterns
Dependencies — count changes, depth shifts
Deployments — release cadence, prerelease patterns
Testing — failure rates, skip rates, suite duration
Coverage — line and branch rate changes

When a metric deviates significantly from your project's baseline, EE raises an advisory — not a bug report, a drift alarm.

No AI APIs required

This was a deliberate design choice. Your code never leaves your machine. EE does pure statistical analysis locally.

When you want deeper investigation, EE generates a structured prompt you can paste into your own AI tool — ChatGPT, Claude, Cursor, whatever you trust. You control what leaves your machine.

How to use it

pip install evolution-engine
cd your-project
evo analyze .

That's it. EE builds a baseline from your git history and flags deviations.

As a GitHub Action

- uses: alpsla/evolution-engine@v1
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}

As a git hook

evo init --hooks

Open source

Evolution Engine is available on PyPI and GitHub. Git analysis is free — no account, no license key needed.

Would love feedback from the community. What signals matter most in your workflow?