DEV Community: Ben Halpern

Design for Churn

Ben Halpern — Wed, 14 Oct 2026 12:03:29 +0000

The biggest design shift of the AI era isn’t about who writes the code. It’s about how long the code is expected to live.

Churn has always been a dirty word in software. Microsoft researchers showed years ago that relative code churn predicts defect density. Adam Tornhill wrote a whole book called Your Code as a Crime Scene, where frequently changed, complex files are the first suspects. GitClear’s research projected that AI assistants would double the share of code rewritten within two weeks of being written, and framed that as downward pressure on quality. The instinct runs deep: code that keeps changing is code that was wrong.

But that instinct comes from a world where writing code was expensive. Durability was how you got your money’s worth. When a feature costs an afternoon instead of a sprint, the math flips. Rewriting something is often cheaper than carefully preserving it, and that’s only going to become more true.

Codebases are gardens now

We’ve always described software in building terms: architecture, foundations, scaffolding, load-bearing walls. Buildings are supposed to stand still.

I think codebases are becoming more like gardens. Specifically, gardens full of annuals. A gardener replants annuals every season, and nobody calls a tomato plant technical debt because it dies in October. The gardener’s real investment goes into what doesn’t turn over: the soil, the beds, the irrigation, the oak tree in the corner. Good structure is exactly what makes replanting cheap.

Code that churns a lot is essentially disposable, and disposable doesn’t have to mean sloppy. It can still be well made. It just isn’t made to last. Quality for an annual means it does its job, it’s tested at its edges, and it can come out without tearing up everything around it.

The problem is that we don’t have a picture of what high-quality churn looks like yet. Our mental model of quality is tangled up with longevity. We brag about code that has run untouched for six years. So churn happens anyway, but implicitly, and it lands wherever is most convenient.

Where churn actually lands

I looked at the last six months of Forem: 563 commits. The most-churned file is app/models/article.rb, touched in 37 of them.

Our spam handler is up there too, at 13. That one I’m happy about. Spam is a seasonal crop. Spammers change tactics constantly, and our code should churn right along with them.

article.rb is the oak tree. When the oak gets dug up every week, it means new features are rooting themselves in the core model because that was the easiest place to plant them. That’s the structural compromise. Churn isn’t the problem. Unplanned churn in load-bearing places is.

Designing for it

A few ideas for making churn explicit instead of accidental:

Name the layers. Say out loud what’s an annual and what’s a perennial. Our AGENTS.md already does a crude version of this by telling AI assistants to tread carefully around core auth, feed sorting, and cache pipelines. That should be more than a warning label. It should be the shape of the codebase.

Put contracts at the edges of the beds. Interfaces, schemas, API specs, and tests that pin behavior rather than implementation are the perennials. Whatever sits behind them can be rewritten freely, by a person or an agent.

Make removal cheap. The test of a good annual is whether you can pull it out without disturbing the bed. If something is meant to churn, it shouldn’t have tendrils in everything.

Measure churn by layer. High churn in the fast layer is a sign of health. Churn in the slow layer is the alarm.

Churn is a design input

Churn is going to happen either way. AI has made it the default rate of change for most of what we build. We can keep treating it as a symptom, or we can treat it as a design input and build the structure that lets us replant with confidence.

I will acknowledge that I have always been more of the perspective that churn is more acceptable than conventional wisdom of software design has implied. I’ve never seen the avoidance of churn as being a good thing in my practical experience. I used to see both sides of the argument, now I am staunchly in the camp that churn is positively associated with the quality and rigidity of our codebases and our systems.

I’d rather tend the garden than pretend it’s a building.

AI Engineering Has Entered Its Audit Era

Ben Halpern — Mon, 12 Oct 2026 21:01:03 +0000

A conference agenda is a pretty decent dataset.

Between what is pitched and what is accepted, it's always an attempt to triangulate on the zeitgeist of the moment. So I dug through the roughly 140 sessions at AI Engineer New York and tried to get a sense of what themes exist at this event that might not have been so collectively front of mind six months ago.

The first finding is the obvious one: this is an agent conference now. Two in five sessions have "agent" in the name. That's table stakes; the interesting part is what people want to do about them.

It's also a finance conference by design, billed as "Where AI Engineering Meets Wall Street." That makes it more useful, not less. Finance is the stress test: an industry where "mostly right" is a liability, every decision needs a paper trail, and someone always asks why. Whatever worries this room now will worry the rest of us soon.

"It works" is no longer the bar. "Prove it" is.

After agents, the biggest cluster is evaluation and verification: nearly 20 sessions, six with the word "trust" in the name. Generation is cheap now. Proving it's right is where the work is. Addy Osmani and Manish Kapur are asking who verifies the software factory, and other talks cover verification loops and agent fixes that quietly break something else.

Finance pushes past correctness into defensibility. One session asks whether you can defend an agent's decision after it passed every eval. That might be the sharpest question at the conference. Vinoo Ganesh is on three sessions, one of which boils down to a single instruction: don't let models invent numbers. Passing evals and earning trust are different things, and the gap between them is where many AI engineering careers will be made.

Agents are becoming principals

About a dozen sessions deal with security, identity, permissions, or payments. We spent twenty years building that stack for humans. Now we're rebuilding it for software that acts on its own. The panel on intent-based auth includes Dick Hardt, who led the design of OAuth 2.0. Others cover agent permissions in regulated industries, financial accounts for agents, and the idea that your next user is an agent.

My favorite pairing: one talk asks why agents don't pay for things even though they can, and another walks through six attacks on x402 agent payments. That's the whole adoption curve in two talks. The rails exist; the trust doesn't yet. If you run an API, ask now what changes when the caller is an agent with a budget.

Intelligence has a price tag

Cost is the other loud signal. There are sessions on tokenomics, the cost of intelligence, and the hidden costs of a faster model. Then Wall Street's own vocabulary shows up: compute markets, the dark pools of compute, and a session from the CEO of Compute Exchange. One of the speaker lounges is literally the Token Billionaires Lounge.

When people describe GPUs in the language of market structure, compute has become a commodity. AI engineers now have to think like CFOs: what does this call cost, and is the better model worth it?

So what do we do with this?

Ryan Carson argues that your job is now agent manager. Laurie Voss is talking about the rise of the product engineer. I think they're right, with a caveat from another session: stop turning your humans into APIs for your agents.

A conference like this, with fairly pragmatic outcomes center of mind, is able to cut through frontier model headlines and help us understand the themes we can apply more agency to. We're trying to do our job as AI Engineers by proving the output, scoping the authority, and paying the bill.

Meme Monday

Ben Halpern — Mon, 12 Oct 2026 14:28:34 +0000

Meme Monday!

Today's cover image comes from the last thread.

DEV is an inclusive space! Humor in poor taste will be downvoted by mods.

You Should Know About System One Models

Ben Halpern — Sun, 11 Oct 2026 19:43:13 +0000

Production AI usually looks like this: stuff context into a prompt, ask a large language model a yes-or-no question, beg it to answer in JSON, then parse whatever comes back and hope.

The punchline is that it actually is incredibly useful to use LLMs for their judgment instead of their writing: is this spam, where does this belong, is this comment actually helpful? LLMs were the best tool for the job, even if they were the wrong tool for the job. The inherent inefficiencies of this idea made scaling an incredibly delicate balancing act of quality, token waste, intelligence and pseudo-determinism.

Then Jev came along, and it seems like the missing piece.

Judgment as a primitive

Jev, from TypeSafe, is a "System One" model. It doesn't generate text. You hand it some state (a comment, a profile, some JSON) and a set of questions, and it returns typed answers with probabilities. There are three shapes: yes/no, pick one from a list, or place it on a scale.

That sounds limiting until you notice how much of our LLM usage fits those three boxes. No parsing, no "respond with only one word." Just a decision your code can branch on, plus a probability that tells it when to act and when to escalate.

How we use it in Forem

In Forem, the open source software that powers DEV, Jev plays two roles.

The first is decision maker. It labels new posts for anti-spam moderation, checks whether a post meets a badge's criteria, picks a highlighting language for unlabeled code blocks, and suggests tags for untagged posts.

The second, my favorite, is traffic cop. Our spam escalation check is a cheap Jev pass that asks a few questions in parallel: is this spam, is it pushing readers to a Telegram or WhatsApp contact, is it promoting something unrelated to this community? It never flags anything itself. It just decides what deserves a closer look from the full check.

Jev is orders of magnitude cheaper than LLMs, and along with lower latency, allows us to inject a layer of intelligence into parts of the system which had to be strictly algorithmic before.

Where this is really headed

These decisions don't need a frontier data center. They need to be fast, consistent, and close to where things happen.

Think about a smart home. Was that a request to turn off the lights, or just a mention of them? Is this motion event worth waking me up for? Those are rigid, reactive, latency-sensitive calls. They should happen locally, without your living room phoning a server farm.

That's why I'm rooting for projects like Laya, an open source take on the idea that you can run yourself. It may not be there yet, but someone will make local judgment models work, and when they do, this whole class of AI moves onto the device.

Practical problem solving

Many production AI use cases wind up as a solution looking for a problem, or simply lack scaled cost-effective usefulness. This tool complements frontier AI intelligence beautifully to maximize the practical roll-out ready usefulness of our systems.

I see the release of Jev as not just a usefulness-first AI innovation, but an articulation for how we should be designing tiered intelligence systems to control for costs and provide algorithmic-like control of our outcomes.

Meme Monday

Ben Halpern — Mon, 05 Oct 2026 13:03:51 +0000

Meme Monday!

Today's cover image comes from the last thread.

DEV is an inclusive space! Humor in poor taste will be downvoted by mods.

Gemini 4 Argon https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-4-argon/

Ben Halpern — Wed, 30 Sep 2026 20:31:35 +0000

Introducing Gemini 4

Announcing Gemini 4 Argon, our frontier model for real-world coding, enterprise knowledge work and cyber defense, rolling out soon.

blog.google

Meme Monday

Ben Halpern — Mon, 28 Sep 2026 15:13:35 +0000

Meme Monday!

Today's cover image comes from the last thread.

DEV is an inclusive space! Humor in poor taste will be downvoted by mods.

Meme Monday

Ben Halpern — Mon, 21 Sep 2026 13:17:30 +0000

Meme Monday!

Today's cover image comes from the last thread.

DEV is an inclusive space! Humor in poor taste will be downvoted by mods.

Meme Monday

Ben Halpern — Mon, 14 Sep 2026 13:08:59 +0000

Meme Monday!

Today's cover image comes from the last thread.

DEV is an inclusive space! Humor in poor taste will be downvoted by mods.

No Meme Monday

Ben Halpern — Mon, 07 Sep 2026 15:23:29 +0000

Labor Day in the US ✌️

💎 Introducing Community Gems: Celebrating Human Curation and the Best of Our Community

Ben Halpern — Wed, 02 Sep 2026 19:28:32 +0000

In an internet full of noise, infinite scrolls, and algorithmically generated content, finding genuinely great, thoughtful writing can feel like searching for a needle in a haystack. We are introducing a new feature we hope will do a bit to make our community more community-oriented.

Algorithms and engagement metrics have their place, but they will never replace human taste. Some of the most valuable articles and comments across our platform aren't necessarily the ones with the loudest headlines or the most clicks—they're the pieces offering deep technical insight, nuanced perspectives, thoughtful answers, and genuine craft.

Today, we're introducing Community Gems: a human-first curation system designed to spotlight, celebrate, and reward exceptional contributions.

We have always had, and will continue to have folks doing moderation, but this is a new dynamic which just takes a different approach. It complements our moderation systems in general, but we also be pruning and simplifying how moderation works as part of the march forward for a healthier ecosystem.

How Community Gems Work

We've granted a small, hand-selected group of trusted tastemakers and community leaders the Community Curator privilege.

Here is how the system works:

1. Limited Daily Curation

A Daily Ceiling of 5 Gems: Each designated curator gets up to 5 gems to award per day.
Scarcity by Design: By capping the daily allowance, every gem given is intentional and meaningful. We aren't creating another generic "+1" or like button—gems represent a deliberate nod from a respected peer that a post or comment is truly exceptional.
Dynamic, Balanced Groups: Only designated curators receive daily recurring gem allowances. Over time, we may expand, rotate, or refine this curator group to preserve balance, diverse points of view, and high community quality.

2. What Happens When Your Post Gets Picked?

If community curators pick your post or comment as a gem:

💎 Milestone Profile Badges: You’ll receive real-time notifications tracking your gem journey. Earning gems unlocks a sequential milestone badge system on your profile (starting at 2 gems, followed by 4, 8, 16, 32, and 64).
📈 Increased Trust & Reputation: Recognition by curators directly boosts your author trust score across the platform.
✨ Curated Visibility: Gem-awarded posts will appear in a dedicated feed which members can switch to at any point. Our feed discovery algorithm is also taking curation signals into account (with the exact long-term weighting evolving as we learn).
🎁 Passing It Forward: Earning a gem also grants you the ability to award a gem to another author's post. This does not mean you become a community curator, but you accumulate a stock of gems you can pass along. This feature is not fully live yet.

The "Why": Leaning Into Human Taste

Our platform already has mechanisms to help separate signal from noise—reactions, comments, bookmarks, and search algorithms.

Community Gems operates under the hood on a different philosophy: valuing human taste and the scarcity of quality. By empowering trusted community members to curate the best hidden gems, we're returning the focus to craft, thoughtful sharing, and authentic discovery.

Keep writing, sharing, and commenting—we can't wait to see which gems you discover and create! 💎

Happy coding!

Fixing Delicate Cache Mismatches in a Brownfield SPA: A Pragmatic Solution

Ben Halpern — Mon, 31 Aug 2026 19:21:38 +0000

Building in public often means talking about shiny new features, but in mature production applications, the most critical engineering work is usually brownfield problem solving—iterating on top of years and years of code logic and/or production system decisions.

On DEV (powered by the open-source Forem codebase), we have a hybrid architecture that blends Rails server rendering, Fastly edge caching, and lightweight client-side navigation (via InstantClick). This setup delivers sub-100ms page transitions, but partial page swaps combined with aggressive edge caching create delicate deployment challenges.

We shipped a fix PR #23789 which helps solve for an inherently delicate caching mismatch issue in web navigation. This is something that has existed practically since the beginning of DEV, and we have had several flakey fixes for the problem at times, but I feel good about this being a step in the right direction — albeit not a perfectly complete fix by any means.

The Problem: Cache Mismatches Across Deployments

When we deploy new CSS updates, Rails generates new asset digest hashes (e.g., views-v2.css replaces views-v1.css).

Full Page Load: A user loads an updated homepage. The browser receives full HTML and loads the latest v2 stylesheets in <head>.
Internal Navigation (Partial Page Swap): The user clicks an article link. Rather than performing a full browser reload, InstantClick requests the article in the background and swaps only the inner #page-content container into the existing DOM.
The Edge Cache Trap: DEV aggressively caches article HTML fragments at Fastly with surrogate keys. Many articles were cached before the latest deploy, meaning the cached HTML fragment still referenced v1 stylesheets.

Why the Previous Solution Was Fragile

To prevent pages from rendering with outdated CSS, we had previously added client-side logic to inspect the incoming page's expected stylesheet paths and swap <link rel="stylesheet"> elements in the DOM dynamically.

In practice, this was brittle:

When navigating from the fresh homepage (v2) to an older edge-cached article (v1), the client script compared currentHref (v2) !== expectedHref (v1).
It assumed the incoming page represented the "target" state and initiated an involuntary downgrade of the DOM back to v1 stylesheets.
If v1 asset files had been pruned after deployment, the browser failed with 404s; if they loaded, new UI components suddenly broke because their updated CSS was ripped out mid-session.
Attempting to asynchronously swap multiple stylesheet tags across both <head> and <body> introduced CSS cascade race conditions and flashes of unstyled content (FOUC).

The Solution: Repurposing Internal Navigation Parameters

Instead of treating this as a client-side DOM mutation problem, we realized it was fundamentally a cache partitioning problem.

For years, Forem has quietly passed an internal query parameter—?i=i—on background AJAX requests so Rails knows to render a lightweight partial layout instead of the full HTML document shell.

In PR #23789, we repurposed this parameter:

Combined Style Fingerprinting: On initial full-page load, Rails computes a deterministic 10-character hash from the combined digests of the core stylesheets (minimal, views, and crayons) and places it on <body> as data-style-fingerprint:

   <body ... data-style-fingerprint="cc9ed033eb">

Parameterizing Internal Navigation: When InstantClick preloads or fetches a link, it reads the active session's fingerprint and appends it to the internal URL:

   https://dev.to/user/post?i=cc9ed033eb

(If data-style-fingerprint is missing—such as on tabs open during deploy cutover—it gracefully falls back to ?i=i).

Natural Cache Partitioning at Fastly: Fastly's edge configuration already whitelists i in its safe parameter list. Because the query parameter is part of the cache key:
- When a user on a v2 session requests /post?i=v2_fingerprint, Fastly checks for a cached v2 fragment.
- If missing, Fastly fetches a fresh fragment from Rails compiled with v2 styles.
- Stale v1 cached fragments are simply bypassed and eventually expire naturally.
Deleting the DOM Swapping Logic: With cache partitioning in place, incoming partial fragments are guaranteed to match the active DOM's stylesheet version. We completely deleted the client-side stylesheet replacement script. The browser's active <link> tags remain static throughout the user session.

Brownfield Development Takeaways

Solve at the Cache Layer, Not the DOM Layer: Trying to coordinate DOM mutations, asynchronous CSS loading, and cascade precedence in client-side JavaScript is almost always more fragile than letting the edge cache serve the right HTML version from the start.
Repurpose Existing Primitives: We didn't need a new router, an edge computing rewrite, or custom HTTP headers. Repurposing our existing ?i=... internal parameter gave us exact cache keying with zero extra infrastructure overhead.
Pragmatic Improvements Over Perfect Rewrites: This fix doesn't magically prevent every theoretical edge-case during rolling deploys, but it definitively solves the most disruptive issue. The system is far less delicate, and we now have a clean, predictable pattern to build upon.

Happy coding!