In 2004, a developer at Sun Microsystems was tired of breaking builds. He wrote a small tool as a side project to catch his own mistakes. That tool became Hudson, then Jenkins, and it ended up teaching the entire industry a habit: run the tests on every commit, automatically, forever.
Twenty years later, the same developer, Kohsuke Kawaguchi, co-founded a company whose core product is built on a very different idea. Use machine learning to predict which tests will fail for a given change, and run only those. His early customers reported cutting test execution time by 90 percent. The tool now ships as CloudBees Smart Tests.
The man who made it cheap to run ten thousand tests per commit is now working on running a fraction of them. That is not a contradiction. It is the same problem seen from both ends, and it tells you something honest about where AI actually fits in developer tooling. Last week he sat down with Josh Long on the A Bootiful Podcast to talk about the journey, and the arc is worth unpacking.
The tool that invented the habit
Kawaguchi created Hudson at Sun in 2004 because continuous integration servers at the time were heavy, expensive enterprise products. His insight was to make automation a free, pluggable server any team could stand up in an afternoon. Within a few years, Hudson was the default CI server of the Java world.
Then came one of the strangest chapters in open source history. Oracle acquired Sun in 2010 and claimed the Hudson trademark. In January 2011, the community voted on a rename: 214 votes for renaming the project to Jenkins, 14 against. Around 85 percent of the developers moved to Jenkins, along with 21 of the top 25 plugins. Oracle kept Hudson, donated it to Eclipse, and by 2017 it was declared obsolete. Jenkins, the "fork" that almost everyone followed, never looked back.
On his own LinkedIn profile, Kawaguchi describes the result plainly: an ecosystem with more than 1,500 plugins that has remained the most popular CI/CD software for over a decade. Whatever you think of Jenkins' aging architecture, that kind of longevity is almost unheard of in developer tools.
But here is the part that matters for this story. Jenkins did not just automate builds. It changed what teams believed they could afford. When running the full suite on every commit became free, suites grew. Nobody had to watch the clock anymore, so nobody pruned.
The problem Jenkins created
Fast forward to 2026 and the arithmetic has flipped. A CI pipeline that takes 10 minutes keeps a team fluid. A pipeline that takes 90 minutes does something worse than waste money. It changes developer behavior. People batch their changes. They merge large branches to amortize the wait. Feedback loops stretch from minutes to hours, and the whole point of continuous integration quietly erodes.
The industry's first answer was brute force: more parallel executors, more machines, sharded suites. That scales the bill along with the minutes. The second answer is the one Kawaguchi chose: stop pretending every test has an equal chance of failing on every commit.
That was the founding insight of Launchable, the company he started in 2019 with Harpreet Singh. Their observation was that Facebook had built predictive test selection internally years earlier as a private optimization. The technique worked, but ordinary teams had no access to it. Launchable's pitch was to turn it into a product any CI pipeline could call.
How predictive test selection actually works
The mechanism is less magical than the marketing suggests, and that is a compliment. From Launchable's own documentation and product descriptions:
- The model trains on your history. Every build, test session, and pass/fail result from your CI gets recorded. There is no giant generic model of "which tests usually fail". The signal is the team's own track record.
- Each commit gets a scored subset. When you push, the system looks at which files changed, cross-references them against the historical relationship between changes and test failures, and returns a ranked list of the tests most likely to fail for this specific change.
- You run a slice, with a confidence dial. Your pipeline asks for the top N percent of the suite. If the model is confident, you can request a smaller slice. One of the features Launchable's team was proudest of was the "confidence curve", a visualization of the model maturing as it accumulates your data.
- It is CLI-first and language-agnostic. The CLI wraps a REST API and integrates with more than twenty test runners across Java, Python, Ruby, Go, JavaScript, .NET, and Perl.
The numbers came from early customers. When CloudBees acquired Launchable in August 2024, their press release cited BMW and GoCardless reporting a 50 percent reduction in machine hours, a 90 percent reduction in test execution times, and a 40 percent reduction in build times. Those are vendor-reported figures from named customers, not independent benchmarks, but they are specific enough to take seriously as a direction.
The catch nobody puts in the launch post
A model that predicts failures is, statistically, allowed to be wrong. If it ranks a test as "unlikely to fail for this change" and skips it, and that test was actually broken by your commit, you find out later. Usually the next night, when a scheduled full run catches it. If your release cadence is measured in weeks, that delay is fine. If you ship to production within the hour, it may not be.
Three risks deserve their own lines:
- Cold start. The model knows nothing on day one. You need weeks of recorded history before the confidence curve means anything. Small projects with a few hundred tests will not accumulate useful signal.
- Drift. Refactor your architecture or introduce a new module, and the learned relationships between changes and failures shift. The system needs to keep relearning, and you need to notice when its accuracy sags.
- Coverage blindness. Predictive selection optimizes for finding failures your history says are likely. Tests that fail rarely sit at the bottom of the ranking forever, which is exactly where the tests guarding against rare regressions live. A scheduled full run is not optional; it is the safety net that makes the optimization safe.
Full disclosure: I have never run Launchable or CloudBees Smart Tests in production. Everything above comes from their documentation, the CloudBees press release, and Kawaguchi's own public writing. My own projects run small suites on GitHub Actions where the entire pipeline finishes in minutes, and for that shape of project this product category genuinely does not pay off. This matters, because the decision is about suite size, not fashion.
The philosophy underneath, and why it is the interesting part
In the recent podcast conversation with Josh Long, Kawaguchi made a point about AI in software development that doubles as a summary of his career. The episode notes, and his own argument, land on this: AI is changing software development fast, but the systems that win may be the ones that bring intelligence into existing workflows, rather than forcing everyone into one giant chat interface.
Look at what that means concretely for testing. Predictive test selection does not ask a developer to do anything new. You do not paste code into a chat window. You do not learn a prompt format. Your pipeline still runs tests the way it did in 2011; the only difference is that a learned model decides which slice of the suite this commit deserves. The intelligence sits inside the workflow you already have.
That is a sharper take than it first sounds, because it cuts against most of what ships under the "AI for developers" banner right now. The default pitch is a new surface: a new chat, a new dashboard, a new place to check. Kawaguchi's bet, backed by two decades of watching tools either embed or die, is that the durable form of AI is invisible. Jenkins won because it fit inside the build. Smart Tests wants to win because it fits inside the pipeline. Nothing to adopt, nothing to leave.
A checklist: does this fit your team?
Since the honest answer to "should you use this" is "depends", here is the decision list I would actually walk through:
- Is your suite over 30 minutes? Below that, predictive selection saves minutes. The operational overhead of recording, training, and monitoring a model will not be worth it.
- Do you have hundreds of commits per week? The model feeds on history and gets retrained by new outcomes. Low-commit repos starve it.
- Do you have at least a few months of CI data to train on? No history, no signal. Nothing to do here except start recording now, which costs nothing.
- Can you tolerate late detection on rare failures? If every merge is a same-day production deploy, weight that cost honestly against the minutes saved.
- Is someone on the team willing to watch the confidence curve? A test-skipping model nobody monitors is a slow leak in your safety net. Assign an owner or skip it.
If most answers are no, the boring alternatives still work: split the suite into a fast commit-level tier and a slower nightly tier, kill flaky tests instead of retrying them, and cache what can be cached. Kawaguchi's company is aiming at the teams for whom those stops are no longer enough.
The full circle
There is a neat symmetry in this story. In 2004, the barrier to testing was cost, and Kawaguchi removed it. The industry responded by building suites so large that cost became the barrier again, just measured in hours instead of licenses. Now the same person is attacking the new version of the old problem, with a technique that only exists because a million Jenkins pipelines spent twenty years recording exactly the data such a model needs.
Whether Smart Tests specifically wins matters less than the pattern: the next useful wave of AI in developer tooling probably will not look like a chat window. It will look like your existing pipeline, quietly running less and catching the same failures.
I write about Java, developer tooling, and AI every week. If that is your corner of the world, subscribe, it is free.
Do you still run Jenkins, or has your team moved to GitHub Actions or something newer? And more importantly: how long does your full suite take on every commit? I am collecting war stories in the comments.
Top comments (0)