DEV Community

Cover image for We replayed 1,201 releases of popular Java libraries: 1 in 9 contains a breaking change
Gilware
Gilware

Posted on Originally published at gilware.com

We replayed 1,201 releases of popular Java libraries: 1 in 9 contains a breaking change

Semantic versioning promises that minor and patch releases are safe to install. We wanted to know how often that holds today, so we replayed every minor and patch release published since January 2024 by 44 of the most-used Java libraries on Maven Central: 1,201 releases in total.

About 1 in 9 of them (141 of 1,201, 11.7%) contains at least one breaking change in its public classes.

The full report, with the per-library table, the data and every caveat, is here: How often does a "compatible" release break its users? This post is the short version.

How we counted

  • The sample was not ours to pick. We used the list of most-used Maven Central libraries that the authors of the Roseau paper published in 2025. Of its artifacts, 44 contain code and had minor or patch releases since 2024.
  • Each release was compared with the previous version, jar against jar, bytecode against bytecode. Pre-releases were left out. A change in the second version number is a minor release; anything after that is a patch.
  • Two tools had to agree. Every pair was checked by two independent checkers, japicmp (0.27.0) and the Bumpwise engine (1.0.6). A release counts only when both flag it.
  • Internal code does not count. Packages named internal, impl or shaded, repackaged third-party code, packages a library's module-info does not export, and code marked @Beta or @API(status = INTERNAL / EXPERIMENTAL) were excluded.
  • "Breaking" means binary or source incompatible: code compiled against the old version fails to link with the new one, or code written for the old version no longer compiles. Counting binary breaks only, the figure is 8.8%.

The numbers

Releases With a breaking change
Minor releases 163 58 (35.6%)
Patch releases 1,038 83 (8.0%)
All minor and patch 1,201 141 (11.7%)

28 of the 44 libraries shipped at least one such release in under three years. The median library is at 6.7%.

Minor releases carry most of the breaks. In these libraries the last number of the version means something: a patch release is about four times less likely to break than a minor one.

It used to be worse

We ran the same replay back to 2019 (3,096 releases):

Year Releases With a breaking change
2019 327 20.5%
2020 422 17.3%
2021 357 14.0%
2022 351 19.7%
2023 438 18.5%
2024 475 12.2%
2025 378 11.4%
2026 (to 8 Nov) 348 11.5%

From about 1 in 6 before 2024 to about 1 in 9 since. Better, not gone.

What breaks

Out of the 141 releases (one release can have several kinds):

Kind of change Releases
Method removed 62
Abstract method added (to a type users implement) 45
Method parameters changed 38
Class removed 30
Field removed 18
Generic types changed 16
Return type changed 15

Nothing exotic. These are removals and changed signatures: changes that compile and pass the library's own tests, and only fail in someone else's build or at runtime. If some of them surprise you, the first two posts in this series walk through the Java and Kotlin cases one by one.

Small libraries: more than 1 in 4

The 44 libraries above are maintained by foundations, companies and experienced teams. We also took a random sample of small libraries published under io.github.* (the namespace Maven Central gives to a GitHub account): 162 libraries with releases of version 1.0 or later.

Most-used libraries Small libraries
Minor releases with a break 35.6% 29.5%
Patch releases with a break 8.0% 28.2%
All minor and patch 11.7% 28.5%

The interesting line is the patch row. In the small libraries, a patch release breaks as often as a minor one: the version number goes up, but it no longer says what kind of change is inside.

Read this second set with more care: we did not check which of these projects promise semantic versioning, a break in a small library reaches fewer people, and ten libraries account for 206 of the 506 flagged releases (21.0% without them).

What these numbers do not say

  • Not "violates semver". A tool sees what is public in the bytecode, not what a project promises. We leave out @Beta, @API(status = INTERNAL) and packages not exported by module-info, but some projects document what is stable in other ways (a FAQ, their own annotations), and we did not read each project's policy. Three libraries account for 53 of the 141 (8.7% without them).
  • Not "broke its users". We did not measure how many clients actually call the changed code. The largest earlier study of Maven Central (Ochoa et al., 2022) found that only 7.9% of clients were affected by the breaking changes it found.
  • Not "Maven Central". This is 44 popular artifacts from about 25 projects, plus one family of small namespaces.
  • Two tools agreeing is not proof. We looked by hand at every case where they disagreed (one of those turned out to be a real break that one tool missed, which is the next post in this series), but we did not build a failing client for every flagged release.

Why it matters anyway

Most of these breaks never hurt anyone. But a modern application pulls in hundreds of libraries, and "it's only a minor update" is the assumption behind every automated dependency bump. When the version number is wrong, the cost lands on the user of the library, who did nothing but upgrade.

Tomasz Nowak and Predrag Gruevski, authors of the best-known study of this (for Rust, 2023), put the conclusion better than we can: "The maintainers are not to blame, and improved tooling is our best way forward." Every change on the list above is detectable before the release, by japicmp, Revapi or Kotlin's binary-compatibility validator in CI, or in the editor.


Disclosure: we make Bumpwise, an IntelliJ plugin that flags these changes while you type, and its engine was one of the two tools in this replay. The method, data and per-library results are public on the report page. If you maintain one of these libraries and think we got your numbers wrong, tell us and we will correct the report.

Top comments (0)