DEV Community: Axelix Labs

The Tragedy of Software Versioning

Sergei Cherkasov — Mon, 21 Sep 2026 15:13:25 +0000

Software versioning goes far beyond SemVer. A practical guide to CalVer, marketing versions, lockstep releases, pain indexes, and compatibility windows.

As we say, two of the most complicated problems in software are naming things and versioning software

Thomas Wuerthinger, the GraalVM project lead, once said something along those lines in the corridors of Devoxx Belgium while explaining the details of this much-discussed article. It is not an exact quote, just a phrase meant to draw attention to the problem. And that problem is what we are going to discuss today.

Hi everyone! My name is Mikhail Polivakha, and I am the technical lead of the Open Source Axelix project.

I have wanted to write a short article about software versioning in general for quite a while because, in my view, the subject gets far less attention than it deserves. For many people, versioning begins and ends with SemVer, but the topic is much, much broader than it might seem.

This article will be useful to technical leads and those aiming for a Staff Engineer position, because your software versioning strategy, its relationship to your release cycle, and the nature of your software updates are very, very important product decisions. Getting them wrong will be costly. Should you use a Release Train? Should you work in lockstep? Do you need a Compatibility Window? Do you need a compatibility matrix? And so on.

You may want to bookmark this article and return to it from time to time. Without further preamble, let us begin.

In the Beginning Was the Word

Our software evolves. We add new capabilities, fix bugs, learn new things about the domain, and discover how users actually use the product. Even if a product is functionally complete, one day it will still need an update because of some zero-day CVE.

As a result, we end up with different states of the same product. Yesterday a feature did not exist, today it does, and tomorrow an old API is marked as deprecated. We need a way to name each of these states precisely.

That is what versions are for.

But this is where things get interesting. There are actually a great many questions:

What is a version, anyway? Is it a number, a string, or a sequence of characters?
When should you release a new version?
If the software consists of several independent parts, how do the versions of those parts relate to one another?

In fact, the requirements for a version depend less on the programming language or build system than on how the product reaches the user and who depends on it.

The version of a desktop application, the version of a Java library, and the version of a twenty-component APM platform like Dynatrace solve three different problems. You can try to apply one scheme to all of them mechanically. Is that a good idea? Let us find out.

A Relatively Simple Case: A Desktop Application

Imagine a regular desktop application for an end user. Not an IDE with a vast plugin ecosystem, not an operating system, and not Excel, around which a company has built half of its processes. Just an application.

Such a product, as a rule, has two important characteristics:

1. The primary consumption model is that the user interacts with the product rather than compiling their own code against its API.

In that case, a backward-compatibility break often looks like this:

Yesterday I saw this button on this page, but today it is gone—it is on another page

In other words, the contract is often not between programs, but between the UI and the user's eyes.

2. There is no large dependency graph of other libraries between the application and the user.

Again, desktop applications vary. Some have many integrations with external systems—Excel, for example—but "many" is a relative term, and every product is unique here.

CalVer. Common Case

In the general case, Calendar Versioning, or CalVer, works quite well. It is a version format in which the release date is encoded into the version in one way or another. For example, the CalVer version 2026.1 might mean the first release of 2026. JetBrains IDEs use a similar scheme.

One well-known example of software that follows CalVer is Canonical's Ubuntu. Ubuntu is released twice a year, in April and October, so its versions follow this format: 22.04 is the April release, while 22.10 is the October release.

This is convenient for end users. The version number immediately answers a question they understand: how recent is the release in front of me, and when did it come out?

Note that CalVer does not say that a product is stable or backward-compatible with anything. It communicates the release date. For consumer software, that is often enough and generally a good choice, because users simply do not have to assess the compatibility of individual libraries.

Marketing Version

But there is a separate class of desktop software that releases relatively infrequently and treats each release as a major event. In those situations, what matters is not so much when the product was released, but the very fact that a release happened at all.

For example, Microsoft quite successfully released Windows 7, 8, 10, and 11. This approach is often called Marketing Versioning, because its primary purpose is marketing—the ability to announce loudly:

Yesterday you had Windows 10, and now, after several years of development, Windows 11 is coming out!

That release will be accompanied by an entire marketing campaign. It is not just another Ubuntu release.

For end users, it is easier to look at 10 and understand that it is less than 11 than it is to compare Ubuntu 22.04 with 24.10. Of course, Marketing Versioning is a very niche approach for major, foundational software products, but it is justified because, from a marketing perspective, people find a single number easier to understand and compare.

Again, this does not mean that every desktop application must use CalVer or Marketing Versioning. My point is simpler: if the version number's main value to the user is distinguishing a recent release from an old one, and releases arrive relatively regularly, a calendar-based scheme solves the problem honestly. If you plan to release very rarely and treat each release as an event in its own right, Marketing Versioning may be worth considering.

When Others Begin to Depend on You

Let us move on and change the situation slightly. We are developing a library rather than an application.

The situation is fundamentally different here. The user does not merely run our product. They compile their code against it and use our classes, methods, annotations, and configuration properties. Our library becomes part of their program.

Now the user is interested in more than how recent the release is. Before upgrading, they want to know:

whether compilation will break;
whether runtime behavior will change;
whether the configuration must be rewritten;
whether a patch can be accepted automatically through Dependabot;
how much time to budget for migration.

Semantic Versioning. The Cornerstone

This is where the familiar Semantic Versioning, or SemVer, enters the picture. Many projects follow its format:

Spring Boot
Quarkus
Jackson, and so on.

Axelix follows it as well, but more on that later. In its classic form, a version has the format MAJOR.MINOR.PATCH:

MAJOR increases when a backward-incompatible change is made to the public API;
MINOR increases when backward-compatible functionality is added;
PATCH increases when a backward-compatible fix is made.

A version may also have a qualifier, though this is optional. Here are a few widely known examples:

Alpha/Beta versions, such as 8.0.0.Alpha1. Hibernate adds qualifiers like these for alpha and beta testing. We will not go into that now.
Milestone versions, such as 4.0.0-M1. We will not explore the purpose of milestone releases either, but I do want to note that large projects such as Spring Boot and Axelix sometimes use them to gather early feedback.

For example, moving from 2.4.1 to 2.4.2 should be boring. Update the dependency, run the tests, and move on. But moving from 2.4.2 to 3.0.0 warns you in advance: folks, some work may be required here.

That is the key value of SemVer. The version number becomes a cheap protocol between the maintainer and the user. You do not need to read the entire changelog just to estimate the scale of a migration. The number itself provides an initial estimate of the migration's complexity.

SemVer Requires a Public API

There is a nuance that people sometimes forget.

If you actually read the specification, SemVer is built around what it calls a "public API." The specification says so explicitly. This is logical: otherwise, it would be impossible to determine whether a change is backward-incompatible. And here is my question for you.

What counts as the public API of a Spring Boot starter?

Only its Java classes? What about configuration properties? Are bean names part of the contract? What about the format of an actuator endpoint? The order in which auto-configurations are discovered? The proxying strategies used by auto-configuration? As the saying goes:

It all looked smooth on paper, but they forgot about the ravines

On paper, the rule "break the API, increase MAJOR" looks simple. But in a real framework, the compatibility surface is enormous.

Sometimes even a critical bug fix in a patch release changes behavior that someone has already come to rely on. Sometimes a CVE cannot be fixed without tightening old rules. Sometimes a major upgrade to a transitive dependency breaks things for the user even though your own Java API has not changed at all.

Experienced engineers need to remember that following pure SemVer is extremely difficult in practice. You are unlikely to sustain it over the long term in a complex product.

It is therefore important to understand that a three-number product version does not necessarily mean that the project follows strict SemVer. And rather than make an unsupported claim, let us look at an example. For many people, it may be an eye-opener.

Spring Boot and the Pain Index

Spring Boot is a very good example.

Spring Boot uses the familiar MAJOR.MINOR.PATCH format, but the project team states explicitly that strict Semantic Versioning is practically impossible for Boot. Otherwise, the major version would have to increase far too often.

Instead, the version number is used as a kind of expected pain index for an upgrade:

a patch should be as close as possible to a drop-in replacement;
a minor release introduces new capabilities and usually requires little upgrade effort;
a major release is reserved for significant breaking changes where migration work is expected.

This resembles SemVer, but it is not strict SemVer. A Spring Boot minor release may contain incompatible changes. Moreover, Boot manages the versions of a large number of third-party libraries, and upgrading Boot itself updates that entire set.

My advice is this: the scheme above is viable, and if you are considering SemVer, take it seriously. On the whole, it fulfills the role SemVer was designed to fulfill: letting users understand the scale of the changes relatively quickly—and roughly how painful the move will be—without diving into release notes, rebuilding things, and so on.

Strict SemVer simply prescribes: Major == backward-incompatible changes. In practice, the formula looks like this:

Patch: upgrade; everything will be fine.
Minor: you may have to put in a little effort, but it will probably be no big deal.
Major: you will definitely need to spend time on migration.

Versioning exists for users, not users for versioning. If a strict scheme forces a mature framework to release Spring Boot 47, then perhaps that scheme does not describe the reality of that particular project very well.

What matters is that the promise be stated explicitly. The user should understand that 3.4 -> 3.5 means:

"Most likely, the upgrade will be relatively straightforward."

It does not mean a mathematical guarantee that there are no incompatible changes.

So, friends, do not confuse SemVer's syntax with its semantics. You can put 1.2.3 on anything. It becomes valuable only when a project has explained the rules and genuinely tries to follow them.

What If the Product Has Multiple Components?

Now let us add even more interesting details to the equation. A modern software platform rarely consists of a single artifact.

Take any APM solution, such as Dynatrace or Datadog. They have a server and separate agents that collect information—different components. Take Kubernetes: it has a kubelet on every node, a Control Plane, and so on. These are all different parts of a single product.

At Axelix, for example, we have:

build plugins for Maven and Gradle;
a Spring Boot starter;
Axelix Master as a standalone executable JAR;
a Docker image for Master;
a Helm chart.

Importantly, all these "components" are generally distributed separately and may be owned by different teams. In our case, for example, the starter and plugins arrive in a service from Maven Central. They are simply Maven artifacts maintained by a separate product team.

Axelix Master, meanwhile, is deployed by the platform/ops team. The Docker image and Helm chart even live in their own distribution channels.

This architecture means that, technically, each component can have an independent version and change at its own pace. For example, nothing may have changed in the starter while we fixed the UI in Master. Or, conversely, we may need to release a patch for a build plugin without touching the server.

This leaves us with a choice between two strategies.

Independent Versioning

Each component gets its own version:

Master 3.7.1;
starter 2.4.0;
Maven plugin 5.1.3;
Helm chart 1.9.2.

From our perspective as maintainers, this is a clean and logical model. When one component changes, we increase only its version according to that "Pain Index". There is no need to publish artifacts whose code has not changed. It makes life easier for us maintainers.

But now the complexity has shifted to the user.

Is starter 2.4 compatible with Master 3.7? Which plugin does starter 2.4 need? Can chart 1.9 be used with image 3.7.1? The result is a "compatibility matrix" alongside the product—one that must be documented, tested, and continuously maintained.

A very good example is the long-running saga between Spring Boot and Spring Cloud. Because they are technically separate projects with their own versioning strategies, they have a Compatibility Matrix that tells you whether a particular Spring Cloud version can be used with a particular Spring Boot version.

And here is the question: is having a Compatibility Matrix inherently bad? You are software users yourselves. Ask yourself: how enjoyable is it to figure out every time whether this version of component A is compatible with that version of component B? It is hardly the most pleasant way to spend your time.

Independent versions make life easier for the release team, but increase the number of states the user sees. Users already have plenty to worry about, and the last thing they want is to search for which version of component A is compatible with a particular version of component B.

Lockstep Versioning

The second strategy is called lockstep versioning. All components of a product are released under the same version.

If we release Axelix 1.4.2, then Master, the starter, plugins, image, and chart all receive version 1.4.2. This strategy is used by modules within projects such as:

Spring Boot (actuator, autoconfigure, plugins, and so on)
React's core modules (react, react-dom, and so on)
Quarkus's core modules

There is a very important distinction here. SemVer and CalVer answer the question, how is the version number structured? Lockstep answers a different question: which components share that number?

So you can absolutely use SemVer together with lockstep. Or CalVer together with lockstep. These are not competing concepts. The advantages of lockstep are clear:

1. The user deals with a single product version.

This is extremely convenient. You need to know only one version. When you ask a colleague:

"Which Spring Boot version does your microservice use?"

You have to remember that Spring Boot is an umbrella for many modules, yet everyone answers 2.7 or 3.5: people deal with a single version. It is just ridiculously convenient. Likewise, the question "Which version of Axelix do you use?" can be answered with 1.1.

2. The compatible combination of components is obvious.

Users understand that with Axelix Master version 1.1.0, they can use Spring Boot Starter version 1.1.0 (in reality, it is a little more complicated; more on that below).

3. Documentation, examples, and release notes are tied to a single number.

For platform products, these are very strong advantages. But there is no free lunch, of course: lockstep has significant problems of its own.

The Cost of Lockstep

Experienced engineers know that lockstep is a rather controversial versioning strategy for related components. It has two classic problems.

1. A backward-incompatible change in one component formally affects the version of the entire product.

In other words, as soon as you realize that component A needs a major version bump, another realization follows immediately: the entire product needs a major version bump.

Suppose we break only the public API of the Maven plugin. If the project follows strict SemVer and all components move in lockstep, Master, the starter, the Docker image, and the Helm chart all need a major version bump even though none of them broke.

This problem can be partially mitigated by following not SemVer in its strict sense, but specifically the Spring Boot team's interpretation of SemVer that I described above.

2. Patching one component results in a patch release of the entire set.

Suppose a CVE exists only in a web dependency of Master. The starter does not have that dependency and has not changed at all. But in the lockstep model, the new product release will still be, for example, 1.4.3 for every artifact.

This is convenient for users. For release engineering, it means additional work:

build all components;
run the shared pipeline;
publish several types of artifacts;
update the documentation in sync;
make sure no registry remains on the old version.

In my experience, this too can be mitigated to a large extent.

Lockstep with Monorepos

Lockstep works well for monorepositories that use Trunk Based Development or GitHub Flow. That is why lockstep was such a good fit for Spring Boot—a monorepo using GitHub Flow—and for Axelix—a monorepo using a slightly modified form of TBD, which I covered in a separate article. If a project is organized as a monorepo using TBD, lockstep often becomes not merely simpler, but the natural strategy.

That is why, as my grandfather used to say, humanity's three most important discoveries are fire, the wheel, and the monorepository.

There is one more problem that has no simple solution: communication. In practice, lockstep requires very smooth coordination among the teams developing the different components of your platform, because the entire platform is released under one version.

But let me remind you that Open Source is developed by small teams. The core teams behind Open Source solutions are small!

Axelix: a core team of four people and dozens of external contributors
Spring Boot: a core team of, at the moment, roughly 5–7 people, plus dozens of external contributors

And because the entire platform is generally developed by one small team, lockstep does not create much communication overhead for that team when coordinating changes, releases, and so on.

So if your team is small, you probably will not notice lockstep's communication overhead. If you are developing a platform product, user convenience is often more important than the elegance of internal release processes, especially when those processes can be simplified through the structure of the codebase.

Nevertheless, that is still not the whole story...

A Lockstep Release Does Not Mean a Lockstep Upgrade

This is where we reach the most interesting part.

You can release all components under version 1.4, but you cannot force an organization to upgrade them simultaneously. For example, Axelix Master is controlled by the platform team. It can upgrade a single deployment in one evening; the documentation describes a dedicated procedure for this.

The starter and build plugin, however, live in dozens or hundreds of services. Every service has its own team, backlog, release window, and adventures.

If moving Master from 1.3 to 1.4 requires a hundred services to be upgraded in the same minute, then the system formally uses lockstep versioning but, in practice, never gets upgraded at all. Such a system cannot realistically be upgraded—it is operationally impossible to make dozens of teams update the versions of various components simultaneously.

In other words, as a Technical Leader or senior engineer, you need to think beyond how you will release versions, what those versions will look like, and how people will perceive them. You must think in advance about how people will upgrade your product in their environments.

Let me share how we approached this in Axelix and how major projects handle it, using Kubernetes as an example.

Introducing Compatibility/Skew Window

To solve this problem, you create what is known as a Compatibility Window. Do not confuse it with a Compatibility Matrix.

Even the most nontrivial systems made up of multiple components have a clear upgrade order: first component A is upgraded, then component B.

The component upgraded first can often be upgraded relatively atomically. In Axelix, for example, Master is always upgraded first. Updating it is fairly simple: you ask the platform team to perform the version upgrade in the evening.

That first component—in our case, Master—must then guarantee a rolling compatibility window, often phrased as follows:

Master supports starters from the latest N releases (excluding patches), including its own release.

For us, N == 4 at the moment. Here is what that looks like in a concrete example:

Starter version	Supported by Master 1.4
`1.4.x`	Yes
`1.3.x`	Yes
`1.2.x`	Yes
`1.1.x`	Yes
`1.0.x`	No

This is a compatibility window directed backward: a new Master understands old starters. You cannot run a starter that is newer than Master.

Existing Example. K8S Skew Policy

A similar principle appears in the official Kubernetes Version Skew Policy:

kubelet cannot be newer than kube-apiserver;
kubelet can be up to three minor versions older than kube-apiserver;
kube-apiserver is upgraded first, and only then can kubelet be upgraded gradually on individual nodes.

For example, kube-apiserver version 1.37 supports kubelet versions 1.37, 1.36, 1.35, and 1.34. The logic is the same: first upgrade the centralized component atomically, then gradually bring the distributed components up to date.

In practice, the "Skew Policy" is exactly that rolling window. It is fundamentally the same thing, simply expressed in different words.

I hope the general logic is clear. In that case, the upgrade procedure looks like this:

The platform team checks the oldest starter still present in the fleet.
It upgrades Master first.
Service teams independently upgrade the starter and build plugin.
Gradually, the entire fleet converges on the new version.

Compatibility Window. Trade-Off

Naturally, this is a trade-off.

The larger N is, the more time users have to upgrade, but the harder your job becomes: you have to support a long tail of old APIs, and remember that all of this must also be tested in matrix builds.

The smaller N is, the less time users have to upgrade and the less comfortable it is for them, but the easier your job becomes: you can aggressively drop old APIs.

Important advice: if you are at the beginning of building a product, prioritize the ability to make aggressive upgrades while the product has not yet achieved widespread adoption or accumulated a huge number of users. Over time, move N upward.

Which Scheme Should You Choose?

There are many more things I have not covered. Perhaps they deserve a separate article. But I think I have already thrown a great deal of information at you, and it is better to stop here before it all turns into a jumble.

Let us summarize the practical takeaways.

If You Have a Consumer Application

Ask yourself two questions:

How often do you release? Is each release a major event for you, or is it a more-or-less routine process?
How many external integration points do you have?

In the general case, CalVer or Marketing Versioning will most often be the right choice for you.

If You Have a Library

SemVer is a good starting point. But first, define the public API explicitly. Without that, the promise of backward compatibility will be extremely vague.

In practice, you will probably come to realize over time that the public API surface is too large. In that case, look at Spring Boot's model and the "Pain Index", and describe your approach explicitly. The worst situation is when users believe they are getting a guarantee that does not actually exist.

If You Have Several Closely Related Components

Consider lockstep. It is especially useful if users perceive the product as a single whole and there are not many compatible combinations. Lockstep has procedural and communication problems of its own, but there are ways to ease those pains too.

If you choose lockstep, invest in release automation immediately, because even a small patch to one component becomes a release of the entire set. Ideally, you will need a monorepo.

If You Have an Ecosystem of Independent Projects

You will inevitably need to establish a rolling compatibility window across releases, because a large organization will not be able to upgrade your solution atomically. A situation is guaranteed to arise in which an older version of component A must work with a newer version of component B.

The size of the compatibility window is a trade-off between your development speed and user convenience. That trade-off should be different at different stages of the product's evolution.

Closing Words

A version is not merely a release counter. It is part of a product's public contract.

A good version should help the user answer at least one important question:

how recent this release is;
how painful the upgrade will be;
whether the components are compatible;
which set of dependencies has been tested together;
how much time the team has left for migration.

CalVer answers the question of time. SemVer tries to describe API compatibility. The "Pain Index" estimates upgrade complexity. Lockstep reduces the number of product versions.

There is no need to choose the trendiest scheme. You need to choose a promise that matches the product's architecture and that your team is genuinely capable of keeping.

Because the number 2.4.1 guarantees nothing by itself. The guarantee comes only from the discipline of a team that has explained what those numbers mean and followed its own rules for years.

Best of luck, everyone!

New Java Features Are Not Just for Interviews. Two Real-World Use Cases from Axelix

Sergei Cherkasov — Wed, 16 Sep 2026 14:40:54 +0000

Hey everyone! Mikhail Polivakha here, technical lead of the Axelix project.

Java moved to a six-month release cycle quite a while ago. Every six months, we get a new version, read about the latest JEPs, and watch conference talks. Then we go back to work and continue writing roughly the same code we were writing five years ago.

And that is not necessarily a bad thing, by the way. Bringing a new feature into production merely because it is new is not exactly the most brilliant engineering strategy. Moreover, many applications are still running on Java 17, while Java 11 is alive and well in some places.

But this raises a natural question:

Do new Java features have a place in ordinary, real-world applications? Not in a presentation, not in a toy pet project, but in code that solves an actual problem?

Yes, they do.

In this article, I will examine two examples from the open-source Axelix codebase:

How we use ScopedValue to pass a security context from an HTTP filter deep into the transport layer.
Why one of our interfaces became sealed, and not for pattern matching or an exhaustive switch.

At first glance, these examples seem completely unrelated. One is about execution context, while the other is about the overall domain design. But they actually share one common idea: a good language feature lets us turn an agreement between developers into a constraint enforced by the platform itself.

Well then, let's get started.

A Little Context About Axelix

So that the code below does not look like a collection of random classes, let me first say a few words about the architecture.

Axelix is the Open Source product that helps to debug problems in Spring Boot applications and also detects common issues and inefficiencies in them. In simplified terms, the system consists of two parts:

Axelix Master aggregates information, handles the RBAC and provides a UI and a built-in MCP server.
Spring Boot starters, together with a small Build Plugin, are installed in the monitored applications and provide Master with the necessary data and operations.

When a user invokes an operation through the UI or an AI Agent, Master sometimes has to contact the starter of a particular application. For example, it may request the environment state, retrieve information about beans, or clear a cache.

And this is where a security-related requirement comes into play: Master has to pass the starter the authorization token associated with the security context in which the operation is being executed.

The token appears fairly high up, inside a servlet filter. It is needed much further down, in the transport layer where the outgoing HTTP request is built.

Of course, we could add a SecurityContext parameter to every method between these two points. Then to five more methods. Then to ten. And six months later, we would discover that half of the application is busy with the fascinating task of moving a token from one argument to another.

Although that works too, we wanted to find a better solution.

Use Case One. Passing Context Through `ThreadLocal`

Historically, the standard solution to this kind of problem in Java has been ThreadLocal. Spring Security, for example, used to work this way.

The idea is very simple:

private static final ThreadLocal<SecurityContext> CONTEXT = new ThreadLocal<>();

void runWithinContext(Runnable action, SecurityContext context) {
    SecurityContext previous = CONTEXT.get();
    try {
        CONTEXT.set(context);
        action.run();
    } finally {
        if (previous == null) {
            CONTEXT.remove();
        } else {
            CONTEXT.set(previous);
        }
    }
}

Code running further down the call stack in the same thread can call CONTEXT.get() and obtain the required value. There is no need to pass the parameter through every intermediate method.

So where is the problem?

`ThreadLocal` Gives Us More Capabilities Than We Need

The first and fairly well-known problem with ThreadLocal is that any code with a reference to it can call not only get(), but also set() or remove(). In other words, a ThreadLocal is essentially shared mutable state, and experienced engineers know that this is a torpedo lodged in the side of our "ship".

The calling code stores the context of the authenticated user, but some distant callee can theoretically replace that context. Some tasks genuinely require this kind of mutability. In our case, however, the data flows strictly in one direction:

The filter authenticates the user.
The filter creates a SecurityContext.
The code further down only reads that context.

Nothing further down the stack should change the user's identity. We do not merely have no need for this capability, but it effectively represents a security breach! Suppose, for example, that some third-party implementation of an SPI interface can mess with the SecurityContext. That would be a serious problem.

But ThreadLocal has another issue. The lifetime of a value in a ThreadLocal is not constrained by the structure of the code. We are responsible for calling remove(), usually in a finally block. Why is that a problem?

Imagine that, for whatever reason, and the exact reason does not matter, we fail to clear the user's identity from the ThreadLocal after processing an HTTP request. We then receive another request that happens to run on the same thread from the pool. For now, let's leave virtual threads and the thread-per-task execution model aside, since that is a separate topic altogether. The security context from one request may now leak into another request.

Now imagine that the SecurityContext left in the pool belonged to a user with the ADMIN role, which allows them not only to read application properties but also to view sensitive values. By default, Axelix has its own policy for handling sensitive values. These values are configured through separate properties, and only an ADMIN is allowed to read them. By leaving a user with the ADMIN role in our SecurityContext, we may effectively allow someone without that role to see secrets through either the UI or the MCP server.

Of course, a careful developer will write a try/finally. They will cover the code with tests. They will leave a comment. But if the correctness of the solution depends entirely on every future developer remembering an important comment, that is far from the strongest guarantee.

`ScopedValue`. Not Storage, but a Bounded Execution Scope

In Java 25, ScopedValue was finalized. It solves a narrower problem than ThreadLocal: it allows a value to be passed down the call chain for the duration of a particular operation.

The key difference lies in the model itself. A ThreadLocal resembles a mutable box attached to a thread. A ScopedValue describes the binding of a value within a dynamic execution scope.

This is how we use it to pass the SecurityContext in Axelix Master:

private static final ScopedValue<SecurityContext> SECURITY_CONTEXT =
        ScopedValue.newInstance();

public <V, T extends Exception> V callWithinSecurityContext(
        ThrowingCallable<V, T> callable,
        SecurityContext securityContext) throws T {

    return ScopedValue
            .where(SECURITY_CONTEXT, securityContext)
            .call(callable::call);
}

The where(...).call(...) invocation means approximately the following:

Execute this callback so that, within its dynamic scope, this ScopedValue is bound to this SecurityContext.

When the callback finishes, the binding disappears automatically. It does not matter whether execution completes normally or throws an exception, because ScopedValue is designed to handle that case as well.

Reading the context looks like this:

public Optional<SecurityContext> getSecurityContext() {
    if (SECURITY_CONTEXT.isBound()) {
        return Optional.of(SECURITY_CONTEXT.get());
    }
    return Optional.empty();
}

Notice that ScopedValue itself has no set method. Code further down can read the binding, but it cannot simply replace its value. If the same ScopedValue needs to be temporarily bound to a different value, a new nested scope is created. Once that scope ends, the previous binding becomes visible again.

In other words, the semantics we need are already expressed in the API:

the value moves down the call stack;
the binding exists only within a particular operation;
a callee cannot mutate the binding;
leaving the scope reliably restores the previous state.

This is exactly the kind of case where a new Java feature does not merely save a few lines involving finally, but makes the correct model apparent from the code itself.

How It Works in a Real Request

Now let's put the whole flow together.

For a regular UI request handled by ExternalApiCookieAuthorizationFilter, we obtain a JWT from a cookie, decode the user, and execute the remaining filter chain inside the security context:

securityContextExecutor.runWithinSecurityContext(
        () -> filterChain.doFilter(request, response),
        new DefaultSecurityContext(user, token)
);

For an MCP invocation, the IAM flow is similar but slightly different. You can see the source code here if you are interested. I will not spend time on it now.

The regular application logic then runs inside the filter chain. It may pass through controllers, services, and other abstractions. They do not need the token at all, so we do not pollute their APIs with it.

The context is finally read in AbstractEndpointProber, immediately before sending the request to the starter:

SecurityContext securityContext = securityContextExecutor
        .getSecurityContext()
        .orElseThrow(() -> new IllegalStateException(
                "Security Context is expected to be bound"));

builder.header(
        HttpHeaders.AUTHORIZATION,
        AuthenticationSchemes.BEARER.prefix() + securityContext.token()
);

Once request processing finishes, the binding is removed, and the token can no longer be obtained through this ScopedValue. Pretty neat!

Some Limitations of `ScopedValue`

I think it is only fair to mention a few limitations of this API. Overall, they are quite acceptable for us at the moment.

The Binding Is Tied to a Thread

ScopedValue is not automatically propagated to an arbitrary CompletableFuture, executor task, or asynchronous servlet operation.

The current Axelix code works because proxying the request to the starter, where the binding is read and the Authorization header is set, happens synchronously inside the filter chain on the same thread. Child tasks can inherit a binding through StructuredTaskScope, but in Java 25 that is still a Preview API.

If you place a value in a ScopedValue and then submit a task to some random thread pool, you cannot expect the value to be available there.

The Binding Is Immutable, but the Object Might Not Be

ScopedValue does not allow the binding itself to be replaced. But if you put a mutable object inside it, its fields can still be changed using ordinary methods. This issue is not specific to ScopedValue. It also applies to ThreadLocal and a number of other APIs. Consider this example:

ScopedValue<List<String>> VALUE = ScopedValue.newInstance();

The binding of the list itself is stable here. Nevertheless, nothing prevents someone from calling VALUE.get().add(...), so the mutability question remains. Again, this problem is not unique to ScopedValue, but it is worth keeping in mind.

It Is Not a Universal Replacement for Parameters

Generally speaking, a value can be passed to a method either explicitly, through its signature, or implicitly, through contexts of this kind. From a code maintainability perspective, if a value is an important part of a method's contract, an ordinary parameter is almost always better. In that case, the API contract is clearly visible in the signature.

You do not end up in a situation where the interface says this:

public interface EndpointProber<O> {

    O invoke(HttpPayload payload);
}

while the implementation is actually doing something like this under the hood:

public class DefaultEndpointProber<O> implements EndpointProber<O> {

    // Declared somewhere else, in another class
    private static final ScopedValue<InstanceId> TARGET_INSTANCE =
            ScopedValue.newInstance();

    @Override
    public O invoke(HttpPayload payload) {
        String accessToken = TARGET_INSTANCE.orElseGet(payload.getToken());

        return probe(instanceId, payload);
    }
}

The code above is deceptive because the caller may think they have passed the token to use for the invocation, while in reality some other token source takes precedence. Again, this is not a problem specific to ScopedValue. I simply thought it was worth highlighting. Experienced engineers most likely know this already.

In general, ScopedValue and ThreadLocal are especially useful for context that is needed far down the call stack but is unrelated to most of the intermediate methods:

security context;
tracing context;
tenant-id context;
request metadata.

In other words, the need for one-way context propagation must come first. Only then should ScopedValue enter the picture.

A Small Conclusion

Use ScopedValue when a value should only travel down the call chain and should live no longer than one bounded operation.

If you need arbitrary mutation, integration with legacy code, or support for an older Java version, ThreadLocal is not going anywhere. Just make sure to encapsulate the set and remove operations properly.

Use Case Two. What Do `sealed` Interfaces Have to Do with It?

Now let's move on to a completely different problem. Or, as will soon become clear, perhaps not entirely different.

I want to warn you right away that the solution below is somewhat debatable, and I explain why later. I am describing it largely to share our experience and hear what other people think in the comments. Nevertheless, this is exactly what we did.

Axelix Master has a built-in MCP server. An AI Agent can invoke MCP tools, including both read operations and potentially dangerous operations, such as clearing a cache. As part of RBAC, we obviously need to check access rights for operations of this kind as well.

Different MCP endpoints require different authorities. In simplified terms, Axelix Master has a structure like this:

private static final Map<McpEndpoint, Authority> MAPPING;

static {
    MAPPING = new HashMap<>(2);
    MAPPING.put(McpEndpoints.CLEAR_ALL_CACHES, OssAuthority.CACHES_CLEAR);
    MAPPING.put(McpEndpoints.CLEAR_SPECIFIC_CACHE, OssAuthority.CACHES_CLEAR);
   // and so on.
}

When we receive a JSON-RPC request to invoke an MCP tool, we parse the request and turn the string name of the MCP endpoint that the caller tried to invoke into an McpEndpoint object. We then use that object as a Map key and obtain the required authority.

Why use it as a key? Because it makes sense from the domain perspective: invoking a particular operation within the MCP server requires a particular authority. Having this kind of mapping seems to be quite right.

Could we simply use a String as the key? Technically, yes, but that would introduce its own inconveniences given our implementation and class structure. I will not go into those details now.

Suppose that McpEndpoint is an interface. At this point, an experienced engineer reaches for the Colt on their belt!

public interface McpEndpoint {
    String name();
}

It is just an interface. What could possibly go wrong?

A Potentially Mutable `HashMap` Key. A Time Bomb

Why did the experienced engineer reach for the Colt? Because, broadly speaking, McpEndpoint is an interface, and we do not know what its implementations might look like. Keys in a Map should be immutable, which is a widely known fact.

Imagine that someone implements the open interface like this:

final class MutableMcpEndpoint implements McpEndpoint {

    private String name;

    MutableMcpEndpoint(String name) {
        this.name = name;
    }

    @Override
    public String name() {
        return name;
    }

    void rename(String name) {
        this.name = name;
    }

    @Override
    public boolean equals(Object other) {
        return other instanceof MutableMcpEndpoint endpoint
                && Objects.equals(name, endpoint.name);
    }

    @Override
    public int hashCode() {
        return Objects.hash(name);
    }
}

Now let's put this endpoint into a HashMap and then change its name:

var endpoint = new MutableMcpEndpoint("clearAllCaches");
var mapping = new HashMap<McpEndpoint, Authority>();

mapping.put(endpoint, OssAuthority.CACHES_CLEAR);
endpoint.rename("clearSomethingElse");

Authority authority = mapping.get(endpoint); // surprise!

During put, the HashMap selected a bucket based on the old hashCode. After rename, the object returns a different hashCode, but it is still physically located in the old bucket. A lookup may fail to find a key that is literally present inside the very same Map.

As a result, we could end up in a situation where we simply fail to detect that a particular McpEndpoint requires an Authority for its execution. That is yet another security breach!

Of course, we could write the following in the Javadoc:

All implementations of McpEndpoint must be immutable and have stable equals and hashCode implementations.

But once again, correctness would depend on a comment.

`sealed` Is Not Just for `switch`

Sealed classes and interfaces were finalized as a language feature back in Java 17. They are usually explained using a closed hierarchy of shapes:

sealed interface Shape permits Circle, Rectangle {
}

The compiler then knows the complete set of alternatives and can verify exhaustive pattern matching. This is useful, but it is far from the only use case.

If you think about it, the very nature of a sealed type as a language feature means that we do not merely know, but have control over every possible implementation of the interface. That control can cover different properties, including, importantly for us, the enforcement of immutability!

This is what McpEndpoint in Axelix looks like:

public sealed interface McpEndpoint permits DefaultMcpEndpoint {

    String name();
}

The only permitted implementation is a record:

public record DefaultMcpEndpoint(String name) implements McpEndpoint {
}

What does this combination give us?

Arbitrary code cannot add an unknown implementation of McpEndpoint.
As I said earlier, we control all permitted implementations.
The current implementation is a record, so its components cannot be reassigned after creation.
Its only component is a String, which is itself immutable.
The record generates component-based equals and hashCode implementations, based here on the stable name component.

As a result, every McpEndpoint that can currently be placed into the Map has the key semantics we need.

And this is an important, though not entirely obvious, use case for a sealed interface: we close the hierarchy because the algorithm relies on the properties of every implementation.

It is fair to say that we are effectively relying on an implementation property while working with a contract, which is a violation of SOLID. That is all true.

Nevertheless, sealed types are, by their very nature, designed around knowing the complete set of implementations in advance and doing things such as exhaustive switch statements. Strictly speaking, that also violates SOLID because we rely on implementations instead of working purely with an abstraction. There is room for a productive discussion here.

Why Not Use an `enum`?

That is a natural question. If the set of MCP endpoints is fixed, why not simply do this:

enum McpEndpoint {
    CLEAR_ALL_CACHES("clearAllCaches"),
    CLEAR_SPECIFIC_CACHE("clearSpecificCacheEntity"),

    // and so on...
    ;

    private final String toolName;

    McpEndpoint(String toolName) {
        this.toolName = toolName;
    }

    String toolName() {
        return toolName;
    }
}

This is a perfectly valid alternative. Moreover, for a permanently fixed set of endpoints, an enum would provide an even stronger guarantee: not only the implementations but also the instances themselves would form a closed set.

So why might the interface still make sense? Because for us enums are too inflexible and, in particular, cannot be extended. Different Axelix Master distributions may have different McpEndpoint instances, while an enum could not be extended to accommodate them.

Nevertheless, always remember that extensibility is not free. Leave an extension point only where you genuinely need one.

It is also important to remember that the sealed model does not guarantee:

unique endpoint names;
registration of every new endpoint in all Map mappings of this kind;
correctness of the authority mapping itself.

Did you add a new constant and forget to register it in one of the resolvers? javac will not save you here. That requires different data models and tests.

In our particular case, sealed solves one particular problem. And that is fine. We should not expect a language feature to wash the dishes and deploy a release in-between.

In other words, this is a balance between extensibility and immutability.

What These Two Features Have in Common

Now let's return to the beginning of the article.

ScopedValue and a sealed interface look like completely different Java features:

ScopedValue works with the execution context.
A sealed interface works with the type system.

But in our two use cases, they solve the same engineering problem: they reduce the set of valid program states.

With ScopedValue, we say:

A value can be bound to a ScopedValue only for a bounded period of time. Code further down can read it but cannot arbitrarily overwrite it.

With a sealed interface:

The abstraction is available throughout the application, but only types under our control can implement it.

This fits well with the overall direction of the platform that Java architects call Integrity by Default: correct and safe behavior should be the default, while a dangerous exception should require an explicit decision.

In my opinion, this is exactly how new language features should be evaluated. Not by the number of lines they allow us to remove, and not by how elegant they look in a conference talk. The real question is:

Which invariant of my application does this feature allow me to express and protect?

If there is no answer, perhaps you do not need the feature yet. If there is an answer, then it is no longer a toy.

Practical Recommendations

Let's sum things up.

When to Consider `ScopedValue`

Use ScopedValue if:

Data only travels down the call chain.
Intermediate methods should not have to accept it as a parameter.
A callee should not replace the binding.
The lifetime of the data matches a bounded operation.

Keep using an ordinary parameter if the dependency is an important part of the method's contract. Keep using ThreadLocal if you need mutation, legacy integration, or support for an older Java version.

When to Consider `sealed`

Use a sealed class or interface if:

The abstraction must be broadly visible.
The set of implementations must remain under the author's control.
The code relies on the properties of all permitted implementations.

The Invariant Comes First, Then the Feature

Do not go looking for somewhere to put a ScopedValue, sealed class, record pattern, or virtual thread merely because you can. As everyone knows, we never do that, right? Right...?

First, identify an actual constraint in the system:

this context must not leak beyond the request boundary;
this binding must not be changed from below;
this hierarchy must not be extended by unknown types;
this object must have stable value semantics.

Only then should you choose the language feature that expresses that constraint.

Closing Thoughts

New Java features really are used in real-world applications. But their value does not appear when we replace old syntax with new syntax. It appears when an entire class of incorrect programs becomes harder to write or impossible to compile.

In Axelix, ScopedValue gave the security context a bounded lifetime and safe propagation down the call stack. A sealed interface allowed us to keep the domain abstraction public while retaining control over the semantics of its implementations and still permitting new McpEndpoint instances to be created.

Both features reduced the room for error. And that, in my opinion, is one of the main things we want from a programming language.

The source code is open, so you can inspect all the examples directly in the Axelix repository.

Take care!

One Branch to Rule Them All

Sergei Cherkasov — Wed, 19 Aug 2026 16:09:22 +0000

Hey everybody. This is Mikhail, the technical lead of the Axelix project.

For quite a long time I wanted to write a small article outlining our git branching model, in hope it may be useful to other people.

There are quite a lot of git branching strategies in the world, e.g.

Gitflow
GitHub-flow
Trunk-Based-Development (TBD) and so on.

Throughout my experience I can tell, that teams usually do not adopt the given strategy in the exact form/shape, rather, they either invent something completely
new themselves, or they just take, for instance, aforementioned GitFlow and modify it for their needs.

So, throughout already a year of development of Axelix, we have adopted a slightly modified Trunk Based Development model, and I think it served us well.
I hope you will find it also useful. This model admittedly has certain downsides, but in general, we're happy with it.

Before we start, you should be mindful of the requirements we tried to address. At the end of the day everybody must judge for themselves.

Small Prerequisite. Axelix Versioning.

The two most difficult problems in software are naming and versioning.

This is not the exact quote, but it is a rough sense of the joke that Thomas Wuerthinger (GraalVM Project Lead) said on BoF on Devoxx Belgium 2025 (explaining the "Detaching GraalVM from the Java Ecosystem Train" article).

To later understand on why we're doing certain things we're doing, it's going to be useful to realize the versioning scheme we're using.

At Axelix, we use semantic versioning of our components. I'd even say that SemVer is a de-facto standard for versioning the modern world. There are, of course, simple versioning schemes, like for Windows OS, when you
have Windows 7, 8, 10 or 11. There also is software that is released under the different variations CalVer. The JetBrains IDEs will be a good example of that,
e.g. 2026.1 or 2026.2.

Still, the majority of projects do SemVer, and Axelix does it as well. But when we're talking about the modern software - it is often composed of multiple components.
For example, Axelix has:

Build plugins for Maven/Gradle
Spring Boot starters
Master as the separate bootable JAR
Docker image of Master
Helm Chart of Master

So, the thing is that each of the points above represent an independent artifact that is generally distributed independently of others, and thus, it has its own coordinates.

And in our case, we have selected to follow the Lockstep Versioning of the Axelix project. In short, this is not the versioning schema, but a versioning strategy where all
components of the system share the same version X.Y.Z. It means, that if we release Axelix 1.4.2, all the components above receive the same version - 1.4.2.

Yes, we know that Lockstep versioning is generally a highly debatable subject for a couple of reasons. And I would say, that team should generally think twice before adopting it.
I should probably write another article outlining why we choose semver with lockstep, and why it may be a good idea for some of you as well. But this is an off-top for this article.

The Problem

So, you're starting a new project, or you're about to revisit the git branching model in your team for various reasons. When you try to implement any technical solution (let's say, you want to choose a git branching model for the project), it is generally a good idea to take a step back and answer the question:

What exactly is the problem that we're solving?

Indeed, before explaining anything, we should first realize what are the specific requirements that we want to address with any git branching model. And these requirements
will be different for different kinds of software that has different release cadence. In the case of Axelix, we wanted a branching model that would:

1. Give us the ability to do patches

We want to be able to deliver patches quickly.

Moreover, the important thing is that we do not know the amount of patch releases in the given minor in advance. The general
requirement is to be able to go to the arbitrary minor release, and apply an arbitrary number of patches for it (for example, to fix the CVE).

2. Avoid the merge hell.

A lot of git branching strategies easily end-up in the merge hell, e.g. GitFlow is infamous for ending up in such conundrums.

So, with that in mind, let's finally discuss what branching model we actually adopt.

Axelix Git Branching Model

At Axelix, we have the monorepo. And in this monorepo we have the main trunk, called 'master' trunk (the thing we borrowed from TBD). Some commits that do
polishing or basic housekeeping happen directly in the master trunk. For features, we create short-lived branches (directly from master) that gets merged
back into the master trunk:

In some extreme cases, adepts of TBD advocate for no feature branches at all with the risk of them being long-lived, and I understand the rationale behind it.
However, for OSS projects such as Axelix, we cannot allow for that because of external contributors and for various other reasons.

Nevertheless, feature branches ideally need to die fast. To facilitate "the death" of feature branches, we impose a couple of rules on the feature
branches, and famously, we introduce a cap of 500-lines PR. In other words, feature branch cannot contribute more than 500 lines of code into the main trunk (there are some
exceptions to that rule, but it does not matter right now). This cap ensures that:

The PR is small enough to be carefully reviewed.
The feature branch will indeed be short-lived, since the smaller PR the faster it will be ready for review.

It is generally well-known that for small PRs the overall TTM (time-to-merge) is smaller,
and the cap ensures feature branches die fast and they do not evolve into the long-lived branches. So here, we're perfectly aligned with the TBD.

When we want to perform a minor/major release, we create a tag on the master trunk of our monorepo and execute the release pipeline (very simplified):

Since we have the monorepo with the lockstep versioning, this tagged commit is the one from which all the release artifacts are built. And the value of the
tag is the version that Axelix release gets.

If we want to do the patch release, we create the branch from the git tag, e.g. a branch named 1.0.x. That is the long lived branch that is going to live by itself.
Once we have created necessary patches in the 1.0.x branch, we create a tag in that long-lived patch branch, call this tag v1.0.1 e.g, and release it then:

And here the pivot point where we significantly and deliberately depart from TBD. In TBD, it is forbidden to have long-lived branches apart from the main trunk.
But I want to take a step back and explain not what is forbidden, but why TBD advocates against it, and why it is okay in our use case.

TBD: Why patch-branches are okay.

TBD is a well-known and proven git branching strategy. It is adopted by Google, Facebook and other companies.
And it does not allow long-lived (apart from the main trunk) branches for a variety of reasons. One of them, and, probably, the most notorious is the merging problem. Merging the long-lived branches is quite challenging and time-consuming,
error-prone, bug-welcome and so on.

This problem becomes real in scenarios, when we develop certain functionality, that is intended to be later merged with something else, like a main trunk. But in case
of such patch branches, we do not intend to merge them ever. They are not supposed to be merged.

According to SemVer, the changes that go into the patch releases are (very broadly):

Backwards compatible bugfixes
Backwards compatible perf. improvements
Backports of CVE patches

Of course, we can discuss all day what exactly backward compatibility means, whether it accounts for "behavioral" backwards compatibility or source-level backwards compatibility and so on.
Let's put that aside.

The core idea is that patch branches will never have any kind of features developed in them, that are going to be merged back into master trunk. That just won't happen.

In our case, for example, the changes that end up in the patch branches are the very specific commits that just got cherry-picked from the main master trunk.

Challenges & Problems

Of course, such a model as above has challenges. There are a couple of them. Let me enumerate them.

1. Minor/Major Dilemma

One of the most important challenges of such architecture is its Minor/Major inflexibility.

So, what do I mean by that?

Well, for example, let's say, that you're developing version 1.7 of your software. And now, you're thinking about releasing a new major version, 2.0. If you're
making that decision, you're effectively starting to merge all the changes that you wanted to end up in the new major (e.g. non-backward-compatible ones, more on them
later) into the master trunk. Well, once you've started doing that, pivoting back and saying stuff like:

Oh, no, we changed our mind, we just want to release 1.8

It becomes really hard, in some cases infeasible. That is the inflexibility I'm talking about.

Still, that problem (in its various levels of severity) of having no simple return follows a lot of similar git models, TBD included.
So, in the industry, this problem is generally a consequence of the trade-off of not having merge hells.

2. Long-Lived Breaking Changes Branches

Well, when I said to you, that we have master trunk and patch branches as the only long-lived branches, I have omitted one thing. One very important thing.

And that is: our git branching model allows for long-lived feature branches that contain breaking changes. And since such branches are intended to be
merged, we have to postpone them until the next major (at very least). And if we combine that with the problem above, then we realize - we're in trouble.

Well, although we are in trouble, the degree towards which we're actually in trouble is debatable.

First of all, it significantly depends on the fact of what changes we consider breaking. And for every
mature system such changes may be different. In our case, we have a whole documentation page that explains our release cadence and overall breaking changes
impression.

Another thing is that we can try to mitigate such problem by constraining the amount of breaking changes subject for the next release. It is not always
feasible/predictable, but tackling that problem not from a technical, but from an administrative PoV may also help here.

So, admittedly, this is also a challenge that we have to live with. We may try to mitigate that, but to some degree we're going to have it.

Conclusions. Axelix, Lessons Learned.

So, again, I did not want to say that we have invented some astonishing or mind-blowing git model. Internally, we just felt that it may be worth sharing with
the world the git branching model we adopted and the reasons why.

The goal of the article was more to outline the way we approached the problem with the hope that you'll learn from it. Axelix has the Open Source core,
so if you want to learn more - you're welcome.

Remember that there're always trade-offs, and sometimes it is okay to diverge from certain well-established practices. Still, you always have to
clearly justify such divergences and, what is also important, realize the weaknesses of your solution as well.

Take care!
Mikhail

Axelix goes GA. A journey of a thousand miles begins with a single step

Sergei Cherkasov — Tue, 11 Aug 2026 12:39:57 +0000

On behalf of the core Axelix team, and everybody who has contributed to the community, I want to declare: we finally did it.

Axelix, finally, goes GA (Generally Available)!

For those who do not know - Axelix is a product with an Open Source core, that allows you to discover the common problems, pitfalls and inefficiencies in Java applications at large scale.

We're available on GitHub (btw - give us a star!).

In this post, I want to share the story and the motivation behind the product overall. I hope you find it interesting.

The Story. Big "Why" Behind Axelix

Java is quite an interesting language and ecosystem in general. I think a lot of people will not argue that it is quite old, and it was one of the first so-called "Object-Oriented" languages, that actually gained massive adoption. It both was, and it still is the backbone of modern enterprise.

For anyone who claims that Java is dead - I recommend checking the JetBrains State of Developer Ecosystem survey or even the Stack Overflow survey for 2025 (and Stack Overflow has, sadly, become a part of history). It is clear that Java as a language and the "ecosystem" around it (including Kotlin) is still relatively popular, and it remains true.

Ecosystems around Languages

The experienced developer knows that today's ecosystems that evolve around languages are typically very diverse. For example, let's talk about JavaScript. If we decide to run JavaScript on the server, then we're probably going to work with a database of some sort. Therefore, we're also going to need a framework, a library to work with the database, e.g. an ORM (I know that we may work without it but let's leave that aside).

And in JavaScript, we have quite a lot of options:

Prisma
TypeORM
DrizzleORM
Kysely and so on.

We can pretty safely state that Prisma ORM is probably the most used ORM on JavaScript. But notice that it is far from being the definitive JavaScript ORM.

It is not like Prisma is the default choice and is by far the most popular ORM - it is just not the case. I want to emphasize - even by Prisma's own assessment of its popularity, it is not far ahead of its competitors. They are all very close. So, the way people work with the database in JavaScript may differ heavily from team-to-team.

The defining trait of Java Ecosystem

In Java, if we're going to take the same problem (working with the database), then we will quickly realize, that the industry is heavily concentrated.

We generally can state that Spring Data JPA as a solution around Hibernate occupies above 50% of all the data access solutions, let alone ORMs. I remember some time ago Josh Long, the much-loved Developer Advocate for Spring, posted a survey on X (formerly Twitter) about the data access technologies usage:

// Detect dark theme var iframe = document.getElementById('tweet-1528655715933597697-638'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1528655715933597697&theme=dark" }

We must of course acknowledge that there are fast-growing alternative solutions to JPA that are explored, such as jOOQ, but Spring Data JPA dominates the field. So, the way we work with the database in Java (at least on the server side) is relatively well-defined and revolves around Spring Data JPA.

But there is more to it, it is not just the ORMs or data-access libraries. It's something bigger.

Spring Framework as the Phenomenon

We cannot talk about Java ecosystem and not talk about Spring Framework. If we're going to look in the aforementioned Developer Ecosystem State report from JetBrains, we're going to get the following insights. I have assembled them carefully for you - share them freely as you want. Also note that the respondents had multiple choices:

Framework	Users	Share of framework users
Spring Framework	3,042	87.8%
Ktor	362	10.5%
Quarkus	287	8.3%
Other	210	6.1%
Vaadin	104	3.0%
Micronaut	95	2.7%
Grails	61	1.8%
Helidon	36	1.0%

This is (at least AFAIK) one of the most recent and most trusted data we have. Many thanks to JetBrains for publishing it.

So, yes, although other ecosystems are also growing, e.g. Quarkus along with its Quarkiverse, but they still have a very, very long way to catch up with Spring Framework. So at the end of 2025 (I wager that even today) we can safely state:

The most common Java app, which by far represents more than half of Java server side deployments is a Spring Framework app.

Again, it is just data, you can download it and check yourself.

But even if we consider different ecosystems: Spring Framework, Quarkus or Micronaut etc - we're going to quickly realize that these are not just "libraries" or "frameworks" in a traditional sense. They are themselves large frameworks that build an entire sub-ecosystem around them. For example, are you working with Data? Well, you have

Spring Data in Spring
Jakarta Data in Quarkus
Micronaut Data in Micronaut

Oh, you want to send an HTTP request? The most common thing on the backend! Well, we have:

Spring Cloud Exchange/OpenFeign in Spring
RestClients in Quarkus
Declarative HTTP Clients in Micronaut

Notice that every framework has its own solution.

In other ecosystems, e.g. in JavaScript on the server side, we may use:

ExpressJS for web server
Axios for sending requests
Prisma as an ORM

And these are all just individual choices made by the team. We may use Axios or we may use Got. We may use Prisma or we may use TypeORM. Some solutions are more popular than others, but they are all independent - they exist outside of a single umbrella, like Spring Framework, for example.

To put it simply:

In the Java world, if we have any kind of problem, most likely, we do NOT need to seek the solution on GitHub, find some fork that does what you want (which is a common JavaScript practice btw). There is almost 100% chance that Spring Framework has a solution for us already.

Why am I telling you about this?

Because it is important to understand, that since Java Ecosystem is so solidified around the Spring Framework technological stack, it often suffers the same problems in production.

What are these problems? Well, there are a lot of them. And we know all of them with our very own butts:

Our connection pool leaks
Our API endpoints are performing slow (maybe the problem is in the database?)
We accidentally expose /actuator/env in production and leak secrets
Our applications consume too much memory and start very slowly...

And so on. And the industry, throughout decades of Java experience in production, has developed a set of recommendations, best-practices, "solutions", guidelines. And these guidelines exist on different levels of the application, e.g:

Guidelines for Hibernate/ORM

and so on. Guidelines for Spring performance:

Disable OSIV
Use Spring Test Profiler! (many thanks to Philip Riecks for such a tool)
Use Spring smart context caching! (Sergey Chernov, a highly experienced Engineer and a buddy of mine, developed the tool)!
Do not set spring.jpa.show-sql=true or other dangerous properties in production!

and so on. Guidelines for even our beloved Java!

For Christ’s sake, use Project Leyden!
Use Compact Object Headers (Project Lilliput)
Configure the logging of GC in production!

And so on and on and on. Because we have that solidified ecosystem we generally know "what is right and what is wrong". And for the long time there was no such tool.

But we thought that this is wrong. And we probably should have one. And this tool is called Axelix (The source code).

In 1.0.0, we do not support the detection of all the issues that I outlined above (most, but not all). And the above is just an example. There is a lot (really, a lot) of knowledge that we as a Java community have accumulated throughout these years that we're going to put inside Axelix, so it can serve you well.

I hope, at that point, the motivation behind the tool is clear, and now, I want to answer a couple of questions that we already have received many times over during Milestones.

Q: Is it free?

Yes, it is. We're going to follow the Open Core model. This is the Open Source software development model that underpins projects that we all love and care about:

Grafana
Keycloak
Vaadin

And so on. It is generally considered to be the most reliable model for the OSS development, and that is why industry leaders choose, and that is why we choose it.

Q: Is the project ready for use and deploy?

Yes, it is ready for use free of charge. We have processed the feedback from literally tens of teams that have installed Axelix milestone releases and shared their insights with us.

Q: Is it safe to deploy in production?

It is designed for production, so the answer is yes. But since this is the first 1.0.0 GA release, the general recommendation is to first install it on development environments, and then for production.

That said, we do have teams that deployed Axelix in production already to get insights about their @Transactional methods behavior and properties resolution.

Bottom Line: The core stays Open Source always, and it always will be free of charge, and it already provides value for the teams.

Where Are We Headed?

The team behind Axelix has many years of experience writing and optimizing Spring Boot applications. We have a lot of work to do ahead of us, and we still have a lot of things to incorporate into Axelix.

For those who want to know how the project is installed & configured, please, visit our website and also our documentation. In case of any questions or difficulties, you can always reach out to the team at: hello@axelix.io We, the maintainers, are happy to answer your questions and assist you during the installation.

We're also going to stay committed to Open Source, and committed to our values, which are making sure that your Java applications are as secure, performant and efficient as they can be.

Thank you all. Mikhail.

Taming Hibernate in Tests

Sergei Cherkasov — Sat, 08 Aug 2026 07:51:22 +0000

Hi everyone! This is Mikhail Polivakha, the technical lead of the Open Source Axelix project (a project dedicated to helping you identify common problems in Java server-side applications, including while working with Hibernate).

In this article, I want to address the question that I think is pretty common amongst the advanced Hibernate users:

"Is there a verify Hibernate behavior in tests (like Junit for example)?"

Here it's really important not to hand you a "fish", but to teach you how to fish - that is, to give you a framework of what's worth keeping in mind when working with Hibernate.

Controlling Hibernate during tests. Overview.

If I asked you: guys, which problems in Hibernate annoy you the most? You'd mention things like the sneaky N + 1, In Memory Pagination (when explicitly specifying setMaxResult or a Pageable), a Cartesian Product loaded into the application's memory, or, say, some baffling swarm of SELECTs followed by UPDATEs/DELETEs under the hood, and so on.

Everything I listed above, and really all "problems" with Hibernate, can be split into 2 categories. Let's break them down.

1. Problems at the ORM level

There are situations that both we (the end users) and Hibernate identify as problems. For example, up until Hibernate 7.4, the query below would perform pagination entirely in memory. This is fairly well known:

@Transactional
@Query(
    value = "SELECT o FROM Owner o JOIN FETCH o.pets",
    countQuery = "SELECT COUNT(o) FROM Owner o")
Page<Owner> findAllWithPets(Pageable pageable);

In version 7.4 Hibernate learned to do pagination within a sub-select. But nevertheless, if you, for example, work on Spring Boot 3, you have Hibernate 6, and in the case of queries like the one above there will be a warning in stdout:

HHH90003004: firstResult/maxResults specified with collection fetch; applying in memory

And it's there because Hibernate itself understands that its actions may blow up your heap. There's even a property that we at Axelix recommend setting for new microservices that don't use 7.4 yet (and there are still many of those):

spring.jpa.properties.hibernate.query.fail_on_pagination_over_collection_fetch=true

If you set it, Hibernate will simply throw an exception where it would previously do In Memory Pagination.

In short, what I want to say is that the problem above is understood by Hibernate as a real problem. It recognizes and admits it. That's one layer of problems.

But oddly enough, these are quite hard to track. Hibernate rarely provides any means to hook into it (although there are some nuances, read below). For instance, at Axelix, to detect In Memory Pagination (if it happens), we simply literally catch an ILoggingEvent and inspect the code within it. Because there are no other acceptable ways (at least none that we're aware of)!

2. Second. Problems at the application level

There's a conceptually different layer of problems. N + 1, for example, belongs to it.

Let's think for a second: what is N + 1?. Try right now, without looking anywhere, to clearly formulate the definition in your head. Done? Here's how I'd formulate it (informally):

It's a situation in which you iterate over a collection loaded by Hibernate, sequentially accessing some lazy field within each individual entity of the collection, thereby triggering "N" additional SELECT queries.

That is, if you think about it a little, it becomes obvious that for Hibernate this is just Lazy Loading. The notion of N + 1 as a problem exists at the application level, not at the ORM level. That is, the problem is not lazy loading itself, but the semantics of lazy loading (what does this particular lazy loading mean? Is it just lazy loading, or lazy loading within an N + 1?).

In such a situation it's obviously pointless to rely on Hibernate, since the N + 1 phenomenon happens at the application level, and the ORM is completely unaware of it — it's not its area of responsibility. You can't get by without fairly strong tooling around Hibernate. Can it be written, and if so, how?

If we're talking specifically about N + 1, then the answer is generally - yes, you can, and we did it at Axelix in Open Source. Let me tell you how exactly we did it. The source code is open, take a look if you're interested.

The General Approach to N + 1 Detection

Hibernate doesn't know what N + 1 is, but it does know what a Lazy Loading association is, and that moment can already be caught. For example, here's how we do it in NPlusOneCollectionLoadListener:

    @Override
    public void onInitializeCollection(InitializeCollectionEvent event) {
        try {
            PersistentCollection persistentCollection = event.getCollection();
            EventSource eventSource = event.getSession();
            CollectionPersister collectionPersister = eventSource
                    .getPersistenceContextInternal()
                    .getCollectionEntry(persistentCollection)
                    .getLoadedPersister();

            LazyLoadingTarget lazyLoadingTarget = parseLazyLoadingTarget(collectionPersister.getRole());

            if (lazyLoadingTarget != null) {
                transactionAccessor.recordLazyLoading(lazyLoadingTarget);
            }
        } catch (Exception ignored) {
        }
    }

    // The general "role" format is expected to look like this: com.example.Order.items
    public static @Nullable LazyLoadingTarget parseLazyLoadingTarget(String role) {
        try {
            int separatorIndex = role.lastIndexOf(".");
            Class<?> ownerEntityClass = Class.forName(role.substring(0, separatorIndex));
            String propertyName = role.substring(separatorIndex + 1);
            return new LazyLoadingTarget(ownerEntityClass, propertyName);
        } catch (ClassNotFoundException | IndexOutOfBoundsException e) {
            log.warn(
                    "Unexpected propertyPath format '{}'. Axelix cannot recognize that, so lazy loading and potential N + 1 is not going to be tracked for this property",
                    role);
            return null; // it means that the role format is not the one that we expect
        }
    }

Here, through a series of manipulations, we can figure out which association was loaded lazily and on which collection. What do we then do with this information?

Axelix, for example, currently makes the following decision: If we noticed that within a transaction there were several lazy loadings of the same association, then we consider this to be an N + 1.

Is that always correct? Well, this is debatable, because, for instance, in this code example, if we assume that Order.items were loaded lazily — we, seemingly, don't have an N + 1:

public void compareOrders(Long previousId, Long currentId) {
    var previous = orderRepository.findById(previousId).orElseThrow();
    var current = orderRepository.findById(currentId).orElseThrow();

    // Two lazy accesses to Order.items, but there's no collection of orders here -
    // we're just comparing two specific versions of the same order
    if (previous.getItems().size() != current.getItems().size()) {
        throw new IllegalStateException("The order contents have changed");
    }
}

Or do we? And what if I change this example to something like this:

public void compareOrders(Long previousId, Long currentId) {
    var orders = orderRepository.findAllById(List.of(previousId, currentId));

    var previous = orders.get(0);
    // Exactly the same comparison as above, but now within an iteration
    // over a loaded collection of orders: at each step there's again a lazy
    // access to Order.items on both previous and current
    for (var current : orders) {
        if (previous.getItems().size() != current.getItems().size()) {
            throw new IllegalStateException("The order contents have changed");
        }
        previous = current;
    }
}

Does it look like N + 1 now? It does.

So, in short, what I want to say is that here you need to clearly fix the definition of N + 1 at the application level. We at Axelix deliberately decided that this case is nonetheless worth treating as an N + 1, so, in the UI, it will be reported to you:

P.S: Among other things, there's a small nuance here: N + 1, strictly speaking, can also happen without an open transaction when OSIV is enabled. That's a separate case which we're not considering for now, since it would complicate the picture.

Micro-Conclusion

I'd like to draw a small micro-conclusion here. In general, detecting some complex phenomena that occur at the application level while working with Hibernate:

N + 1
Blocking calls inside transactions, and so on.

This can be done. It just requires you to write a fairly large and non-trivial amount of tooling. Using N + 1 as an example, I hope you've roughly understood what such things will look like.

General Hibernate Problems

As a separate section, I'd like to address general Hibernate problems. Quite often people have some conditional hot-path for saving or updating data. And devs simply want to make sure that
Hibernate doesn't "sabotage" the entire process undercover. Sounds familiar, right?

Here you need to take a step back and ask: and what do we mean by "sabotage"?

Having talked with people, we'll discover that most often people want roughly the following: to have confidence that a person wrote repository.save(), and that it will execute specifically an INSERT into the table and nothing more.

For example, having this piece of code:

@Service
public class OwnerService {

    private final OwnerRepository ownerRepository;

    public OwnerService(OwnerRepository ownerRepository) {
        this.ownerRepository = ownerRepository;
    }

    @Transactional
    public Owner registerOwner(String firstName, String lastName) {
        var owner = new Owner(firstName, lastName);
        return ownerRepository.save(owner);
    }
}

The person wants to be sure that there won't be any hidden SELECTs and so on there. I think some people recognized themselves. Can this be done?

The answer: generally yes, it can. Hibernate has a Statistics API, which on the one hand isn't part of the JPA standard, and on the other hand can give us some details about query execution within a session. For example, for the code above you can write a test like this:

// Requires hibernate.generate_statistics=true
@SpringBootTest
class OwnerServiceTest {

    @Autowired
    private OwnerService ownerService;

    @Autowired
    private EntityManagerFactory entityManagerFactory;

    @Test
    @Transactional
    void registerOwner_issuesExactlyOneInsert() {
        // given
        Statistics statistics = entityManagerFactory
                .unwrap(SessionFactory.class)
                .getStatistics();
        statistics.clear();

        // when
        ownerService.registerOwner("John", "Doe");

        // then
        assertThat(statistics.getEntityInsertCount()).isEqualTo(1);
        assertThat(statistics.getEntityUpdateCount()).isZero();
        assertThat(statistics.getEntityDeleteCount()).isZero();
    }
}

This API lets you verify that only one INSERT was executed within the method. It works based on the fact that in the test I opened a transaction (Spring Boot Test of course recognizes this and opens a transaction that will have to roll back at the end),
thereby opening a Hibernate Session - this is the default behavior, so I think there are no surprises here.

Since the Statistics API collects statistics at the SessionFactory level, not at the level of individual Sessions, in each test we have to access the SessionFactory in order to first clear all its statistics.
And only then can we make the checks we're interested in (Strictly speaking, some telemetry is also collected at the Session level, but it's rather scarce and in practice can help you little).

Overall, the telemetry collected (the number of INSERT, UPDATE, DELETE statements, and so on) will be enough for you to make some basic checks.

Conclusions

In practice, the answer to the question:

"Is there a verify Hibernate behavior in tests (like Junit for example)?"

Strongly depends on what exactly you want to detect. Some tooling can be written — for example, Axelix can detect both In Memory Pagination, and N + 1, and a number of other problems. In some situations tests will help you a lot.
In general, first of all it's better to ask yourself — what problem exactly are we trying to solve here. And only then build a solution: is it possible (considering what I wrote above) to write an effective test,
is it possible to delegate this to external tooling, or maybe (the ideal case) the problem doesn't need to be solved in the first place.

Best of luck to everyone!

Natural IDs in your database. I am telling you the last time!

Mikhail Polivakha — Fri, 24 Jul 2026 10:42:21 +0000

Hi everyone! This is Mikhail Polivakha, tech lead of the Axelix project (btw, give us a star!). In my experience consulting teams that build enterprise applications, I keep getting asked:

What about natural keys in the database? Say I have a column that lets me explicitly identify a record, should I use it as the Primary Key?

And over my years of designing enterprise systems, and over the time spent designing Axelix, I've come to a conclusion: just never use natural keys, ever. When you feel the urge to do it, step outside, take a walk, get some fresh air, and it'll pass.

I understand this answer is categorical, so I'll add a couple of clarifications about what to do if you do happen to have a unique discriminating column that, as it seems to you, lets you uniquely identify a record in a database table. The full, nuanced answer is of course more complicated, but if I have to give you a straight TL;DR:

When designing new systems, in my opinion, you should always use surrogate primary keys.

Now let's get into why I think so.

Rules Written in Blood

Rules like the one above are usually born out of getting burned on real projects several times. I have an absolutely perfect story straight from Open Source Axelix. The source code is on GitHub, so if you feel like it, go and check for yourself. I won't go too deep into the details, but so that you grasp the depth of the problem, I'll give you a bit of context. In some places I'll also deliberately simplify the parts I consider non-essential to understanding the problem.

At its core, Axelix consists of two components. The first is Master, a standalone application that acts as the "brain" of the system. It's deployed either in a K8S cluster, or launched as a separate Docker container, or even just run as a plain JAR.

Master aggregates information from your Spring Boot services and stores it in its database. This information is later used to understand the "maturity" of your ecosystem, the distribution of versions of key components (for example, Spring Boot or Java versions), tracking known tech-debt issues, and so on.

If we picture a typical company, they usually have a K8S/OpenShift cluster where their production runs. Almost always, a given application is deployed to production not as a single copy, but as a set of Instances (a K8S Deployment + a configured HPA, and so on). So in practice we have one logical application that is physically a set of different containers.

Now I think we have enough context to discuss the problem.

The Beginning. Natural Keys: Sure, Why Not!

As I said, Axelix stores data in its database to understand the overall state of your application. Let's call this table "Application" (in reality this abstraction is named differently in Axelix, but again, I'm simplifying heavily). This is where application-level data lives.

Master can collect data from Spring Boot microservices via both a push and a pull model, but regardless of the model, it collects data at the Instance level, not at the Application level, i.e. not for the whole application. So Master polls each Instance, and it's then Master's job to somehow figure out that all those Instances belong to the same application.

The question is: How is Master supposed to do that?

How does it figure out that these Instances belong to the same application? (Don't forget: Axelix isn't always deployed in K8S. Relying on ClusterIP services and the like is not an option.)

Actually, if you think about it a little, the solution is right on the surface: we can just aggregate information at the level of the GroupID/ArtifactID pair from the GAV coordinates (the standard format of a Maven distribution). After all, all the Instances are required to have the same GroupID/ArtifactID, right?

Broadly speaking, yes, that's true. Some might think we could key off other things, for example spring.application.name or similar, but unfortunately that won't work, for a number of reasons. That's another story, though, and it's not important right now.

So imagine we're designing such a relation in the database. Here's my question for you: what primary key would you want for a table like this? When we designed the "Application" entity, it seemed right to make the {groupId/artifactId} pair the primary key, i.e. a Natural Composite Key.

And it's so convenient! When information about some Instance arrives in Axelix Master (whether via the push or the pull model):

We can update the data with a simple ANSI SQL MERGE or INSERT ... ON CONFLICT DO ..., because artifactId/groupId is the primary key! Spring Data JDBC (which we use as the ORM in Axelix Master) in 4.1 finally learned how to do UPSERTs on the primary key, and now we can just do this via JdbcAggregateTemplate:

@Transactional
public void reloadCurrentState(BasicRegistrationMetadata metadata) {

    Application application = converter.currentSnapshot(metadata);

    jdbcAggregateTemplate.upsert(application);
}

And how nicely it works out for the front-end! And here we arrive at the fact that a natural key carries business meaning by itself! That, by the way, is one of the genuinely nice properties of natural keys. What do I mean? For example, in a situation where we just need to display the name of our "Application", and the name alone is enough, we can use the artifactId, i.e. a part of the composite natural key. No need to "fetch anything extra", and so on.

So where's the problem? Given everything I've said so far, is this problem really so critical that I claim you shouldn't use natural keys at all? Yes, it's that serious. And here's why.

So What's the Deal? A Bit of Philosophy

The older a person gets, the more prone they are to doubting various things (for example, my claim in this article! And that's okay!). This is because people accumulate experience.

People who have been doing engineering for a good while accumulate experience and come to understand just how much everything changes, and how much they still don't know (experienced engineers understand me 100% right now). A vendor comes and goes. So does an employee. The uniqueness of a natural key...

Ray Dalio (an amazing person and macro investor, I highly recommend reading him) wrote in his book "Principles":

Sincerely believe that you might not know the best possible path and recognize that your ability to deal well with "not knowing" is more important than whatever it is you do know.

This is incredible wisdom. The idea is to accept the fact that your knowledge of the outside world is limited, and it will always be many times smaller than the set of things you don't know but which nonetheless affect your life/system/etc. And the most important thing in such a situation is to be able to work WITH YOUR OWN NOT-KNOWING of something, to hedge risks.

How does this relate to natural keys?

Very simply: if some discriminator seems like an obvious key in the moment, just remember that the scope of your knowledge is incomparably small next to what you don't know. And that "invariant" you're pinning your hopes on, the one you think will be unique: it can very easily stop being unique half a year later.

What's more, the scope of your knowledge will keep growing. Over time, you (yes, you, my friend) grow as an engineer. After a while you'll look at this code, or at the design of this system, and say:

How on earth!? How could I have done this? This crap is just awful, it was obvious this key would break uniqueness in case X!

And it'll be obvious to you. But later. When you become wiser. By the way, if you don't have these moments of "enlightenment" in your career, where you scold yourself for your own past decisions, that's a very strong warning sign that you've stopped growing as a specialist.

Back to Engineering

Let's get back a bit closer to the technical side.

The main point of the previous section is that what seems like the uniqueness of a natural key today can easily stop being uniqueness later. Now let's think like engineers: how bad is that, really? How bad is it that we'll be wrong about our natural key (composite or not, doesn't matter right now) turning out not to be unique?

The truth is that a record's primary key must always (!) have (among others) the following two distinguishing properties:

1. It must be immutable

When we assign a record some key by which we identify it, we then have no right to change it. Why? Because the outside world that depends on our system stores exactly this ID, this primary key, to identify the record. It stores a reference, not the record itself.

For example, imagine you have a third-party service that stores user profiles: user-service. And there they decided to use email as the natural key. You write a service that orchestrates users' subscriptions to various services within the ecosystem. And now you need to fetch a user's profile from that user-service system for your operations. How will you fetch it? By email, of course! It's the "unique key", after all.

And now imagine that the Product Owner comes along and says:

In our service we want to let a user change the email tied to their account.

That is, effectively, an already-existing record in the database will have its identity changed. By changing a record's ID in this user-service, any other system, including yours, can no longer find the profile it needs. That would be a mass incident. That's why an ID must be immutable always.

2. It must uniquely identify a record at any moment in time

Now imagine that, all of a sudden, the folks from the team that develops user-service get a requirement. They're told:

Hey, we sometimes run into a situation where a user once created an account and tied their email to it. And now they want to somehow delete the old account (which they created some 10 years ago) and create a new one, and attach the same email to it. We wouldn't want to delete the old account (in enterprise, for various reasons, hard deletes are rarely done). So, shall we do it?

Here the problem is even more obvious. Not only can your system, which depends on user-service, no longer find the right user profile (there could be several of them now!), all existing contracts break, and to "fix" them you'll have to "re-define" the ID (it's no longer unique, and you can't rely on the ID alone anymore). And if we have to change the ID, then see the section above.

Cause of Death: Natural Id

Mistakes come in varying degrees of severity. There are mistakes that have a local effect and can be fixed relatively quickly and easily.

But keys that identify data in distributed systems are something that spreads across the entire distributed system into its most varied corners. So the moment you suddenly realize with horror that the natural id is no longer unique, the so-called blast radius will be fantastic. Especially in modern microservice architecture.

So, friends, people die of different causes. Someone died of cancer, someone died of heart failure. And someone simply chose a Natural Id as their primary key, and then received an email in their inbox, or suddenly heard at a daily standup that the uniqueness assumption of this key was about to be shaken.

I suggest a moment of silence before reading on, in memory of those engineers who paid the price for choosing Natural ID as their primary key…

Thank you.

The Axelix Case

Let's get back to the real case we had at Axelix.

We haven't hit GA yet (it'll ship this summer, 2026, we're actively working on it), but we already have several Milestone releases. We embed with various companies to gather feedback, potential bugs, problems, and so on.

And one company tells us:

You know, it just so happens that we essentially have two services: service A and service B. They're basically identical, just deployed in different network segments. They have the same groupId and artifactId. Nevertheless, service A is maintained by this team, and service B by that team.

I've simplified all the details, but this is the general message. So, we have a problem in this case - we can no longer identify an application the way we wanted, via the artifactId/groupId pair. This is exactly what typically
happens after a while, when the system is already deployed in production.

Remember Ray Dalio!

... What exists within the area of "not knowing" is so much greater and more exciting than anything any one of us knows.

It's precisely because of situation like this that you need to ask users to provide Axelix with the information about what the unique ID of a given application is themselves (for example, in application.yaml).

But Natural IDs Do Have Advantages...

In my experience, the fact that a Natural ID carries business meaning that can be used somewhere (for example, displaying an application's name on the UI as the artifactId, as I already showed with the Axelix example) - this is solved simply by designing your API.

In other words, even with surrogate keys, you can design your API so that you don't have to fetch extra data from the backend; it's not a big problem (for example, get some metadata, put it into the state manager on the front-end, and so on; there are plenty of ways).

What's really important about natural keys is that they force you to think about the invariants of your data. That is, for example, logically, if your email is unique, then it makes sense to create an index on it (which is, for instance, what Postgres does when you ask it to create a Primary Key). And to avoid having two different indexes, why not make email the primary key, since in that case there would be just one index, only on email?

That's a broadly valid argument, but I'll put it this way: it's not worth it. If you don't use email as a Natural ID, then whether or not to create a unique index on email is a decision to make case by case. I'd say that for 95%+ of cases the answer is definitely yes, and there won't be any problems with it. That said, for large write-heavy systems with a lot of data, this may create a certain overhead, but again, usually negligible at the scale of the system.

And finally, regarding MERGE / INSERT ON CONFLICT operations. You can perfectly well do them not on the primary key, but on any constraint, for example, on a UNIQUE constraint that you explicitly define in a migration.

Conclusions

Based on my experience, I can tell you one thing: remember that the scope of your not-knowing is by nature far larger than the scope of your "knowing". That's why it's very dangerous to build an assumption that a Natural ID, which seems unique to you for a given record in the moment, will make a good Primary Key.

That said, it's worth acknowledging that the main advantage of a Natural ID is that it forces you to think about what invariants your data has in general. And these invariants should give you insights into how to model your data access and storage patterns, for example, defining unique b+tree indexes for the email column.

Remember: indexes and things like that can later be removed without consequences for the whole system. Changing primary keys, on the other hand, is a dead end.

DEV Community: Axelix Labs

The Tragedy of Software Versioning

In the Beginning Was the Word

A Relatively Simple Case: A Desktop Application

CalVer. Common Case

Marketing Version

When Others Begin to Depend on You

Semantic Versioning. The Cornerstone

SemVer Requires a Public API

Spring Boot and the Pain Index

What If the Product Has Multiple Components?

Independent Versioning

Lockstep Versioning

The Cost of Lockstep

Lockstep with Monorepos

A Lockstep Release Does Not Mean a Lockstep Upgrade

Introducing Compatibility/Skew Window

Existing Example. K8S Skew Policy

Compatibility Window. Trade-Off

Which Scheme Should You Choose?

If You Have a Consumer Application

If You Have a Library

If You Have Several Closely Related Components

If You Have an Ecosystem of Independent Projects

Closing Words

New Java Features Are Not Just for Interviews. Two Real-World Use Cases from Axelix

A Little Context About Axelix

Use Case One. Passing Context Through ThreadLocal

ThreadLocal Gives Us More Capabilities Than We Need

ScopedValue. Not Storage, but a Bounded Execution Scope

How It Works in a Real Request

Some Limitations of ScopedValue

The Binding Is Tied to a Thread

The Binding Is Immutable, but the Object Might Not Be

It Is Not a Universal Replacement for Parameters

A Small Conclusion

Use Case Two. What Do sealed Interfaces Have to Do with It?

A Potentially Mutable HashMap Key. A Time Bomb

sealed Is Not Just for switch

Why Not Use an enum?

What These Two Features Have in Common

Practical Recommendations

When to Consider ScopedValue

When to Consider sealed

The Invariant Comes First, Then the Feature

Closing Thoughts

One Branch to Rule Them All

Small Prerequisite. Axelix Versioning.

The Problem

1. Give us the ability to do patches

2. Avoid the merge hell.

Axelix Git Branching Model

TBD: Why patch-branches are okay.

Challenges & Problems

1. Minor/Major Dilemma

2. Long-Lived Breaking Changes Branches

Conclusions. Axelix, Lessons Learned.

Axelix goes GA. A journey of a thousand miles begins with a single step

The Story. Big "Why" Behind Axelix

Ecosystems around Languages

The defining trait of Java Ecosystem

Spring Framework as the Phenomenon

Why am I telling you about this?

Q: Is it free?

Q: Is the project ready for use and deploy?

Q: Is it safe to deploy in production?

Where Are We Headed?

Taming Hibernate in Tests

Controlling Hibernate during tests. Overview.

1. Problems at the ORM level

2. Second. Problems at the application level

The General Approach to N + 1 Detection

Micro-Conclusion

General Hibernate Problems

Conclusions

Natural IDs in your database. I am telling you the last time!

Rules Written in Blood

The Beginning. Natural Keys: Sure, Why Not!

So What's the Deal? A Bit of Philosophy

Back to Engineering

Use Case One. Passing Context Through `ThreadLocal`

`ThreadLocal` Gives Us More Capabilities Than We Need

`ScopedValue`. Not Storage, but a Bounded Execution Scope

Some Limitations of `ScopedValue`

Use Case Two. What Do `sealed` Interfaces Have to Do with It?

A Potentially Mutable `HashMap` Key. A Time Bomb

`sealed` Is Not Just for `switch`

Why Not Use an `enum`?

When to Consider `ScopedValue`

When to Consider `sealed`