DEV Community: Jihed Ben Arfa

Instrumenting a Keycloak SPI provider so the next silent JDBC bug doesn't need a code review to find it

Jihed Ben Arfa — Sun, 19 Jul 2026 18:08:13 +0000

Project: keycloak-spi-workbench — custom Keycloak SPI providers, one of them a read-only user storage federation (legacy-user-storage) that reads users from an existing JDBC table instead of forcing a big-bang migration into Keycloak's own store.

What happened first, no Sentry involved: LegacyUserStorageProvider#searchForUserByUserAttributeStream was calling getUserByUsername twice per invocation — once for a null check, once again to build the returned stream:

return getUserByUsername(realm, attrValue) != null
        ? Stream.of(getUserByUsername(realm, attrValue))
        : Stream.empty();

Two calls, two separate JDBC round trips, against a repository with no connection pooling (LegacyUserRepository opens a fresh Connection per query, on purpose — see the repo's README on why). Every username-attribute lookup, twice the DB load it needed. No exception, no wrong result, just silently double the work on every call. Caught by reading the method, not by any tool — fixed by storing the result once and reusing it, with a regression test that fails if it comes back.

That's exactly the kind of bug that's invisible until someone happens to read that method, and the reason I added Sentry next: catching this again shouldn't depend on a code review catching it first.

Where Sentry goes in: LegacyUserRepository's three query methods (findOneWhere, search, count) now open a child span under whatever transaction Keycloak's own request tracing has active — legacy_db.find_one, legacy_db.search — and search records how many rows came back:

private ISpan startSpan(String operation, String description) {
    ISpan parent = Sentry.getSpan();
    return parent != null ? parent.startChild(operation, description) : NoOpSpan.getInstance();
}

Falls back to NoOpSpan when there's no active transaction — never throws, never blocks a login on Sentry being reachable. Any SQLException also gets Sentry.captureException(e) before it's wrapped and rethrown, so a failing legacy database shows up as an event, not just a stack trace in a Keycloak log nobody's tailing.

The part that mattered more than the instrumentation itself: making it opt-in.

@Override
public void init(Config.Scope config) {
    String dsn = System.getenv("SENTRY_DSN");
    if (dsn != null && !dsn.isBlank() && !Sentry.isEnabled()) {
        Sentry.init(options -> {
            options.setDsn(dsn);
            options.setTracesSampleRate(1.0);
            options.setEnvironment(System.getenv().getOrDefault("SENTRY_ENVIRONMENT", "production"));
        });
    }
}

No SENTRY_DSN set, Sentry.init never runs, every span call resolves to NoOpSpan, every captureException call is a no-op. This is a provider other people install into their own Keycloak instance — it doesn't get to assume they want a Sentry account wired into their auth server, and it doesn't get to silently start sending data anywhere without an explicit environment variable making that choice.

Where this actually earns its keep: a per-user JDBC query duplicating itself is the boring, common case. The interesting one is a legacy database that's slow for one specific customer's data shape, or a driver that starts throwing intermittently under load — the search span's result_count and the per-query timing are exactly what would surface a slow query pattern before it shows up as "login is slow" tickets with nothing pointing at the actual repository call underneath.

The bug that only showed up because my test was too fast

Jihed Ben Arfa — Sun, 19 Jul 2026 17:59:34 +0000

A couple hundred milliseconds cost me an hour, and the annoying part is I'd written the fix for it before I even had the bug.

I was writing a Testcontainers deployment check for a custom Keycloak authenticator — boot a real Keycloak server with the built provider jar dropped into providers/, hit the admin REST API, confirm the provider shows up. First pass, I picked the wait strategy the way you'd expect:

.waitingFor(Wait.forLogMessage(".*Running the server in development mode.*", 1))

Keycloak prints that line during start-dev, so it felt like a reasonable "the server is up" signal. The test ran, and about one time in three it failed with a plain connection reset trying to hit the admin token endpoint. No stack trace pointing anywhere useful, just "connection refused," which is the least helpful error message a network client can give you.

The instinct is to blame Testcontainers, or Docker networking, or just add a Thread.sleep(2000) and move on. I've done that before and regretted it — a sleep just moves the flake to a slower CI runner instead of fixing it.

What was actually happening: that log line prints the moment Keycloak's startup sequence reaches "development mode," which is before the HTTP listener has finished binding and accepting connections. The gap is small — a couple hundred milliseconds most of the time — which is exactly why it only failed sometimes. Fast enough CI runner, you never see it. Slightly loaded runner, the test wins the race against the listener and gets a reset.

The fix was smaller than the investigation:

.waitingFor(Wait.forHttp("/realms/master")
        .forStatusCode(200)
        .withStartupTimeout(Duration.ofMinutes(3)))

Poll the actual endpoint you're about to call, instead of trusting a log line that tells you what the server is doing, not what it's ready for. Obvious in hindsight. It always is.

The part that stuck with me: a log-message wait strategy will pass code review without question, because it reads as "wait until the server says it's running" — which sounds exactly right. It just isn't the same claim as "wait until the server is accepting connections," and nothing about the code makes that distinction visible. You only find the gap between those two claims by having the test flake on you, ideally in CI where you can't step through it, which is the least convenient possible place to learn it.

Same class of bug, one level up: don't trust a stated intention, wait on the actual side effect you depend on. Applies well past Testcontainers — anywhere something logs "starting" before it's actually ready to be used.

Keycloak lets you pick a login option that's guaranteed to fail — here's the fix

Jihed Ben Arfa — Sun, 19 Jul 2026 17:59:32 +0000

Project: keycloak/keycloak — the identity and access management server. PR: #51021, fixing #46481.

The bug: if you write a custom Keycloak Authenticator that isn't a CredentialValidator — anything that isn't a password, OTP, WebAuthn, that kind of thing — and it requires a user to already be identified, Keycloak's "Try another way" screen will offer it as an option even when it isn't actually usable for that user. Pick it, and you get an immediate CREDENTIAL_SETUP_REQUIRED error. The option shouldn't have been on the screen in the first place.

Root cause, traced through two files in the services module:

AuthenticationSelectionResolver#addSimpleAuthenticationExecution decides what goes on the selection list. It only checks one thing:

Authenticator localAuthenticator = processor.getSession().getProvider(Authenticator.class, execution.getAuthenticator());
if (!(localAuthenticator instanceof CredentialValidator)) {
    nonCredentialExecutions.add(execution);
} else {
    ...
}

Not a CredentialValidator? Straight onto the list, no further questions. Nothing here checks authenticator.configuredFor(...) or the factory's isUserSetupAllowed().

There's a second, separate instance of the same gap. DefaultAuthenticationFlow.processSingleFlowExecutionModel has its own configuredFor/isUserSetupAllowed guard meant to catch exactly this — but only for requiresUser() == false authenticators, and only if they're a CredentialValidator:

if ((authUser != null) &&
        !authenticator.configuredFor(processor.getSession(), processor.getRealm(), authUser) &&
        !factory.isUserSetupAllowed() &&
        (authenticator instanceof CredentialValidator)) {
    throw new AuthenticationFlowException(...);
}

That instanceof CredentialValidator at the end means a plain authenticator in that exact shape runs authenticate() completely unguarded, even as a REQUIRED step — a code path that never goes through the selection list at all, so fixing the resolver alone wouldn't have caught it.

The fix: added isSelectableForCurrentUser to the resolver — authenticators that don't require a user pass straight through, the rest only get listed if they're configured for the current user or the factory allows self-setup on the spot. And dropped the instanceof CredentialValidator restriction in the flow's own guard, so it applies the same way the requiresUser() == true branch right above it already does.

Test: AuthenticationSelectionResolverTest, five cases covering the full matrix — no user yet, configured, not configured but self-setup allowed, not configured and not allowed, and authenticators that don't require a user at all (always selectable, regardless of the rest).

This one didn't need any special tooling — just reading the two files the original report pointed at and following the logic all the way to where it actually breaks, which is most of what real bug fixing is.

Testing Keycloak SPIs with Testcontainers — the part every tutorial skips

Jihed Ben Arfa — Sat, 18 Jul 2026 23:39:15 +0000

Most "here's how to write a custom Keycloak authenticator" posts end at mvn package. You get a class that implements Authenticator, a factory, a META-INF/services file, and a "drop the jar in providers/ and restart" instruction — and that's where the tutorial stops. Nobody shows you how to actually test the thing, which is a problem, because SPI development has a genuinely nasty feedback loop: package, copy the jar, restart Keycloak, click through a login flow by hand, repeat. I've spent enough evenings doing that manually on custom authenticators to want it gone.

The unit-testable part is easy. If your authenticator's logic is a pure function of some inputs (a user attribute, a client ID, a config value), you extract that into a method and mock the Keycloak model objects around it. That's normal JUnit + Mockito, nothing SPI-specific. The part that actually needs a real server is: does the jar deploy? Does META-INF/services point at the right class? Did you get a method signature wrong that only shows up as a stack trace on server startup? None of that shows up in a unit test, because a unit test never boots Keycloak.

Booting a real Keycloak in a test

Testcontainers makes this less painful than it sounds. You start the official image, copy your built jar straight into providers/ before the container starts, and let Keycloak boot normally:

@Container
private final GenericContainer<?> keycloak = new GenericContainer<>("quay.io/keycloak/keycloak:26.6")
        .withCopyFileToContainer(MountableFile.forHostPath(builtJarPath()), "/opt/keycloak/providers/my-spi.jar")
        .withEnv("KEYCLOAK_ADMIN", "admin")
        .withEnv("KEYCLOAK_ADMIN_PASSWORD", "admin")
        .withExposedPorts(8080)
        .withCommand("start-dev")
        .waitingFor(Wait.forHttp("/realms/master").forStatusCode(200));

The wait strategy is the one detail that'll cost you an hour if you get it wrong. My first pass used Wait.forLogMessage(".*Running the server in development mode.*", 1) — that line prints during startup, but before the HTTP listener is actually accepting connections. The test would then immediately try to hit the admin API and get a connection reset, because the server said "I'm running" a couple hundred milliseconds before it actually was. Polling the real endpoint with Wait.forHttp(...) instead of trusting a log line fixed it — obvious in hindsight, and exactly the kind of thing you only learn by having the test flake on you in CI.

Once the container's up, you hit the admin REST API the same way an admin console would:

HttpRequest providersRequest = HttpRequest.newBuilder()
        .uri(URI.create(baseUrl + "/admin/realms/master/authentication/authenticator-providers"))
        .header("Authorization", "Bearer " + accessToken)
        .GET()
        .build();

and assert your provider ID shows up in the list. That's it — that's the check that actually catches packaging mistakes: wrong META-INF/services entry, a factory that throws in init(), a missing dependency that only fails at classload time. None of that surfaces in a unit test.

Same idea, different provider type

I built a second provider — an event listener that publishes login events to Kafka — right after, and the same "don't mock the thing you're trying to prove works" logic applied to it, just with a different target. Mocking the KafkaProducer in a test would let a subtly wrong producer config (bad serializer, bad acks setting) pass silently. So instead of mocking, the integration test spins up a real Kafka broker via Testcontainers, constructs the provider with a real producer pointed at it, fires an event, and reads the message back with a real consumer. If the JSON shape is wrong, or the key/value serializers are misconfigured, the test catches it — a mock never would have.

The Maven wiring

One thing that trips people up: these integration tests need the packaged jar to already exist, so they can't run in the same phase as unit tests. Surefire runs unit tests at test (excluding anything named *IT.java), and Failsafe runs the integration tests at verify, after package has already produced the jar:

<plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        <excludes><exclude>**/*IT.java</exclude></excludes>
    </configuration>
</plugin>
<plugin>
    <artifactId>maven-failsafe-plugin</artifactId>
    <executions>
        <execution><goals><goal>integration-test</goal><goal>verify</goal></goals></execution>
    </executions>
</plugin>

mvn test for the fast feedback loop while you're writing the logic, mvn verify when you actually need to know the thing deploys — which is also what CI runs, so a green build actually means something.

None of this is complicated once it's set up. It's just work nobody writes about, because "here's a class that implements an interface" is a much shorter blog post than "here's how you prove the class actually works on a real server." The second one's the only one that matters once you're shipping.

Your Keycloak roles aren't working in Spring Security. Here's the actual reason.

Jihed Ben Arfa — Sat, 18 Jul 2026 23:39:14 +0000

If you've wired up a Spring Boot resource server behind Keycloak and your @PreAuthorize("hasRole('ADMIN')") is silently returning 403 for a user you can see has the ADMIN role in the admin console, you're not going crazy. This happens on basically every first Keycloak + Spring Security integration, and the reason is boring once you see it: Spring Security's default JWT converter has no idea Keycloak exists.

I've hit this on every Keycloak-secured backend I've built over the past couple of years — most recently on a multi-service Spring Boot 3 platform where I ended up owning the IAM layer, including custom Keycloak SPI authenticators. Same wiring problem, every time, until I finally pulled the fix out into a small library instead of copy-pasting it again.

The actual token shape

Spring Security's JwtAuthenticationConverter expects authorities to come from a scope (or scp) claim — a flat, space-separated string, OAuth2-style. That's the generic spec-y default. Keycloak doesn't do that. A Keycloak access token puts roles here instead:

{
  "realm_access": {
    "roles": ["offline_access", "uma_authorization"]
  },
  "resource_access": {
    "my-client": {
      "roles": ["ADMIN", "VIEWER"]
    }
  }
}

Realm roles under realm_access.roles, client roles nested under resource_access.<clientId>.roles. Spring Security never looks there. So JwtAuthenticationConverter runs, finds no scope claim, produces zero GrantedAuthority objects, and every hasRole(...) check quietly fails — no error, no stack trace, just "access denied" for a user who is very clearly in the right group.

The fix, the manual version

You write your own Converter<Jwt, AbstractAuthenticationToken> that reads both claims, flattens them into SimpleGrantedAuthority with a ROLE_ prefix (Spring Security's convention), and registers it on the JwtAuthenticationConverter:

public class KeycloakRealmRoleConverter implements Converter<Jwt, Collection<GrantedAuthority>> {

    @Override
    public Collection<GrantedAuthority> convert(Jwt jwt) {
        Map<String, Object> realmAccess = jwt.getClaim("realm_access");
        List<String> roles = realmAccess != null
                ? (List<String>) realmAccess.getOrDefault("roles", List.of())
                : List.of();

        return roles.stream()
                .map(role -> new SimpleGrantedAuthority("ROLE_" + role))
                .collect(Collectors.toList());
    }
}

Wire it into a JwtAuthenticationConverter, register that as a bean, and roles start working. That's maybe twenty lines and it's fine for a single service. The annoying part is you'll write this exact converter again on the next service, and the one after that, usually slightly differently each time (some people prefix with ROLE_, some don't, some forget client roles exist at all and only handle realm roles).

Where I landed

After the third time writing basically the same class, I pulled it into spring-keycloak-toolkit — an auto-configuration that registers the converter for you, handles both realm and client roles, and also fixes the second thing that bites you right after the first one: Spring Security's default 401/403 responses are empty bodies, which is useless if anything downstream (a frontend, an API gateway, a support engineer reading logs) needs to know why a request got rejected. The library adds RFC 7807 problem+json bodies for both cases instead.

What it deliberately does not do is touch your SecurityFilterChain or guess which endpoints should be public. I thought about auto-wiring that too, and decided against it — guessing your endpoint matchers wrong and failing silently is worse than making you write four lines of config yourself. A library that gets your security posture subtly wrong is more dangerous than one that does less.

If you're wiring up Keycloak with Spring Security and hitting this, the manual converter above will get you unstuck in five minutes. If you're doing it more than once, the library's on Maven via JitPack — link's in the repo.