The JVM Speaks AI: Cobrainer's Journey with Kotlin, Koog and Exposed

One afternoon, one of our LLM providers started throttling us. A large customer had triggered a burst of AI-assisted work, and the provider's capacity in our region could not keep up. Our prompt executor followed its fallback chain, the circuit breaker took the affected models out of rotation, and each request moved to a second provider without interrupting the workflow. There was no incident to handle and no change in system behaviour for the customer. We noticed it only the next morning on our LLM observability dashboard.

We built all of our AI integration in Kotlin. A controversial decision? We do not think so. In 2025, one could still argue that AI integration required Python. By 2026, that debate is no longer a useful basis for architecture decisions. Python remains important for research and model training, but production AI is mostly about integrating APIs, enforcing contracts, and operating reliable systems. Every prompt, agent, and structured-output contract in our core platform lives on the JVM: one Kotlin monorepo with types shared end to end, built on Koog, Exposed, coroutines, and kotlinx.serialization.

TL;DR

• Kotlin lets six developers and their coding agents work in one stack. Sealed types, kotlinx.serialization, null safety, and coroutines make contracts and control flow explicit.
• Koog was the best fit because its lightweight abstractions build on Kotlin instead of hiding it. We moved the proof of concept into production for all customers in one day.
• Production reliability comes from cross-provider fallback chains, circuit breakers, LLMObs, and contract tests. Removing blocking code from coroutine paths also increased throughput by 5×.

We at Cobrainer

Cobrainer is a Munich-based B2B HR-tech company that grew out of a Technical University of Munich research project in 2013. We serve enterprises across industries and regions. Our AI-native Skills Platform creates and governs living Job and Skill Architectures: structured models of an organization's functional roles, levels, tasks, and skills. They answer three basic questions: Which distinct jobs exist in the organization? What is needed to perform them? How do they relate to each other? A patented Skills Graph and ML/NLP power the platform.

Our customers use it for skills-based (AI) workforce transformation: internal mobility, recruiting, learning, workforce planning, and EU pay-transparency readiness, with integrations into SAP SuccessFactors, Workday, ServiceNow and other HCM systems.

Our Summer 2026 release includes a context-aware Personal Agent for the Job Architecture and the wider platform, agentic background tasks, agentic (g)RAG, and collaborative multilingual commenting for governance. The AI can resolve structural and language-specific comments on job profiles in more than 25 languages.

Our agentic engineering setup centres on six developers with end-to-end ownership, from cloud infrastructure through backend and frontend to AI. Coding agents help us explore the repository, implement changes, run checks, and investigate operational issues. Together, we serve more than 10,000 active users at companies around the world.

Small, fast-moving team, big surface. That constraint explains almost every technology decision in this article.

Kotlin for AI-powered features

Why Kotlin? Reliability meets velocity

The control plane and core domain of our platform use Kotlin and Spring Boot. The stack is type-safe and null-safe end to end, with kotlinx.serialization for wire contracts, Exposed for SQL, and coroutines for non-blocking I/O. A central monolith is the entry point for the platform, and everything lives in one monorepo. With six engineers working across the full system, every additional deployable adds work to development, operations, and CI.

A large monolith rewards explicitness. Its behaviour can span distant files, and coding agents cannot safely fill gaps left by implicit conventions. Kotlin's types and compiler checks make those relationships visible. Koog follows the same principle with small abstractions that keep contracts, capabilities, and control flow in the code. This helps developers review changes and gives coding agents firmer boundaries in which to work.

When our LLM work moved from isolated features to core platform components, we asked whether it justified a separate stack. The models sit behind HTTP APIs, so another runtime would not improve their output. It would give us a second deployment pipeline, add serialization boundaries between the AI layer and domain model, and remove compile-time guarantees at the boundary where LLMs introduce the most uncertainty.

We wrote down what we actually needed from an LLM framework:

• Lean and explicit. The framework should behave like a library while keeping control flow visible in the code.
• A release cadence that matches innovation speed and model churn. New models and paradigms arrive monthly, so infrequent framework releases become a bottleneck.
• Clean provider and model-capability abstractions. We need multiple providers, and their models support different features.
• Type safety end to end. An LLM's output should enter the domain as a typed value or not at all.

We tested these requirements by building working implementations instead of comparing feature matrices.

The turning point: why Koog fits Kotlin

We built and ran production implementations with Spring AI, LangChain4j, and Koog. The important difference was not the length of their feature lists. It was how clearly each framework expressed its behaviour in Kotlin for developers, coding agents, and production tooling.

Spring AI is the most comprehensive and deeply integrated Spring option. It offers a robust ecosystem and good portability for teams already using Spring Boot. We still use it for our MCP endpoints, where its integration with REST infrastructure and the Spring Security filter chain is exactly what we need. In the agentic core, however, it added another layer between our code and the model when we wanted direct control and rapid iteration.

LangChain4j is a large, feature-rich library that brings LangChain concepts to the JVM. It helps teams move quickly, but we found failures inside a chain difficult to trace back to the underlying LLM call. We eventually replaced the last utilities that depended on it with a few dozen lines of Kotlin.

Koog changed the direction of the evaluation because it made Kotlin itself the foundation of the integration. Our @Serializable domain types become structured-output contracts directly. Sealed hierarchies model the possible results, and exhaustive when expressions force every caller to handle them. Coroutines carry naturally through provider calls, retries, and agent workflows. Koog's DSL builds on familiar Kotlin features such as extension functions and lambdas with receivers, so agent graphs remain normal Kotlin code that can be traced, changed, and verified with the compiler.

The provider abstraction was equally important. Koog represents model capabilities explicitly instead of reducing every model to a lowest common denominator. We can adapt temperature, token limits, and structured-output handling for each model while keeping the same domain contract and call site. These decisions are visible in types and code instead of being hidden in configuration or prompts. When a developer or agent investigates a failure, the underlying request remains accessible rather than disappearing behind a chain of abstractions.

That combination gave us the level of abstraction we wanted: enough structure to avoid rebuilding provider clients and agent orchestration, but close enough to Kotlin that our existing tools, types, and debugging habits still applied. A coding agent can trace a change from a domain type through its LLM contract to every consumer without crossing languages or repositories. Koog did not introduce a second programming model into the codebase. It made the existing one more useful to both humans and agents.

Once we chose Koog, we cleaned up the proof of concept and rolled it out to all customers in one day. That speed came from keeping the domain model, serialization, concurrency model, tests, and deployment pipeline we already understood.

AI in production: structured output as the workhorse

In production, much of an LLM's value comes from filling data structures that domain logic can trust, rather than generating free-form chat. Koog has powered our Job Architecture generation since summer 2025, and structured output is the foundation of that core product capability.

The contract is a Kotlin sealed hierarchy with kotlinx.serialization. Koog turns it into a JSON schema (docstrings included, via @LLMDescription), gets the provider to comply, and hands back a typed value:

@Serializable
enum class ChangeKind { RENAME, ARCHIVE, MOVE }

@LLMDescription("A single proposed change to the job architecture")
@Serializable
sealed interface ProposedChange {
    val kind: ChangeKind

    @Serializable
    @SerialName("rename")
    data class Rename(
        @property:LLMDescription("The item to rename")
        val itemId: String,
        @property:LLMDescription("New title, in the tenant's base language")
        val newTitle: String,
    ) : ProposedChange {
        override val kind get() = ChangeKind.RENAME
    }

    @Serializable
    @SerialName("archive")
    data class Archive(
        @property:LLMDescription("Items to archive, including their descendants")
        val itemIds: List<String>,
    ) : ProposedChange {
        override val kind get() = ChangeKind.ARCHIVE
    }
    // ...
}

Requesting it is one call. If a weaker model returns almost-JSON, a StructureFixingParser repairs it with a lightweight model before we ever see it:

val plan = executor.executeStructured<ChangePlan>(
    prompt = prompt("change-planner") {
        system(plannerInstructions)
        user(request)
    },
    model = model,
    fixingParser = StructureFixingParser(model = lightweightModel, retries = 2),
).getOrThrow().data

plan.changes.forEach { change ->
    when (change) { // The compiler enforces exhaustive coverage.
        is ProposedChange.Rename -> preview(change.itemId, change.newTitle)
        is ProposedChange.Archive -> confirmArchive(change.itemIds)
        is ProposedChange.Move -> validateMove(change)
    }
}

Kotlin provides specific safeguards here. Null safety exposes schema drift at compile time: add a required field to the contract and every construction site that omits it stops compiling. A when expression over a sealed result is exhaustive, so a new change kind breaks the build until every consumer handles it. Default arguments and small DSLs keep configuration readable, allowing us to review prompt code like domain code.

We also balance traffic across providers whose models differ in capabilities, and Koog represents those differences explicitly. A provider that uses an existing API format requires little integration code:

// xAI uses the OpenAI wire format.
@Serializable
object XAI : LLMProvider("xai", "xAI")

val xaiClient = OpenAILLMClient(
    apiKey = settings.apiKey,
    settings = OpenAIClientSettings(baseUrl = "https://api.x.ai"),
)

Our executor adds three behaviours to Koog's PromptExecutor: retry with backoff, a config-driven fallback chain, and a per-model circuit breaker. Before each attempt, it adapts the prompt to the target model's declared capabilities:

fun adaptPromptFor(model: LLModel, prompt: Prompt): Prompt = prompt
    .letIf(!model.supports(LLMCapability.Temperature)) { it.withoutTemperature() }
    .letIf(prompt.maxTokens > model.maxOutputTokens) { it.cappedTo(model.maxOutputTokens) }
    // Model can't do native structured output? Ask for it in the prompt
    // instead. The sealed contract and fixing parser stay identical.
    .letIf(!model.supports(prompt.schemaCapability)) { it.withSchemaAsUserMessage() }

The call site never knows which provider answered. That indirection is what made the silent rebalancing from the opening of this article a non-event.

Growing into an agentic platform

At Cobrainer, AI powers skill inference, Job Architecture generation, transcription, chat agents, and agentic (g)RAG. Our Job Architecture chat agent brings several of these capabilities together.

The agent lets users create, review, and maintain a job architecture conversationally: "merge these two levels", "draft the missing task descriptions for this job family", or "archive everything nobody has touched since the reorg". At a high level, it is a supervisor tool loop with deterministic sub-workflows. The LLM decides what to do, while a Koog strategy graph routes every consequential action through code we control. A human must approve every change.

The agent is our one exception to the monolith. It runs in the Agent Runtime, a reactive Spring Boot Kotlin service designed for long-running conversations, streaming, and structured concurrency. The monolith remains the single entry point for the platform, including REST and MCP. Both services live in the same monorepo, so they share types directly and the compiler checks their contract. SurrealDB stores checkpoints for resumable turns, message history, and three layers of memory: a small window of recent conversation chunks, a running session briefing, and a fact-based long-term graph. Platform data remains in Postgres behind the monolith, accessed through Exposed.

Koog's graph DSL makes the tool loop almost declarative. We use marker tools that the model can call but that never execute directly. A guarded edge intercepts the call and routes it into a deterministic sub-workflow:

val toolLoop = strategy<String, String>("supervisor") {
    val callLlm by nodeLLMRequest()
    val executeTool by nodeExecuteTool()
    val sendResult by nodeLLMSendToolResult()

    val editFlow = installEditCapability(approvalGate)
    val bulkFlow = installBulkCapability(approvalGate)

    // Marker tools are visible to the model but never executed.
    // The edge routes into a deterministic sub-workflow instead.
    edge((callLlm forwardTo editFlow).soleMarker(EDIT_TOOL_ID))
    edge((callLlm forwardTo bulkFlow).soleMarker(BULK_TOOL_ID))

    // Edges match in declaration order, so the generic loop comes last.
    edge(callLlm forwardTo executeTool onToolCalls { true })
    edge(callLlm forwardTo nodeFinish onAssistantMessage { true })
    edge(executeTool forwardTo sendResult)
}

The same graph drives the progress UI. Instead of streaming tokens, we show meaningful progress. Koog's event hooks fire as the agent moves through nodes and tools, and each transition maps to a localized status such as "Analyzing your architecture…" or "Preparing changes for review…". Showing graph progress improved perceived latency more than token streaming would have for this workflow:

install(EventHandler) {
    onNodeExecutionStarting { ctx -> emitStatus(ctx.node.name) }
    onToolCallStarting { ctx -> emitToolStatus(ctx.toolName) }
    onAgentExecutionFailed { ctx -> events.close(ctx.error) }
}

Security is where the two frameworks meet. The agent accesses tools only through central MCP servers, which Spring AI serves inside the monolith. Every tool call therefore passes through the same Spring Security filter chain as our REST APIs. We run one MCP server per permission scope (read, architect, reviewer), guard each at the path level, and give the agent a short-lived delegated token so it always acts as the user. Permissions filter the available tools before any LLM call. If a user cannot edit the architecture, their agent never sees editing tools. This reuses our existing authorization infrastructure without relying on prompts for access control.

Koog runs the agent in the Agent Runtime, Spring AI exposes and protects its tools in the monolith, Exposed and Postgres store platform data, and SurrealDB stores agent memory. The result is two JVM services in one monorepo, sharing one type system.

Discoveries and challenges

Production use also exposed several important problems and lessons.

A 5× throughput gain from coroutine cleanup. Our most difficult performance problem had nothing to do with Koog or LLMs. Legacy blocking code inside coroutines starved dispatchers under AI-driven load, causing vague and load-dependent slowdowns. Once we found and removed it, throughput on the affected paths increased by 5×.

Which leads to the sharpest lesson we can offer:

Do not leave runBlocking in production request paths. This applies even to a blocking Spring Boot servlet stack where it appears to work. Under load, it can deadlock a dispatcher without crashing the application; requests simply stop completing.

Build deep fallback chains across providers. Several fallback models from the same provider still share one failure domain. Our chains cross provider boundaries, and per-model circuit breakers skip throttled or unhealthy routes so retries reach a viable model quickly. We trace every attempt and fallback in LLMObs, including latency, token usage, and errors. That makes it possible to reconstruct a request across providers and distinguish a model failure from a parser or application problem. This setup paid off as soon as one provider lost capacity.

Contract-test the LLM layer. Structured output depends on our schema, the framework's schema generation, and each provider's implementation. We run end-to-end structured-output tests with real calls and parsing for every provider, as well as LLM contract tests in CI. Together with LLMObs, these checks form our AI operations loop: observe production behaviour, investigate failures with full context, and validate model or provider changes before rollout. This lets developers and agents move quickly without turning model churn into guesswork.

Final words

Building AI features in Kotlin once sounded unconventional. Today, its explicit contracts and lightweight abstractions give our six developers and their coding agents a system that is fast to change, safe to refactor, and straightforward to operate.

If you are considering AI development on the JVM, the ecosystem is ready. Have a look at Koog, Exposed, and Kotlin. We thank the JetBrains teams behind all three for their frequent releases and responsiveness.

Questions, disagreements, and production stories are welcome in the comments. We read them.