DEV Community: Cobrainer

Inside Cobrainer's Engineering Workflow with CodeRabbit

Chetan Basuray — Tue, 03 Nov 2026 11:01:05 +0000

Context-aware reviews, shared engineering conventions, and a collaboration that improved the integration.

TL;DR

Cobrainer's engineering conventions are made explicit through shared CodeRabbit instructions and repository-specific lint configuration.
CodeRabbit applied that context at an appropriate stage to recommend a shared query abstraction introduced in the same merge request, making its feedback directly actionable by the developer.
Our teams found and traced unwanted lint findings to a configuration-precedence defect, which CodeRabbit fixed in under two weeks. In this way, authors assess the recommendations and human reviewers retain the merge decision.

The reviewer goes first. The merge decision stays with a colleague.

A code review asks more than whether a piece of code is correct. It asks whether the change belongs in this codebase, whether it follows the project's decisions, and whether the proposed remedy uses what the team has already built. Much of what makes an experienced reviewer useful lies in answering those questions together.

At Cobrainer, we build a Skills Platform that helps organisations understand their jobs, tasks, and skills. We introduced CodeRabbit into our engineering workflow to raise issues before a colleague began reviewing, leaving more of that colleague's attention available for behaviour, design, and maintainability. To make its recommendations useful, we had to give it access to the knowledge behind our own reviews.

Some of that knowledge was explicit: a linter configuration, for example, records which rules a project has chosen. Other expectations were less formal. Engineers knew which abstraction to reach for because they had helped build it, or because they had discussed a similar change before. Our work with CodeRabbit involved expressing those expectations and observing what happened when the reviewer applied them. Two findings made the problem concrete.

Making our conventions available to the reviewer

We put CodeRabbit in front of every merge request. The author considers its findings, makes changes where appropriate, and presents the result for a colleague's review. A human reviewer retains the approval decision. In this arrangement, the practical value of an early review depends on the relevance of the work it asks the author to do.

We kept our shared CodeRabbit configuration in a central repository that projects in the group inherited. This gave review-profile settings and shared instructions a common source. A change to those instructions could itself receive a review, and we could apply it without maintaining separate copies across repositories.

Writing the instructions required us to examine what we expected from code review. Which patterns were deliberate? Which apparent violations were acceptable in a particular context? When should a developer use a shared abstraction? One path instruction expressed a recurring expectation:

Do flag changed code that duplicates an existing shared capability instead of reusing it.

Applying this instruction requires knowledge of both the change and the surrounding code. The reviewer must find the shared capability, determine whether it applies, and explain the connection to the author. The instruction makes an engineering preference available for use in a specific review.

Native tool configuration provides another kind of guidance. A repository's pyproject.toml records its Ruff rule selection. These settings express decisions the team has already made, and the integrated tool needs to respect them.

We also compared review profiles using a probe containing nine deliberate defects. The comparison profile reported four; assertive reported seven and left five additional negative cases unflagged. We chose assertive. The small test gave us a reason for that choice, though it could not tell us how accurately either profile would perform across production changes.

Applying an abstraction within the change that introduced it

Three weeks into the rollout, a merge request introduced a typed expression API for storage queries. Two predicates in the same change still used string interpolation. One had the following shape, shown here in a simplified example:

async def urls_for_platform(self, platform: str) -> AsyncIterator[list[str]]:
    async for batch in self.stream(
        where=f"platform = '{platform}'",
        columns=["url"],
    ):
        yield batch

The interpolation is easy to recognise. Deciding what to replace it with requires more context. CodeRabbit's comment connected the platform and source values to a registry populated through an ingest CLI and HTTP API. The query code therefore could not assume that they were quote-free constants.

The comment also identified store.expressions.Columns, introduced in the same merge request, and its literal-escaping path through lit(...).to_sql(). This gave the author a remedy that already belonged to the project. CodeRabbit recommended using it for both predicates:

_columns = Columns(CorpusRecord)

async def urls_for_platform(self, platform: str) -> AsyncIterator[list[str]]:
    async for batch in self.stream(
        where=sql(_columns.platform == platform),
        columns=["url"],
    ):
        yield batch

The finding as it appeared on the merge request. Identifiers and code are redacted.

The recommendation applied our reuse instruction to an incomplete adoption of a new abstraction. The expression layer existed, but these call sites had not yet adopted it. Bringing them into the same design reduced the need for each caller to handle literal construction independently.

Our engineers supplied the abstraction and the convention; CodeRabbit connected them at the point where the author could act. The author assessed the finding and made the change. The review comment records path instructions as its source and marks the finding as addressed in a subsequent commit.

Respecting repository policy and the scope of a refactor

A second finding showed how easily the meaning of a review comment can depend on facts outside the line it highlights.

In app/logging/__init__.py, a merge request extracted log-level selection into a new helper, _configured_level() -> int. The existing _setup_handlers() signature appeared further down as an added line in the diff. CodeRabbit's Ruff-backed finding requested a return annotation on that existing function, citing ANN202.

The engineer pointed out that the signature predated the refactor, while the new helper already carried a return annotation. The repository also selected a specific set of Ruff rules:

[tool.ruff.lint]
select = ["E4", "E7", "E9", "F", "I"]
ignore = []

Under Ruff's rule-selection semantics, this selection does not enable the ANN family. The engineer reported that the repository's Ruff check passed on the file.

CodeRabbit inspected pyproject.toml, quoted the selection, and withdrew its suggestion. Its reply also acknowledged that the function signature was unchanged.

Challenged on the finding, the reviewer reads the repository configuration and withdraws it.

There were two distinct questions here: had the change introduced the issue, and did the reported rule apply to this project? The diff alone did not settle the first; knowledge of the rule alone did not settle the second. Examining the refactor and the repository configuration changed the conclusion. That exchange gave us a concrete example of how an author could challenge a finding with evidence.

Making project policy hold across the integration

Resolving the comment left a further question. Why had a rule outside the repository's selection reached the review at all?

The findings identified their sources. The SQL recommendation came from path instructions, while the annotation finding came from the linter integration. This distinction helped us investigate the unwanted comments at the level where they originated: the configuration supplied to the integrated tool.

We collected inline findings from the rollout, grouped them by the analysis tool cited, and extracted their rule codes. Eleven findings named a specific rule:

Findings	What the configuration check showed
8	The checked-in configuration did not enable the reported rule
3	The project had no such configuration, so the fallback behaviour was expected

The three expected findings corrected an assumption on our side. We had expected a project policy that the repository did not contain. The other eight provided concrete cases for CodeRabbit's engineering team.

They reproduced the issue and identified a precedence defect: a fallback configuration could take priority over the repository's checked-in configuration. They explained the cause, shipped a fix, and told us when it was available. The interval from our detailed report to the delivered fix was under two weeks.

The work depended on both teams. We could compare the comments with the policies and code they were meant to reflect. CodeRabbit's engineers could identify and correct the integration behaviour that produced the mismatch.

We then observed five days of real merge requests:

Measure	Result
Merge requests reviewed	33
Merge requests containing Python	23
Inline findings across CodeRabbit reviews	46
Findings attributed to the affected integration	0
Instances of excluded rules counted in reviewed files	2,392

The unwanted findings did not recur during this period, while the broader review workflow continued to report findings.

Interpreting that absence required care. Our first attempt had used two merge requests without Python. Their quiet results could not tell us anything about the affected Python analysis. Even with relevant input, the table supports a limited conclusion: we observed no recurrence. Activity elsewhere in CodeRabbit does not establish successful Ruff execution, and patterns present in reviewed files are not necessarily eligible for inline reporting.

Bringing engineering conventions into everyday review

The two examples show different uses of project knowledge. In the query change, the shared abstraction and reuse instruction supported a specific recommendation. In the logging refactor, the rule selection and the history of the function explained why a recommendation did not apply. The collaboration with CodeRabbit's engineers addressed how that rule selection reached the integrated tool.

For our team, this made explicit conventions a practical part of adopting AI review. Instructions under version control can be revised. Repository policy can be inspected. A finding can be accepted or challenged against those sources. CodeRabbit contributes recommendations and context, while authors and human reviewers remain responsible for the changes that merge.

A useful starting point for another team is to examine the comments its reviewers repeatedly make. Some may express knowledge that could be written down and supplied to the reviewer. Others may expose decisions the team has never made explicit. Observing how those instructions affect real reviews provides a way to refine both the guidance and the workflow.

Our work with CodeRabbit gave us examples of that process and a direct collaboration when the integration needed correction. We thank its engineering team for their part in making our project's conventions more consistently available to the review.

Your Job Is a Bundle of Tasks. AI Is Unbundling It.

Daniel Alker — Fri, 30 Oct 2026 07:32:20 +0000

When I joined Cobrainer, my title was Cloud Platform Engineer. In 2024 that title made sense: design the AWS architecture, maintain backend systems, keep the platform running. In 2026 it no longer describes the work. I do not think of myself as a platform engineer any more, but as part of Product & Engineering: someone who contributes to the whole product, not one layer of the stack.

There was no big announcement. The work changed one task at a time, until the title stopped describing my day.

I still own the platform. But I also translate product requirements into architecture, ship frontend and backend features, and build services where agents act under human approval. Writing the instructions those agents follow is now part of the job itself.

That opens up a lot of possibilities. It also raises an uncomfortable question: when an agent takes over work you spent years learning to do, where does your contribution begin and end?

My answer is taking shape in the work: deciding what matters, giving agents enough context, and knowing when their output is safe to use. The execution is increasingly shared. The responsibility is still mine.

That is why I think the useful starting point for AI transformation is the work inside a role: its tasks, its handoffs, and the decisions someone still has to own.

What changed in my day

This is one person's experience, not a forecast for every engineer. But putting the tasks side by side makes the shift visible.

Task	A year ago	Today
Shape product and roadmap decisions	Rarely	Regularly, with product
Translate product requirements into architecture and acceptance criteria	Occasionally	Core of the job
Design tenant-isolated cloud architectures	Me	Me, with agents exploring options and drafting decision records
Provision infrastructure as code	Me and agents	Agents draft; I verify the plan and blast radius
Build frontend and backend capabilities	Me and agents, mostly backend	Mostly agents; I review based on risk
Build agent execution services with human approval and audit trails	Did not exist	New, and growing
Validate security and resilience	Me, partly manual	Agents write and run tests; I approve the release
Operate observability and incident recovery	Me	Agents investigate; I decide
Write specs and agent instructions	Regularly	Core of the job

The responsibilities in the old description are still here, but much of my effort has moved towards defining and checking the work. Alongside that, new tasks have appeared: designing approval boundaries, making agent actions traceable, and writing instructions precise enough to act on.

“Agents write the code” sounds simple. Deciding whether that code solves the right problem, respects tenant boundaries, and can be operated at three in the morning is still engineering.

Anthropic describes a similar shift in its own workforce. Work on agent instructions, reviews and monitoring grew from 10% of the average employee's interactive Claude sessions in fall 2025 to 22% in spring 2026. Its report also describes human attention becoming a bottleneck as agents produce more output. This is evidence from one unusually AI-intensive company, not a prediction for every workplace. But the growing effort required to direct and review agents is a useful signal. 1

A job title hides too much

Two engineers can have the same title and spend their weeks doing very different things. One might maintain a stable service; another might spend most of their time untangling requirements across teams. Their exposure to automation will differ too.

This distinction predates today's agents. In 2016, Arntz, Gregory and Zierahn estimated that about 9% of jobs across 21 OECD countries were automatable under their task-based approach, substantially below earlier occupation-based estimates. They accounted for differences in what people actually do within the same occupation. That number is not a forecast for today's AI. The lasting lesson is that the unit of analysis matters. 2

For engineers, this should feel familiar. You would not estimate a system's performance from its name in the service catalogue. You would look at what it does and where it spends its time.

We need that same curiosity about our work.

Follow the handoffs

Even a task can hide a whole workflow. “Provision infrastructure” might include clarifying requirements, drafting a change, checking permissions, reviewing the plan, applying it, and verifying the result.

Demirer and colleagues model work as chains of interdependent steps. In their model, the cost of handing work between humans and AI can make it worthwhile to automate a whole sequence, even when a human would perform one isolated step better. Where a step sits in the chain matters. 3

Consider an agent that drafts an infrastructure change and runs its checks. If I have to reconstruct its reasoning before I can approve the plan, generating the code faster may save less time than expected. If it gives me a clear diff, relevant test results and a credible rollback path, the review becomes easier.

The approval still serves a purpose. The engineering challenge is to give the person approving the change enough evidence to make a good decision.

“Six out of ten steps are automatable” tells me very little until I know how those steps connect.

Ask what people want to keep

There is a human question here that a capability benchmark cannot answer.

Shao and colleagues gathered preferences from 1,500 workers across 104 occupations and compared them with AI experts' assessments of technical capability. The correlation between workers' desire for automation and assessed capability was weak: Spearman ρ = 0.17. Feasible automation and wanted automation often pointed to different tasks. 4

That matters because work is also a source of pride, identity and learning. A task someone wants to hand off may be the task a colleague enjoys most. And work an experienced engineer can safely delegate may be how a junior engineer learns to recognise a bad solution.

I think we owe people a better conversation than “this percentage of your job can be automated.” Start with what frustrates them, what they value, and what they need to learn. Then work through what should change together.

Start with one role and a real week of work

For engineering and HR leaders, I would start small:

Map the work with the people doing it. Take one role, list its tasks, and break a few into steps. Include the interruptions and judgement calls that never made it into the job description.
Trace one workflow end to end. Mark where AI can assist, where it can complete a step, and where a person must decide. Look closely at the handoffs and the evidence needed for review.
Separate capability from preference. Ask both what AI can do reliably today and what people want help with. Record uncertainty and revisit the assessment as tools and workflows change.
Make room for the new work. Instructions, review, monitoring and recovery take time. Put them into role descriptions and development plans, including how less experienced colleagues will learn.
Agree how the map will be used. Use it to redesign work with the team. It should not become a ranking of whose job is easiest to remove. Involve employee representatives early where relevant.

The first map will be incomplete. It only needs to be concrete enough for someone to say, “That is not how this task actually works,” and help you improve it.

Why we are working on this at Cobrainer

At Cobrainer, we are building on this idea across our Work Architecture, Talent Intelligence and AI Transformation capabilities: making the tasks inside roles visible, so organisations and their people can understand and shape how the work changes.

We will share more in our Fall Release, followed by posts about the engineering behind it.

For me, this is already personal. The title I joined with describes a 2024 job. In 2026 the line between product and engineering has blurred: I contribute across the whole product, while the decisions I make still carry consequences for the platform and the people using it.

A useful task map gives us somewhere to start that conversation. It lets us talk about what is changing while there is still time to shape it.

Which part of your work has changed most this year, and how do you feel about that change?

References

Wilmers, N., Catipon, R., Massenkoff, M., & McCrory, P. (2026). How employment is changing at Anthropic. Anthropic.
Arntz, M., Gregory, T., & Zierahn, U. (2016). The Risk of Automation for Jobs in OECD Countries: A Comparative Analysis. OECD Social, Employment and Migration Working Papers No. 189.
Demirer, M., Horton, J. J., Immorlica, N., Lucier, B., & Shahidi, P. (2026). Chaining Tasks, Redefining Work: A Theory of AI Automation. NBER Working Paper 34859.
Shao, Y., Zope, H., Jiang, Y., Pei, J., Nguyen, D., Brynjolfsson, E., & Yang, D. (2026). Future of Work with AI Agents: Auditing Automation and Augmentation Potential across the U.S. Workforce. arXiv:2506.06576v3.

The JVM Speaks AI: Cobrainer's Journey with Kotlin, Koog and Exposed

Daniel Alker — Tue, 13 Oct 2026 07:51:48 +0000

One afternoon, one of our LLM providers started throttling us. A large customer had triggered a burst of AI-assisted work, and the provider's capacity in our region could not keep up. Our prompt executor followed its fallback chain, the circuit breaker took the affected models out of rotation, and each request moved to a second provider without interrupting the workflow. There was no incident to handle and no change in system behaviour for the customer. We noticed it only the next morning on our LLM observability dashboard.

We built all of our AI integration in Kotlin. A controversial decision? We do not think so. In 2025, one could still argue that AI integration required Python. By 2026, that debate is no longer a useful basis for architecture decisions. Python remains important for research and model training, but production AI is mostly about integrating APIs, enforcing contracts, and operating reliable systems. Every prompt, agent, and structured-output contract in our core platform lives on the JVM: one Kotlin monorepo with types shared end to end, built on Koog, Exposed, coroutines, and kotlinx.serialization.

TL;DR

Kotlin lets six developers and their coding agents work in one stack. Sealed types, kotlinx.serialization, null safety, and coroutines make contracts and control flow explicit.
Koog was the best fit because its lightweight abstractions build on Kotlin instead of hiding it. We moved the proof of concept into production for all customers in one day.
Production reliability comes from cross-provider fallback chains, circuit breakers, LLMObs, and contract tests. Removing blocking code from coroutine paths also increased throughput by 5×.

We at Cobrainer

Cobrainer is a Munich-based B2B HR-tech company that grew out of a Technical University of Munich research project in 2013. We serve enterprises across industries and regions. Our AI-native Skills Platform creates and governs living Job and Skill Architectures: structured models of an organization's functional roles, levels, tasks, and skills. They answer three basic questions: Which distinct jobs exist in the organization? What is needed to perform them? How do they relate to each other? A patented Skills Graph and ML/NLP power the platform.

Our customers use it for skills-based (AI) workforce transformation: internal mobility, recruiting, learning, workforce planning, and EU pay-transparency readiness, with integrations into SAP SuccessFactors, Workday, ServiceNow and other HCM systems.

Our Summer 2026 release includes a context-aware Personal Agent for the Job Architecture and the wider platform, agentic background tasks, agentic (g)RAG, and collaborative multilingual commenting for governance. The AI can resolve structural and language-specific comments on job profiles in more than 25 languages.

Our agentic engineering setup centres on six developers with end-to-end ownership, from cloud infrastructure through backend and frontend to AI. Coding agents help us explore the repository, implement changes, run checks, and investigate operational issues. Together, we serve more than 10,000 active users at companies around the world.

Small, fast-moving team, big surface. That constraint explains almost every technology decision in this article.

Kotlin for AI-powered features

Why Kotlin? Reliability meets velocity

The control plane and core domain of our platform use Kotlin and Spring Boot. The stack is type-safe and null-safe end to end, with kotlinx.serialization for wire contracts, Exposed for SQL, and coroutines for non-blocking I/O. A central monolith is the entry point for the platform, and everything lives in one monorepo. With six engineers working across the full system, every additional deployable adds work to development, operations, and CI.

A large monolith rewards explicitness. Its behaviour can span distant files, and coding agents cannot safely fill gaps left by implicit conventions. Kotlin's types and compiler checks make those relationships visible. Koog follows the same principle with small abstractions that keep contracts, capabilities, and control flow in the code. This helps developers review changes and gives coding agents firmer boundaries in which to work.

When our LLM work moved from isolated features to core platform components, we asked whether it justified a separate stack. The models sit behind HTTP APIs, so another runtime would not improve their output. It would give us a second deployment pipeline, add serialization boundaries between the AI layer and domain model, and remove compile-time guarantees at the boundary where LLMs introduce the most uncertainty.

We wrote down what we actually needed from an LLM framework:

Lean and explicit. The framework should behave like a library while keeping control flow visible in the code.
A release cadence that matches innovation speed and model churn. New models and paradigms arrive monthly, so infrequent framework releases become a bottleneck.
Clean provider and model-capability abstractions. We need multiple providers, and their models support different features.
Type safety end to end. An LLM's output should enter the domain as a typed value or not at all.

We tested these requirements by building working implementations instead of comparing feature matrices.

The turning point: why Koog fits Kotlin

We built and ran production implementations with Spring AI, LangChain4j, and Koog. The important difference was not the length of their feature lists. It was how clearly each framework expressed its behaviour in Kotlin for developers, coding agents, and production tooling.

Spring AI is the most comprehensive and deeply integrated Spring option. It offers a robust ecosystem and good portability for teams already using Spring Boot. We still use it for our MCP endpoints, where its integration with REST infrastructure and the Spring Security filter chain is exactly what we need. In the agentic core, however, it added another layer between our code and the model when we wanted direct control and rapid iteration.

LangChain4j is a large, feature-rich library that brings LangChain concepts to the JVM. It helps teams move quickly, but we found failures inside a chain difficult to trace back to the underlying LLM call. We eventually replaced the last utilities that depended on it with a few dozen lines of Kotlin.

Koog changed the direction of the evaluation because it made Kotlin itself the foundation of the integration. Our @Serializable domain types become structured-output contracts directly. Sealed hierarchies model the possible results, and exhaustive when expressions force every caller to handle them. Coroutines carry naturally through provider calls, retries, and agent workflows. Koog's DSL builds on familiar Kotlin features such as extension functions and lambdas with receivers, so agent graphs remain normal Kotlin code that can be traced, changed, and verified with the compiler.

The provider abstraction was equally important. Koog represents model capabilities explicitly instead of reducing every model to a lowest common denominator. We can adapt temperature, token limits, and structured-output handling for each model while keeping the same domain contract and call site. These decisions are visible in types and code instead of being hidden in configuration or prompts. When a developer or agent investigates a failure, the underlying request remains accessible rather than disappearing behind a chain of abstractions.

That combination gave us the level of abstraction we wanted: enough structure to avoid rebuilding provider clients and agent orchestration, but close enough to Kotlin that our existing tools, types, and debugging habits still applied. A coding agent can trace a change from a domain type through its LLM contract to every consumer without crossing languages or repositories. Koog did not introduce a second programming model into the codebase. It made the existing one more useful to both humans and agents.

Once we chose Koog, we cleaned up the proof of concept and rolled it out to all customers in one day. That speed came from keeping the domain model, serialization, concurrency model, tests, and deployment pipeline we already understood.

AI in production: structured output as the workhorse

In production, much of an LLM's value comes from filling data structures that domain logic can trust, rather than generating free-form chat. Koog has powered our Job Architecture generation since summer 2025, and structured output is the foundation of that core product capability.

The contract is a Kotlin sealed hierarchy with kotlinx.serialization. Koog turns it into a JSON schema (docstrings included, via @LLMDescription), gets the provider to comply, and hands back a typed value:

@Serializable
enum class ChangeKind { RENAME, ARCHIVE, MOVE }

@LLMDescription("A single proposed change to the job architecture")
@Serializable
sealed interface ProposedChange {
    val kind: ChangeKind

    @Serializable
    @SerialName("rename")
    data class Rename(
        @property:LLMDescription("The item to rename")
        val itemId: String,
        @property:LLMDescription("New title, in the tenant's base language")
        val newTitle: String,
    ) : ProposedChange {
        override val kind get() = ChangeKind.RENAME
    }

    @Serializable
    @SerialName("archive")
    data class Archive(
        @property:LLMDescription("Items to archive, including their descendants")
        val itemIds: List<String>,
    ) : ProposedChange {
        override val kind get() = ChangeKind.ARCHIVE
    }
    // ...
}

Requesting it is one call. If a weaker model returns almost-JSON, a StructureFixingParser repairs it with a lightweight model before we ever see it:

val plan = executor.executeStructured<ChangePlan>(
    prompt = prompt("change-planner") {
        system(plannerInstructions)
        user(request)
    },
    model = model,
    fixingParser = StructureFixingParser(model = lightweightModel, retries = 2),
).getOrThrow().data

plan.changes.forEach { change ->
    when (change) { // The compiler enforces exhaustive coverage.
        is ProposedChange.Rename -> preview(change.itemId, change.newTitle)
        is ProposedChange.Archive -> confirmArchive(change.itemIds)
        is ProposedChange.Move -> validateMove(change)
    }
}

Kotlin provides specific safeguards here. Null safety exposes schema drift at compile time: add a required field to the contract and every construction site that omits it stops compiling. A when expression over a sealed result is exhaustive, so a new change kind breaks the build until every consumer handles it. Default arguments and small DSLs keep configuration readable, allowing us to review prompt code like domain code.

We also balance traffic across providers whose models differ in capabilities, and Koog represents those differences explicitly. A provider that uses an existing API format requires little integration code:

// xAI uses the OpenAI wire format.
@Serializable
object XAI : LLMProvider("xai", "xAI")

val xaiClient = OpenAILLMClient(
    apiKey = settings.apiKey,
    settings = OpenAIClientSettings(baseUrl = "https://api.x.ai"),
)

Our executor adds three behaviours to Koog's PromptExecutor: retry with backoff, a config-driven fallback chain, and a per-model circuit breaker. Before each attempt, it adapts the prompt to the target model's declared capabilities:

fun adaptPromptFor(model: LLModel, prompt: Prompt): Prompt = prompt
    .letIf(!model.supports(LLMCapability.Temperature)) { it.withoutTemperature() }
    .letIf(prompt.maxTokens > model.maxOutputTokens) { it.cappedTo(model.maxOutputTokens) }
    // Model can't do native structured output? Ask for it in the prompt
    // instead. The sealed contract and fixing parser stay identical.
    .letIf(!model.supports(prompt.schemaCapability)) { it.withSchemaAsUserMessage() }

The call site never knows which provider answered. That indirection is what made the silent rebalancing from the opening of this article a non-event.

Growing into an agentic platform

At Cobrainer, AI powers skill inference, Job Architecture generation, transcription, chat agents, and agentic (g)RAG. Our Job Architecture chat agent brings several of these capabilities together.

The agent lets users create, review, and maintain a job architecture conversationally: "merge these two levels", "draft the missing task descriptions for this job family", or "archive everything nobody has touched since the reorg". At a high level, it is a supervisor tool loop with deterministic sub-workflows. The LLM decides what to do, while a Koog strategy graph routes every consequential action through code we control. A human must approve every change.

The agent is our one exception to the monolith. It runs in the Agent Runtime, a reactive Spring Boot Kotlin service designed for long-running conversations, streaming, and structured concurrency. The monolith remains the single entry point for the platform, including REST and MCP. Both services live in the same monorepo, so they share types directly and the compiler checks their contract. SurrealDB stores checkpoints for resumable turns, message history, and three layers of memory: a small window of recent conversation chunks, a running session briefing, and a fact-based long-term graph. Platform data remains in Postgres behind the monolith, accessed through Exposed.

Koog's graph DSL makes the tool loop almost declarative. We use marker tools that the model can call but that never execute directly. A guarded edge intercepts the call and routes it into a deterministic sub-workflow:

val toolLoop = strategy<String, String>("supervisor") {
    val callLlm by nodeLLMRequest()
    val executeTool by nodeExecuteTool()
    val sendResult by nodeLLMSendToolResult()

    val editFlow = installEditCapability(approvalGate)
    val bulkFlow = installBulkCapability(approvalGate)

    // Marker tools are visible to the model but never executed.
    // The edge routes into a deterministic sub-workflow instead.
    edge((callLlm forwardTo editFlow).soleMarker(EDIT_TOOL_ID))
    edge((callLlm forwardTo bulkFlow).soleMarker(BULK_TOOL_ID))

    // Edges match in declaration order, so the generic loop comes last.
    edge(callLlm forwardTo executeTool onToolCalls { true })
    edge(callLlm forwardTo nodeFinish onAssistantMessage { true })
    edge(executeTool forwardTo sendResult)
}

The same graph drives the progress UI. Instead of streaming tokens, we show meaningful progress. Koog's event hooks fire as the agent moves through nodes and tools, and each transition maps to a localized status such as "Analyzing your architecture…" or "Preparing changes for review…". Showing graph progress improved perceived latency more than token streaming would have for this workflow:

install(EventHandler) {
    onNodeExecutionStarting { ctx -> emitStatus(ctx.node.name) }
    onToolCallStarting { ctx -> emitToolStatus(ctx.toolName) }
    onAgentExecutionFailed { ctx -> events.close(ctx.error) }
}

Security is where the two frameworks meet. The agent accesses tools only through central MCP servers, which Spring AI serves inside the monolith. Every tool call therefore passes through the same Spring Security filter chain as our REST APIs. We run one MCP server per permission scope (read, architect, reviewer), guard each at the path level, and give the agent a short-lived delegated token so it always acts as the user. Permissions filter the available tools before any LLM call. If a user cannot edit the architecture, their agent never sees editing tools. This reuses our existing authorization infrastructure without relying on prompts for access control.

Koog runs the agent in the Agent Runtime, Spring AI exposes and protects its tools in the monolith, Exposed and Postgres store platform data, and SurrealDB stores agent memory. The result is two JVM services in one monorepo, sharing one type system.

Discoveries and challenges

Production use also exposed several important problems and lessons.

A 5× throughput gain from coroutine cleanup. Our most difficult performance problem had nothing to do with Koog or LLMs. Legacy blocking code inside coroutines starved dispatchers under AI-driven load, causing vague and load-dependent slowdowns. Once we found and removed it, throughput on the affected paths increased by 5×.

Which leads to the sharpest lesson we can offer:

Do not leave runBlocking in production request paths. This applies even to a blocking Spring Boot servlet stack where it appears to work. Under load, it can deadlock a dispatcher without crashing the application; requests simply stop completing.

Build deep fallback chains across providers. Several fallback models from the same provider still share one failure domain. Our chains cross provider boundaries, and per-model circuit breakers skip throttled or unhealthy routes so retries reach a viable model quickly. We trace every attempt and fallback in LLMObs, including latency, token usage, and errors. That makes it possible to reconstruct a request across providers and distinguish a model failure from a parser or application problem. This setup paid off as soon as one provider lost capacity.

Contract-test the LLM layer. Structured output depends on our schema, the framework's schema generation, and each provider's implementation. We run end-to-end structured-output tests with real calls and parsing for every provider, as well as LLM contract tests in CI. Together with LLMObs, these checks form our AI operations loop: observe production behaviour, investigate failures with full context, and validate model or provider changes before rollout. This lets developers and agents move quickly without turning model churn into guesswork.

Final words

Building AI features in Kotlin once sounded unconventional. Today, its explicit contracts and lightweight abstractions give our six developers and their coding agents a system that is fast to change, safe to refactor, and straightforward to operate.

If you are considering AI development on the JVM, the ecosystem is ready. Have a look at Koog, Exposed, and Kotlin. We thank the JetBrains teams behind all three for their frequent releases and responsiveness.

Questions, disagreements, and production stories are welcome in the comments. We read them.