DEV Community: Viktoria

One symptom, ten problems: a Kotlin + Spring Boot + Kafka debugging case

Viktoria — Tue, 08 Sep 2026 13:21:38 +0000

A single visible symptom led to many defects, half unrelated to it but all tied to one invariant about async TID processing. Debugging is invariant discovery.

One visible symptom opened the session. By the end of it the developer and the agent had fixed many defects, about half of them unrelated to that symptom. All of them, though, sat inside the same scope, and the developer summed that scope up in one sentence:

Any operation on a TID must go through the single asynchronous path and must not let TMS and EMV drift apart.

I am the product manager at Explyt, where we build an AI agent for JetBrains IDEs. This session ran in our own agent, in regular chat mode, with a current frontier model, on a Kotlin and Java codebase with Spring Boot and Kafka in a payments system. I did not run it myself; I went through the log afterwards as part of our review of how people debug with an agent in practice. Debug mode was not used. What follows is a review of the shape of the work, with the session's own details left out. The defects are described as kinds, and the number in the title is the topic's working name; the log gives no count I could verify.

Previously on the Explyt blog, we compared two agent runs on one JVM bug, one working from the log and one from a breakpoint: Superpowers vs Plain Old Debugger in Explyt. That case was one bug with one cause. This one has the opposite shape, one symptom with many causes, and it stresses a different part of the debugging loop.

TL;DR

One symptom, many defects, half of them unrelated to the symptom, all of them violating one invariant about asynchronous TID processing.
Our agent was tactically helpful: it proposed SQL, warned about risks, explained code locally, and did not scatter new debug logging. It was strategically passive: the developer drove every step.
It did not separate verified facts from hypotheses, and it proposed fixes before the problem was localized. Those fixes compiled and passed lint. Afterwards something else broke, differently.
The developer was the agent's only sensor. Kafka and Spring logs, SQL results, HTTP responses, Kubernetes and Compose messages, config files: all pasted into the chat by hand. In the terminal the agent ran only git.
Debugging a distributed system is a search for the invariant that all the defects violate. The agent needs instruments for that, and I say below which of them Explyt has today and which it does not.

What the session looked like

I cannot show you the symptom or the defects; the log belongs to the developer's project. What I can describe is the shape.

The developer came in with one observable problem. Working through it with the agent, they found a defect, fixed it, and something else broke, in a different way. Then another defect, another fix, another different breakage. About half of what they fixed had nothing to do with the original symptom. Every fix, however, touched the same scope: how the system handles a terminal ID (TID) across its asynchronous path and keeps two downstream systems, TMS and EMV, consistent. The sentence in bold above is how the developer described that scope.

A few times the developer asked the agent to add something and later asked it to roll that back. That is the texture of the session: iterative, exploratory, with the agent along for the ride.

What the agent did well

The agent proposed SQL queries to inspect the current state in TMS and EMV, so the discussion started from actual rows. It warned about risks in code it was asked to change. It explained what a given piece of code did, when asked, and got it right. It did not add new debug logging; it worked from the log statements that already existed, which kept the diff clean.

All of that is real help; a developer working alone would have done that typing themselves.

Where it fell apart

Strategic passivity

The first failure was strategic passivity. The agent did not run the investigation. It waited for the developer to bring the next piece of evidence, reacted to it, and waited again. At no point did it say: here is my current theory, here is what would falsify it, let me check. The developer decided what to look at next, every time.

Facts and guesses in one voice

The second: verified facts and guesses came out in the same confident voice. The developer had to keep track of which statements had been checked and which had not. In a session with many defects that bookkeeping is most of the job, and the agent was not doing it.

Fixing before localizing

Third, the agent proposed fixes before the problem was localized. The developer applied them; the log shows no rollback of those fixes, at least not in the chat. The agent checked that each change compiled and passed lint. It did not check that the fix removed the symptom. Several fixes turned out to be ineffective: something else broke afterwards, differently. Reading that part was uncomfortable. I have accepted a plausible fix at the end of a long day for the same reason: it compiled.

No model of the entities

The fourth failure is the one that matters for tooling. The agent never built a model of how the entities relate: what TID is, what TMS and EMV each own, what "consistent" means between them. Without that model, every defect looked like a separate bug; with it, they are one invariant violated in several places.

The developer as the only sensor

The developer pasted into the chat, by hand, fragments of SQL scripts and their results, Spring Boot and Kafka log excerpts, HTTP responses, messages from Kubernetes and Docker Compose, and configuration files. The agent's own use of the terminal was limited to git.

So the agent's picture of the running system was whatever the developer chose to show it, in the order they chose, with their framing already applied. That is a poor position from which to form an independent theory, and it explains the passivity. This is the failure I keep coming back to, and I do not have a tidier answer than "give it instruments".

The review proposed two fixes. Give the agent a browser and a terminal, and make it use them. And give it a debugging method with a written journal: hypotheses, experiments, results, and a working theory that includes the data model and its invariants, along the lines of the Superpowers systematic-debugging skill and the parallel-debugging variant. Both fixes assume the agent can run an experiment and read its result without a human in between. A method without instruments is a checklist.

Debugging is invariant discovery

The framing I took away from this log: debugging a distributed system is a search for the sentence that all the defects violate, and the individual fixes are almost a byproduct. A debugging agent therefore needs four things.

A working theory in writing, separate from the chat scroll, so that "checked" and "assumed" stay apart over a long session.
The ability to run the experiment that would confirm or refute the current hypothesis, and to read the result itself.
Direct observation of state, a variable at a breakpoint or a row from a query it ran, because pasted excerpts are already interpretations.
After the second unrelated defect, the question of what the two have in common.

The session failed on all four. The model was a current frontier model. The harness gave it chat, file access and git, and asked it to debug an asynchronous payments pipeline. That is where the four failures come from, and it is our harness.

What Explyt does about it today, and what it does not

Explyt covers part of this loop today, and I want to be precise about which part.

Explyt's documented Debug mode exists for the "observe state directly" requirement: for a bug you can reproduce by running a test, an application or an existing IDE run configuration, the agent is asked to confirm the cause under the debugger first, with breakpoints, variable values and the call stack, before it changes any code. Then a minimal fix, then a rerun of the original scenario and the related tests. That is the experiment loop this session never had.

Run configurations close the other gap: the agent runs the build or the test through the IDE and gets back console output, test results and compilation errors as a structured tool result, so it has something to read beyond "compiles and lints". The docs are explicit that a single debugger run confirms the fix only for that scenario and does not replace the related tests.

Nothing in the Explyt documentation reads Kafka topics, runs SQL against your database or pulls logs from Kubernetes on its own. Those sources still enter the session the way they did in this log, through the developer, or through an MCP server you connect yourself; the tools documentation describes MCP as the way to attach browsers, knowledge bases and other external services, and what a given server exposes is up to that server. Explyt's Memory Bank keeps facts and project agreements across chats, which is useful, and it is a different thing from a per-session hypothesis journal. The working theory is still the developer's to keep.

So if this symptom can be reproduced by a test or a run configuration, Debug mode would have changed the middle of the loop: the agent could have seen the actual state at the moment of the write, and rerun the scenario after each fix, with lint no longer the last check. It would not have changed the edges. Someone still has to bring the Kafka and infrastructure evidence, and someone still has to notice that the latest defect rhymes with the first one.

If you want to try this on your own bug

This part is a suggestion for your next session, and nothing here describes what happened in the one above.

Whatever agent you use, put the invariant question into its instructions so it stops being your job to remember it. A repository-level rule along these lines works with most agents that read AGENTS.md or a vendor equivalent:

## Debugging

Keep a written working theory in a file under .tasks/, updated after
every experiment: what is verified (with the command or breakpoint that
verified it), what is assumed, and the current hypothesis.

Before proposing a fix, localize: reproduce the failure under a test or
run configuration and confirm the cause at a breakpoint or in a query
result you ran yourself.

After a fix, rerun the original scenario. "Compiles and lints" is not a result.

After the second defect in the same area, stop and write down what the
two have in common before fixing either.

Then reproduce one failing test in the debugger and make the agent read the variables before it edits anything. If the agent has no debugger, the rule still helps; it just makes the missing instrument visible.

Limits

The evidence here is one session log from one developer using our agent, reviewed by me after the fact. I did not run it and I am not publishing its details, defect list or code. Whether the symptom was reproducible under a local test, which is what Debug mode requires, I do not know from the log. Nothing here is a benchmark.

Try the loop in a JetBrains IDE

If you already work in a JetBrains IDE, install Explyt from the Marketplace and reproduce one failing test in Debug mode. Ask the agent to confirm the cause at a breakpoint before it edits anything, then make it rerun the scenario after the fix.

When your agent fixed a bug and something else broke, did you go back and ask what the two had in common? I am collecting the invariants people found that way; the comments are open.

Sources

Superpowers vs Plain Old Debugger in Explyt

Viktoria — Thu, 03 Sep 2026 11:04:36 +0000

An AI agent can follow a debugging checklist and still miss the bug.

The question I kept coming back to was simple: who ran the experiment?

I interviewed our users, mostly senior developers. In those interviews, debugging a hard problem came up most often. In practice, the investigation is still manual: a developer supplies logs, sets breakpoints, tests a theory, and pastes the result back into the chat.

A systematic debugging method gives the work a clear order: gather facts, form a hypothesis, run an experiment, then change the code.

Superpowers was created by Jesse Vincent and the Prime Radiant team. Matt Pocock helped popularize it; the collection includes a systematic-debugging skill. I ran Opus 5 on the same real bug and the same prompt twice. The only thing I changed was the debugging workflow: Superpowers in one run, Explyt's built-in Debug skill in the other.

I ran one controlled case; it is not a benchmark of either product. One number makes the contrast plain: Explyt used 67k tokens to solve this case, while Superpowers used 132k. The transcripts explain the gap: the debugger supplied a runtime fact early, while the other run spent more context on speculative code, tests, and refactoring.

The bug: one protected Windows directory, one crashed coroutine

The test case came from a Kotlin project on Windows. The IDE log contained:

java.nio.file.AccessDeniedException: C:\Config.Msi
    at java.nio.file.Files.list(Files.java:3422)
    at FacadeMentionCandidates.listDir(FacadeMentionCandidates.kt:83)
    at FacadeMentionCandidates.listChildren(FacadeMentionCandidates.kt:69)
    at MentionSuggestionMapperKt$suggestionsFor$2.invokeSuspend(MentionSuggestionMapper.kt:36)

The relevant code first checked whether a path was a directory and then enumerated it with Files.list. The exception came from a protected Windows Installer directory. FacadeMentionCandidates makes filesystem roots available to the @ mention flow, so a user navigating through a path such as @/C:/ can eventually reach a directory that the IDE process cannot enumerate.

The error was not handled at the flow boundary. It escaped the suggestion flow and was reported as an unhandled coroutine exception in WorkspaceBootstrap, with the IDE blaming the plugin. The same listing path is also used while searching around the focused file, so this was a small function with more than one way to fail.

Run one: a plausible patch from Superpowers

The Superpowers run started well. It searched the log instead of reading the whole file, followed the stack into FacadeMentionCandidates, found the unguarded Files.list, and stated a plausible root-cause hypothesis.

Then the investigation stopped before the experiment. The run never set a debugger breakpoint or inspected a value from the failing execution. The proposed fix was to wrap the directory listing in a try/catch, log the exception, and return emptyList().

That choice made the failure disappear from the call stack, but it changed the meaning of the result. An empty directory and a directory denied by ACL became the same value; other I/O failures still had no user-facing outcome. The mention UI could not tell the developer what happened, and the option to mention the directory remained indistinguishable from a normal empty-folder case.

The regression test followed the same assumption. It asserted that an unreadable directory produced no children. On Windows, the first version could not create the required POSIX permission state and was skipped. To force the exception, the agent then added a readDirectory function to the production constructor so the test could inject a throwing fake. That seam existed for the test, not for the product behavior.

The run finished with seven passing tests, but those tests established that the workaround returned an empty list. They did not establish that the user-facing behavior was correct, or that the error had been verified against a real paused execution. I discarded the patch: it hid the distinction we needed and left a test-only abstraction in the production API.

That is how a checklist turns into a ritual. Every step appears in the transcript; the experiment never does. A plausible local edit stands in for evidence.

Run two: Explyt puts the hypothesis inside the IDE

With Explyt's built-in debugging skill, the model started from the same log and source files, but it could use the JetBrains debugger as part of the investigation.

It created a minimal reproducer, set a line breakpoint on the Files.list call, and ran the test under the debugger. When execution stopped, the value was concrete:

Debugger evaluation	Observed value
`dir`	`C:\Config.Msi`
`Files.isDirectory(dir)`	`true`
`Files.isReadable(dir)`	`false`
`Files.list(dir)`	throws `java.nio.file.AccessDeniedException`

The call stack matched the production trace: listDir → listChildren → MentionSuggestionMapper.suggestionsFor, inside the coroutine that builds mention suggestions. The debugger turned "a protected path probably causes this" into a verified statement about the running program:

the directory check passes, the process cannot read the directory, and the unguarded enumeration kills the suggestion flow.

The evidence kept the fix at the filesystem boundary, with focused regression coverage and the normal listing path left intact. The mention protocol and runtime model stayed unchanged, and debug prints were unnecessary.

The narrow fix passed review and is now in production. A try/catch was enough here. What mattered was choosing it after observing the failing execution, instead of using it as the first explanation for the failure.

The practical difference

	Superpowers run	Explyt run
Starting point	Same log and source	Same log and source
Investigation channel	Text, stack trace, and tests	JetBrains debugger plus source and tests
Key fact	Inferred from `AccessDeniedException`	Observed at a breakpoint: directory, permissions, and throw
Token use in this run	132k	67k
First implementation	Swallow the error as `emptyList()`	Keep the fix at the proven failing boundary
Test signal	Passing tests for the fallback	Runtime evidence followed by a focused regression test
Result	Patch discarded as technical debt	Reviewed minimal fix deployed to production

I could see the cleanup cost in the diff. The discarded patch carried a workaround-driven test seam. The shipped patch stayed at the failing boundary and left the normal listing path intact.

For a runtime bug, a coherent explanation from a log is still only a theory. I want one fact captured from the running program: a variable value, a call stack, an execution path, or the exact branch that contradicts the model's theory.

Explyt is an AI agent for JetBrains IDEs. In this run, it used a supported JetBrains IDE and run configuration. Its Debug workflow started the code under the JetBrains debugger and inspected breakpoints, variable values, call stacks, and execution paths before editing. The IDE supplies the evidence; the developer still decides whether the hypothesis and the resulting diff are acceptable.

What I can say from these transcripts is limited. Superpowers can be combined with a debugger, and Explyt's debugger depth depends on the IDE, language, and run configuration. Check the feature matrix for the supported combinations.

The token count points in the same direction. Explyt solved this case in 67k tokens; Superpowers consumed 132k. A debugger did not merely add another tool to the menu. It gave the model the fact it needed before more speculative turns accumulated.

JetBrains has been running paired A/B tests on "token-saving" skills, with a useful warning against reading too much into their README claims. In the Caveman test, the advertised 65% reduction became 8.5% fewer output tokens on real agentic tasks, with activation forced. In the newer rtk test, the with-rtk arm cost a median 7.6% more per task at low reasoning effort, and showed no saving at high effort. The latest Ponytail test found a 10.3% cost reduction on its own benchmark. The results are mixed; the consistent lesson is to measure the whole agent run, not a skill's self-reported counter.

Try the first useful task

If you already work in a JetBrains IDE, install Explyt from the Marketplace, open a reproducible failing test, and let the agent inspect the first runtime value that contradicts its theory. Then review the diff before accepting the fix.

The first essay in this series argued that CMMI's core insight survives and its quiet assumption does not: process maturity still predicts delivery reliability, but the model was sized for code written by people at a human pace.

Viktoria — Wed, 02 Sep 2026 18:11:31 +0000

Viktoria

Sep 2

ADLC: The Lifecycle Taking Shape

#ai #programming #webdev #llm

11 min read

ADLC: The Lifecycle Taking Shape

Viktoria — Wed, 02 Sep 2026 18:10:05 +0000

The first essay in this series argued that CMMI's core insight survives and its quiet assumption does not: process maturity still predicts delivery reliability, but the model was sized for code written by people at a human pace. This essay is about what takes the place of that assumption. A lifecycle built for agents is starting to take shape, in published methodologies and in the day-to-day practice of teams that have moved past the pilot stage. Its outline is consistent enough to draw. Agents execute the work. Humans govern intent, validation and outcomes. And two artifacts that used to rot between releases, the specification and the evaluation set, become the durable core of the project.

I will draw the lifecycle in the abstract and then, in every section, point at where each box lives in one concrete implementation: Explyt, an agent that runs inside JetBrains IDEs and uses the IDE as its source of facts about the project. I work at Explyt, so treat those passages as an author's worked example. The argument does not depend on the product; the product makes the argument easier to check.

The problem with drawing the old diagram faster

The first instinct of most teams is to keep the SDLC diagram and put an agent inside the box labelled "implement". Nothing else moves. Requirements are still written as prose for a developer who will ask a follow-up question in stand-up. Review is still sized for one or two diffs a day per reviewer. Tests are still written after the code, by the person who wrote the code. Deployment still waits for the end of a two-week cycle.

Then the agent produces a working change in twenty minutes, and every box around it starts to queue. The requirement was too loose to check against, so the reviewer reconstructs the intent from the diff. The tests the agent wrote pass, but nobody has decided whether they test the right thing. The two-week cycle now holds forty changes instead of eight, and the review step is where the sprint actually happens. This is the throughput gap from the first essay, now visible as a lifecycle problem rather than a staffing one. The shape of the process, not the speed of the coder, is the constraint.

You can watch this happen inside a single IDE session. Give an agent a one-line task in a Spring service and let it run in the default mode: it edits the service, touches a test, reports done. The diff is small and plausible. What you do not have is the question it should have asked about which callers were in scope, a record of which run configuration it used to claim the tests pass, or any reviewer output that was produced independently of the author. Every missing piece is a box on the old diagram that the agent skipped because nobody redrew it. The rest of this essay is about redrawing them.

What the emerging lifecycle looks like

Drawn without vendor labels, the lifecycle that keeps appearing has five stages and two lanes. Humans own the top lane. Agents own the bottom lane. The arrows between lanes are where the work actually gets accepted.

![ADLC lifecycle: two lanes and five stages. Humans govern Intent, Specification, Validation and Outcome; agents execute Plan, Generate and Evidence. Specification hands off to Plan, Evidence hands back to Validation, and the failure loop returns to the specification rather than the code.]

The lifecycle without vendor labels. Durable artifacts along the bottom: intent record, spec, eval set, change record, runtime evidence.

Three things in this picture are new relative to a classic SDLC diagram. The specification is written before generation and is the thing agents plan against, so it cannot be a loose paragraph. Validation is a distinct stage with its own artifacts, the evaluation set and the change record, rather than a gate inside "review". And the failure loop goes back to the specification, not to the code: if a regeneration fails validation, the fix is usually a more precise spec or a stricter eval, because regenerating code is now the cheap step.

Here is the same diagram with Explyt's names on the boxes, so each one points at something you can open. Intent and boundaries: the task you type plus Edit Scope, the list of files and directories the agent is allowed to modify for this task. Specification: the output of Plan mode, a plan with related files that you read and edit before implementation starts. Plan and Generate: the built-in agent working through the IDE's project model, symbol resolution, Find Usages, native Rename, Move and Delete, run configurations, inspections. Evidence: named run configurations executed by name, Debug mode where a failure is reproducible, and Agent Changes, the full diff of what was touched. Validation: Auto Review, a separate read-only agent that receives the original task and the final diff, and your decision on its findings. Outcome: the application running under the configuration you named. The lanes are the same; the boxes just stopped being abstract.

Where AWS AI-DLC fits

The most explicit published version of this pattern so far is the AI-Driven Development Lifecycle (AI-DLC) from AWS, described by Raja SP on the AWS DevOps blog on 31 July 2025. It is worth reading because it names the moving parts.

AI-DLC has three phases. Inception: "AI transforms business intent into detailed requirements, stories and units through 'Mob Elaboration', where the entire team actively validates AI's questions and proposals." Construction: AI "proposes a logical architecture, domain models, code solution and tests through 'Mob Construction', where the team provides clarification on technical decisions and architectural choices in real time." Operations: AI "applies the accumulated context from previous phases to manage infrastructure as code and deployments, with team oversight."

The operating rule underneath all three is the same one the diagram above draws: "AI creates a plan, asks clarifying questions to seek context, and implements solutions only after receiving human validation." AI proposes and executes; humans decide. The post also states that AI "saves and maintains persistent context across all phases by storing plans, requirements, and design artifacts to your project repository", which is the durable-artifact point made concrete: the spec lives in the repo, next to the code it produced.

At the scale of one developer and one task, Plan mode is Mob Elaboration with a team of one: the agent studies the project and the task, asks its clarifying questions, and writes a plan with related files that you approve before any code is generated. The plan is a file in the project, which is AWS's persistent-context point in its smallest form. What Explyt adds to the AWS sketch is the boundary. AI-DLC describes what the agent proposes; Edit Scope describes what it is allowed to touch while proposing it, so the Inception hand-off carries both the intent and its limits.

What the AWS post does not do is use the word evaluation. It talks about tests, generated in Construction. I treat that as a vocabulary gap rather than a disagreement: an eval set is a test suite whose purpose is to accept a regeneration, and once code is cheap to regenerate, that is what most acceptance tests become. AI-DLC is cited here as evidence that the lifecycle is being drawn the same way by people who arrived at it independently. It is not an endorsement of any tool, and the AI-DLC white paper was not reviewed for this essay.

Bolts versus sprints: what changes when the cycle is hours

AI-DLC replaces sprints with "bolts", which it defines as "shorter, more intense work cycles measured in hours or days rather than weeks", and replaces epics with units of work. The renaming matters less than the reason behind it.

A sprint is two weeks long because that is roughly how long a team of people needs to implement a coherent slice of work and get it reviewed. The sprint ceremonies exist to protect that human implementation window: planning at the start so people are not interrupted, review at the end so stakeholders see a batch, retrospective so the team adjusts. When implementation takes hours, the window the ceremonies protect no longer exists. Running planning, review and retro for every four-hour bolt is impossible, and skipping them means the bolt has no validation at all.

That is the fork every team hits. One branch keeps the sprint cadence and accepts that agents idle for most of it, which is safe and slow. The other branch adopts bolts and lets validation lag behind generation, which produces the tell from the first essay: throughput without verified output, reported as velocity. The branch that works is the one where validation is mostly automated, as an eval set that runs per bolt, and human attention is spent on the two things automation cannot do: deciding what the intent is and deciding whether the outcome meets it. Bolts without evals are simply a faster way to accumulate unverified change.

In practice a bolt in Explyt is one chat: Plan at the top, the approved plan as the spec, generation against Edit Scope, the named run configurations executed and their results in the transcript, then Auto Review starting automatically in the same chat when the main agent finishes. The ceremonies did not disappear; they collapsed into artifacts that are produced per bolt without a meeting. Planning became a file you approve. Review became a separate agent's findings tied to lines of code. The retrospective became the question you ask when validation fails: was it the spec, the eval, or the code. That is the only cadence at which bolts stay verified, and it is the reason the reviewer has to be a separate agent with no write access rather than the author re-reading its own diff.

Humans govern three things

The phrase "human in the loop" is too vague to design a lifecycle around. The emerging pattern is more specific about what humans keep.

Intent. What is being built, why, and what must not change as a side effect. This is written before an agent plans anything, and it is what a reviewer later checks the result against. If intent lives only in someone's head or in a chat thread, the agent will guess, and the reviewer will reconstruct. In Explyt the "must not change" half of intent has a mechanical form: Edit Scope is set per task, directly in the agent, and an agent that cannot write to a directory does not need to be trusted about it. The "what and why" half is the plan you edit before approving it.

Validation. Deciding what counts as evidence for this change, and then reading that evidence. The decision part happens before generation, as the eval set. The reading part happens after, and it is the only place where a green result gets turned into an accepted result. A passing eval that nobody has looked at is a claim, not a decision. Auto Review is built for the reading part: it receives the original task and the final diff, reads the code, uses the IDE's inspections, and returns findings tied to lines. It has no write access. Running it on a model from a different provider tends to surface a different set of mistakes. You decide which findings are valid, check the fixes, and re-run the affected checks; the merge decision does not move.

Outcomes. Whether the delivered behaviour, in operation, does what the intent said. This closes the loop back to intent and is where the next unit of work usually comes from. When an outcome is wrong and the failure is reproducible, Debug mode is how the agent is asked to confirm the cause before touching code: breakpoints, variable values and the call stack, then a minimal fix, then the original scenario and the related tests re-run. One such run confirms the fix for the scenario that was run, and nothing wider, which is exactly the boundary a validation lane should have.

Everything else, proposing a plan, breaking intent into units, generating code, generating tests, refactoring, drafting the change record, is work an agent can execute under those three controls. Where teams get into trouble is when they hand one of the three to the agent as well: letting it infer intent, letting it decide its own acceptance criteria, or treating "tests pass" as the outcome.

Specs and evals become durable artifacts

In a classic SDLC, code was the only artifact that reliably survived. Specifications were written to start the work and drifted the moment implementation began. Tests were owned by the code and rewritten when the code was. Six months later the code was the documentation, because it was the only thing that had been maintained.

In an agent-driven lifecycle the economics invert. Code is the cheap, regenerable output. The specification is the input to every regeneration, so it has to be correct or every regeneration is wrong in the same way. The eval set is how any regeneration gets accepted, so it has to be maintained or nothing can be accepted. The change record ties a spec to a change to the evidence that the change behaves, and it is what makes the whole thing auditable. These become the artifacts a team cannot afford to let rot, and the code becomes the one it can afford to throw away.

Look at which of these an IDE-native agent produces for free and which it cannot. The plan from Plan mode is a spec in the repository. The named run configurations are the eval set in the form the IDE already understands, so "run the evals" is a real operation rather than a summary. Agent Changes is the change record: the full diff, reviewable before you accept it. The Auto Review findings are the validation record for that diff. What the agent does not produce is the judgement that the plan is the right plan and the run configurations are the right evals. Those two stay human, which is why they are the two artifacts worth maintaining by hand.

This is the CMMI practice of a defined, measured process, rebuilt for a fast producer, which is the bridge back to the first essay of the series. It is also the reason the rule for choosing a first agent task is what it is: the tasks that fit first are the ones where a spec and an eval can be written before generation, an owner can be named, and the cost of an error is bounded. A team that cannot yet write those two artifacts for a task is not ready to hand that task to an agent, whatever the demo looks like.

Questions for an engineering lead

Where is intent recorded in a form an agent can plan against and a reviewer can check against? If the answer is a ticket title, the agent is guessing. If the answer is an approved plan file and an Edit Scope, you have the first two boxes.
What is the validation artifact for a change, and does it exist before the change is generated? If it is written after, it tests what was built rather than what was meant. A named run configuration chosen before the bolt is the simplest version.
If a bolt finishes in four hours, what happens in the next four? If the answer is "wait for review", the cadence is still a sprint with an agent inside it. If a separate review agent has already returned findings, the human step is a decision rather than a reading.
Which of your artifacts would you keep if the code were regenerated tomorrow? Whatever you would keep is what has to be maintained now.

Conclusion

The lifecycle that is taking shape is not the old diagram with an agent inside one box. Its stages look familiar, but the ownership has moved: agents execute planning, generation and evidence collection, and humans keep intent, validation and outcomes. The cycle shrinks from weeks to hours, and that only works if validation is automated per cycle and human attention is spent where automation cannot go. The artifacts worth maintaining change from code to the specification and the evaluation set that accept it.

The practical next step is small. Pick one unit of work. Write the intent and the eval set before any generation, run the unit as a single bolt, and note which artifact you had to fix when validation failed. If it was the spec or the eval and not the code, you have drawn the new lifecycle for yourself, on your own repository.

If your team works in JetBrains IDEs, that bolt is one Explyt chat: Plan for the spec, Edit Scope for the boundary, named run configurations and Debug mode for the evidence, Auto Review before you make the call. Documentation and download are at explyt.ai.

Series: from the 40-year playbook (BR-BR-SDLC-01), to the lifecycle taking shape (this essay), to how individual steps in the IDE change under it. Bridged to the R-08 anchor and to R-07, the rule for choosing a first agent task you can verify.

Claude Code Refactored Your Repo. Is the Diff Safe to Merge?

Viktoria — Tue, 25 Aug 2026 15:27:23 +0000

Claude Code or Codex edits a cross-module payment service and reports a green build. Before you merge, you still need to know which declaration changed, what references moved with it, which configuration passed, and what the checks missed.

Use this hypothetical task as the test case:

Rename PaymentService.process() to authorizePayment() and move its implementation from legacy-payment-core to payment-application.

This is a representative scenario, not a recorded product run. A multi-module payment platform may have several process() implementations, Spring wiring and proxies, integration tests in another Gradle source set, generated clients, configuration values that name classes, and run configurations with environment-specific parameters.

Explyt supports two distinct workflows. Explyt MCP lets an external client call selected tools from the open JetBrains IDE: project navigation, run configurations, the debugger, and IDE refactoring. It is free and requires no subscription. The external client still controls the model, context, and tool sequence, so its MCP log and final diff are the record of what it used.

The built-in Explyt agent has broader access to symbol navigation, Find Usages, native refactorings, run configurations, IDE inspections, and debugger state. The sections below use that built-in workflow to show the evidence an IDE can provide. For Claude Code or another MCP client, count a capability only when the configured server exposes it and the call appears in the log.

Resolve the declaration before editing

JetBrains IDEs use indexes and the Program Structure Interface (PSI) to represent PaymentService.process() as a code entity with a class, module, type, hierarchy position, and resolved relationships.

If three modules each contain a method named process, text search returns three matches. Symbol resolution shows which declaration each call targets.

The built-in Explyt agent can start from the resolved declaration. Confirm its class, module, signature, and source location before the edit. A wrong declaration is a reason to stop the refactoring.

With an external client, check that the MCP server exposes the symbol operation you need and that the call log records its use. File contents, selections, and text-search results cannot establish symbol identity on their own.

Trace usages tied to that symbol

After resolving the declaration, the built-in Explyt agent can use JetBrains Find Usages to follow references to that symbol.

For PaymentService.process(), the result can reveal direct calls, test references, and other relationships recognized by the IDE. The resulting change map belongs to the selected declaration rather than every identical string in the repository.

Find Usages has a defined boundary. Reflection, string-based lookup, generated code, templates, configuration files, and external systems may depend on the old name without creating a resolvable IDE reference. The result also depends on completed indexing and language or framework support. A clean result means the current IDE project model found no additional resolved references; dynamic dependencies still need separate checks.

Preview native IDE refactorings

The built-in Explyt agent can invoke JetBrains Rename, Move, and Delete refactorings.

For the payment task, Rename targets the resolved PaymentService.process() declaration and updates references recognized by the IDE. Move can update the package declaration, imports, and resolved references where the language and IDE support it. Delete checks usages before removing an entity so the developer can review its dependencies.

Before applying a refactoring, verify the selected declaration, affected files, related elements, package and import changes, reported conflicts, and any unexpected module boundary. Also note known dependencies missing from the preview.

Compare that preview with the final diff. A semantic rename should update references connected to the selected symbol rather than every process string.

Name the configuration that passed

A semantic refactoring can still fail at a project boundary. Generated sources, annotation processors, Gradle variants, Maven profiles, framework processors, and module dependencies may break only during compilation or a focused test run.

The built-in Explyt agent can run JetBrains configurations and return the result in project context. A useful report identifies the configuration, module or test scope, tests executed, failures, skipped checks, and source locations for errors. A green result covers that named configuration and scope.

The built-in agent can also call IDE inspections. Those findings can locate unresolved symbols, nullability problems, unreachable code, API misuse, and framework-specific issues.

If no test executes the renamed path, the report should record that gap. Compilation and passing tests cover only the selected configuration and test scope.

Use the debugger for the runtime question

The code may compile while a changed module dependency or component-scanning boundary makes Spring select a different implementation after the move. Turn that risk into a runtime hypothesis: the application selects FallbackPaymentService instead of the moved implementation.

Launch the existing configuration that reproduces the path, stop at the service boundary, and inspect the concrete receiver, call stack, and relevant variables. Compare the observed state with the expected dependency wiring.

That debugger run answers the stated hypothesis for one execution. Dynamic references outside the IDE model and paths that did not run remain separate risks.

Read the external agent's MCP record

An external client such as Claude Code, Codex, Cursor, or OpenCode keeps its own model and conversation while calling selected Explyt tools from the open JetBrains IDE. The public Explyt release documents configurable groups for project navigation, run configurations, the debugger, and IDE refactoring.

It does not document separate external calls for Find Usages, implementations, overrides, or IDE inspections. Nor does it guarantee that a client will choose the right build, test, and debugger sequence.

For an external run, keep a compact acceptance record:

enabled tool groups and MCP calls, including arguments, results, and failures;
the selected run configuration and checks that did not run;
the refactoring preview, if the configured operation returns one;
the final repository diff.

The call log shows which Explyt MCP tools the external agent used. To see this workflow before testing it on your own refactoring, watch the Explyt MCP demo.

Compare published MCP operations

JetBrains also publishes an MCP Server for external clients. The IntelliJ IDEA documentation lists operations including get_symbol_info, search_symbol, rename_refactoring, build_project, execute_run_configuration, and get_file_problems.

The published list does not separately document Find Usages, implementations, overrides, Move, or Delete. Compare the operations exposed by your installed versions, then check the call logs for the task you ran.

AgentLens provides a configuration-level comparison

The AgentLens Java coding-agent leaderboard evaluates complete coding-agent trajectories: requests, messages, tool calls, file edits, verification attempts, final answers, and repository state. It combines formal checks with written reviews by LLM judges.

The published Opus 4.7 Java task set reports these results for complete Explyt AI Agent and Claude Code configurations:

Configuration	Model	Overall score	Median time, s	Median generated tokens	Formal result
Explyt AI Agent	Opus 4.7	81.5	256	15,621	81.2
Claude Code, xhigh effort	Opus 4.7	76.2	312	16,534	81.2

Explyt reached the same formal result as Claude Code with a 56-second lower median time and 913 fewer median generated tokens. Its overall score was 81.5, compared with 76.2 for Claude Code.

These numbers describe the benchmarked configurations as a whole. They do not isolate IDE access as the cause of the difference or predict results for your repository.

Keep one acceptance record for the merge decision

Run a refactoring you already understand and save one record with five parts:

Identity: the resolved declaration, module, signature, and source location.
Scope: resolved usages, affected files, related elements, conflicts, dynamic dependencies, and unexpected module crossings.
Checks: the named build or test configuration, tests executed, failures, skipped checks, inspection findings, and remaining gaps.
Runtime evidence: the hypothesis, reproducing configuration, concrete receiver, call stack, and relevant variables.
MCP provenance: enabled tool groups, calls and arguments, returned results, failures, unavailable built-in capabilities, and omitted checks.

The final diff should match the preview and the identity recorded at the start. Together, these artifacts tell you what the agent changed and which risks you tested. You decide whether that evidence is sufficient to merge.

Sources

Explyt release: Explyt 5.17: IDE refactoring and an MCP server for external agents.
Explyt documentation: Explyt 5.17: IDE refactoring and an MCP server for external agents.
Explyt comparison: Cursor Just Entered JetBrains IDEs. Can It Finally See What the IDE Sees?.
AgentLens: Java coding-agent leaderboard.
AgentLens paper: Production-Assessed Coding Agents.
JetBrains documentation: MCP Server.

Which development tasks are worth delegating to an AI agent?

Viktoria — Tue, 25 Aug 2026 12:41:59 +0000

An AI agent can return a large patch quickly. The developer still has to understand the changes, verify the result, correct missed cases, and decide whether the patch is safe to accept. If that work takes longer than completing the task manually, the agent has added cost rather than saved time.

Request counts, generated lines, and token use describe model activity and cost. To measure value, the team needs two stronger signals: the change in active developer time and whether the result met predefined acceptance criteria.

Evaluate each task on three parameters:

how precisely the team can define the required result;
which independent checks can confirm it;
what happens if an error passes every check.

This article calls the parameters certainty, verifiability, and error cost. Each level has observable criteria. Two assessors record their evidence and rationale independently, while the team measures assessor agreement separately.

Before a pilot, the team estimates the share of tasks that may support autonomous agent work within defined boundaries. Afterward, it calculates the confirmed share: tasks where the agent reduced active developer time and the result met the predefined acceptance criteria.

What does "hand a task to an agent" mean?

In one team, delegation means asking the agent to find the required class. In another, the agent receives a task, changes the code, runs checks, and returns a patch for review. Comparing such results with a single percentage is meaningless.

Define the workflow in six steps:

Gather project context and find the affected locations.
Clarify requirements and constraints.
Propose a change plan.
Change code, tests, configuration, or documentation.
Run checks and fix the problems found.
Accept the result and approve it for merge, release, or publication.

In this method, a delegated task includes the agent’s autonomous work on actions one through five within agreed boundaries. The developer sets the acceptance criterion, reviews the evidence, and makes the final decision.

For the remaining tasks, the team chooses one of two modes:

short-step collaboration: the agent investigates or changes a small part, and a person checks the direction before the next step;
preparatory assistance: the agent collects information, looks for dependencies, prepares options or test data, while a person makes the decision and performs the main change.

This division prevents preparatory assistance from being presented as autonomous delegation. Searching documentation with a model is useful in itself. The ability to assign the agent a change to payment logic requires a separate check.

Unit of evaluation: a specific task in a specific project

A category name is too broad. “Write tests” may mean three unit tests for a pure function with a clear contract or restoring integration coverage after the migration of several modules. “Perform a refactoring” may mean renaming a local symbol or moving a shared data model between services.

Before scoring, the task card must state:

expected result;
permitted change area;
known constraints;
available checks;
affected users and systems;
rollback method;
person responsible for acceptance.

If a card combines several independent results, it is better to split it. An agent may handle updating a DTO well and handle changing authorization rules poorly, although both changes are part of one tracker task.

Three task-fit parameters

Score each parameter from 1 to 3, but keep the scores separate. A combined score would distort the evaluation: high verifiability does not reduce error cost, and a clear requirement does not create an independent checking method.

1. Certainty of the result

Certainty answers the question: can the team describe, before implementation begins, what should change and what should remain unchanged?

Level 1: low

Assign 1 if at least one condition applies:

the product owner has not yet selected the required behavior;
the task contains several architectural directions without an agreed selection criterion;
it is unknown which systems or processes depend on the change;
the documentation and code diverge, and the source of truth has not been defined;
the expected result can be formulated only after investigation;
the boundaries of the change cannot be named even at the module or component level.

Example: determining the boundaries for splitting a monolith into services when transactional relationships and data-consistency requirements are unknown.

Level 2: medium

Assign 2 if the main behavior is agreed but one or more limited questions remain:

the result for the main scenario is known, but individual edge cases are not defined;
the change area is clear at the subsystem level, while the exact file list will appear after analysis;
there are two acceptable technical options, and the selection criterion is defined;
some dependencies must be confirmed in code or configuration;
work can begin without another product decision, but the team must reach a human checkpoint after the investigation and before proceeding.

Example: adding request retries with an agreed number of attempts when the method for storing state must be chosen after checking the existing infrastructure.

Level 3: high

Assign 3 only if all conditions are met:

the result is described by a contract, examples, or precise acceptance scenarios;
the boundaries of permitted changes are defined;
unchanged behavior is listed;
external dependencies and constraints are available to the agent or explicitly stated;
product and architectural decisions have already been made;
two developers can independently read the task description and expect the same result.

Example: fixing a reproducible serialization defect in one module while preserving the public schema and existing behavior of the other formats.

If assessors disagree between levels 2 and 3, keep the task at level 2 until the team provides evidence that resolves the disagreement. That evidence may be a product-owner decision, an API contract, an edge-case table, or a list of permitted modules.

2. Verifiability of the result

Verifiability answers the question: will the team be able to confirm the result independently of the agent’s explanation?

An independent check relies on a basis different from the agent’s implementation. A test written by the agent together with the code from one ambiguous task description helps find some errors. A predefined acceptance scenario provides stronger confirmation.

Level 1: low

Assign 1 if at least one sign is present:

correctness can be assessed only by reading a large or complex change;
there is no reproducible scenario, and observable behavior depends on the production environment;
tests are absent, unstable, or test another part of the system;
the success criterion is formulated subjectively;
the team cannot show a state before the change in which the criterion has not yet been met;
critical dynamic dependencies are not visible in the available checking environment.

Example: changing a distributed process whose error appears rarely and is diagnosed only from incomplete logs of several services.

Level 2: medium

Assign 2 if the main behavior can be checked but a noticeable area of manual control remains:

there are tests for the main scenario, while some edge cases are checked manually;
the build and static analysis are reproducible, but the integration environment is unavailable;
the defect is reproducible, but confirmation that there are no regressions is limited;
the diff can be checked in a reasonable time, but individual dynamic connections require review;
there is one independent criterion and several confirmations created together with the implementation.

Example: a local fix with a failing unit test when the related integration scenario is checked manually only in a test environment.

Level 3: high

Assign 3 only if all conditions are met:

there is an independent acceptance criterion;
the unmet criterion can be shown before the change and the met criterion after it;
the build and required tests are reproducible in the available environment;
static or structural checks cover the affected connections;
an observable runtime scenario exists for significant behavior;
the final diff is limited and available for review;
the team knows in advance which risks remain outside automated checks.

Example: fixing a defect in an isolated module when an existing test fails before the patch and passes after it, the entire module builds, the regression suite is stable, and the diff affects several understandable locations.

The level depends on the quality of the checks. Ten tests built on one incorrect expectation are weaker than one acceptance scenario specified by the behavior owner before implementation.

3. Error cost

Error cost describes the consequences of a defect that reaches a user or an internal process. Diff size does not predict impact: one line in an access-rights check may be more dangerous than a new internal tool consisting of hundreds of lines.

Assessors score the worst plausible consequence. An average scenario may conceal rare but critical damage.

Level 1: low

Assign 1 if all conditions are met:

the error affects a limited internal audience or reversible work material;
there is no risk of data loss, financial damage, access violation, or breach of contract;
rollback takes a short time and does not require state restoration;
the error is quickly detected by current checks or a user;
dependent systems receive no irreversible changes.

Examples: a draft of internal documentation, report formatting, or a local developer tool without access to production data.

Level 2: medium

Assign 2 if the consequences are limited but require noticeable recovery:

the error may disrupt part of the users or an internal process;
data can be recalculated or restored from a reliable source;
rollback is known but requires coordination or a time window;
the public interface does not change, or the change is compatible;
there is no direct risk to security, payments, legal obligations, or irreversible data.

Examples: a failure of an additional feature that can be quickly disabled by a flag, or an internal analytics error that can be recalculated.

Level 3: high

Assign 3 if at least one condition applies:

financial losses or an incorrect payment calculation are possible;
data may be lost, disclosed, or irreversibly damaged;
the change affects authentication, authorization, secrets, or access boundaries;
the error may violate a law, a contract, or a mandatory control;
a critical public API or data format is affected without safe compatibility;
rollback does not return the system to its original state;
a failure may stop a critical service or affect a large share of users.

Examples: a production-data migration, changing a commission calculation, configuring access rights, or removing a public API field.

A release flag, backup, and tested rollback reduce practical risk but do not always change the level. If an incorrect operation manages to disclose data, a subsequent rollback will not eliminate the consequence.

How to choose a mode without conflicts in the matrix

The rules are applied from top to bottom. Once a condition is met, the lower rows are no longer considered. This order removes overlaps between high error cost, low verifiability, and high certainty.

Priority	Condition	Work mode
1	Error cost = 3	A person owns the decision and acceptance. The agent performs only limited steps defined and approved in advance. Independent review and a safe-release plan are required
2	Verifiability = 1	Build a checking method first. Until then, assign the agent research, reproduction, and preparation of checks
3	Certainty = 1	Work in short investigative steps. A person confirms the direction after each step
4	(Certainty = 2 or verifiability = 2) and error cost = 1 or 2	Delegate a limited stage with a mandatory checkpoint and expanded review of the result
5	Certainty = 3, verifiability = 3, error cost = 2	Work collaboratively: the agent performs a limited stage, a person confirms checkpoints and accepts the result after independent checking
6	Certainty = 3, verifiability = 3, error cost = 1	The agent completes the full cycle to an acceptance-ready result. A person reviews the evidence and the final diff

The infographic shows the same constraints in abbreviated form: autonomous delegation requires high certainty, high verifiability, and low error cost. With medium error cost, use a collaborative mode with human checkpoints. High error cost always leaves the decision with a person.

The matrix selects the initial mode for the task's current state. After investigation, certainty may rise from 1 to 2. A failing test and a reproducible run may raise verifiability. Controls such as isolation and tested rollback can reduce exposure or recovery time. Reassess error cost only when new evidence changes the worst plausible consequence itself.

Reassess the card before changing modes. You cannot assume that one successful attempt automatically raises the verifiability of an entire task category.

What evidence to attach to the assessment

Adjectives without artifacts quickly turn an assessment into an opinion. For each parameter, the card must contain a reference, file, test, decision, or described consequence.

For certainty, use:

agreed input and output examples;
an API contract or data schema;
an edge-case table;
a list of permitted modules;
a product-owner decision;
a list of behavior that must not change.

Verifiability requires executable or observable confirmations:

a test that reproduces the defect before the change;
separate acceptance or contract tests;
a specific build and test configuration;
static analysis and inspections with problem locations;
semantic usage and reference resolution, plus compiler or IDE type checks, with the exact tool or configuration recorded;
a run scenario with an observable result;
debugger values and a call stack when runtime behavior is checked;
the final diff and list of affected files.

For error cost, record:

whom and which system the defect will affect;
what data may change;
how the team will detect the problem;
whether error propagation can be stopped;
how long recovery will take;
which consequences will remain after rollback.

The wording “the risk is medium” does not help choose a mode. The record “the error will distort the internal report for the current day, source events are preserved, and recalculation takes up to an hour” provides a verifiable basis for level 2.

Where JetBrains IDE adds facts for acceptance

Verifiability depends on what information about the project the agent receives. File text and command output cover only part of the picture. In Java and Kotlin projects, JetBrains IDE stores a structural code model, symbol links, run configurations, test results, inspection messages, and application state during debugging.

Explyt receives these facts from JetBrains IDE during agent work. The agent can select a code entity, resolve its semantic usages and references, invoke an IDE operation, run a named configuration, and receive a diagnostic with its code location.

IDE facts strengthen checking when there is a predefined expectation. Successful compilation confirms the selected build area but says nothing about the product meaning of the change. A green test confirms only the scenario encoded in it. The debugger shows the state of one reproduced run, so a regression suite is still needed.

Explyt invokes a JetBrains IDE refactoring. This artifact shows the performed operation and the affected symbol; the compatibility of the change is confirmed by separate builds and tests.

Debug mode provides evidence from one reproduced debug session, including configured breakpoints and observed runtime values. A regression suite is needed to check the remaining scenarios.

The card must name every confirmation and its limits. A report saying “everything passed” is weak for acceptance. The record “the payment-service:test configuration completed successfully, 126 tests were run, and the integration suite was not run” allows a person to understand what was checked and what remains.

Examples of assessing real tasks

Update a guide from agreed source materials

Conditions:

the document structure is approved;
the sources are listed;
adding facts without a source is forbidden;
an editor checks the final text;
publication goes through separate approval.

Assessment: certainty 3, verifiability 3, error cost 1. The agent can prepare a complete draft and check the links. A person verifies the facts and wording and accepts the publication.

If sources conflict and the team has not identified an authoritative source, certainty falls to 1. If positioning has one limited unresolved question and a defined human checkpoint, certainty falls to 2. The task category remains the same, but the work mode changes.

Add unit tests to a pure function

Conditions:

the function contract is specified with examples;
edge cases are listed;
neighboring tests show the accepted style;
the test suite is reproducible;
the change does not affect external systems.

Assessment: 3, 3, 1. The agent writes the tests and runs the suite. The developer checks that the tests follow the contract and can fail for an intentionally incorrect implementation.

If the expected behavior can be inferred only from the current code, certainty is no higher than 2. Such tests may lock in a defect.

Fix a reproducible defect in an isolated module

Conditions:

an existing test reproduces the failure;
the change area is limited to one module;
the regression suite is stable;
rollback is simple;
the module does not process critical data.

Assessment: 3, 3, 1. The agent can go through the cycle from analysis to patch. Acceptance includes evidence that the test failed before the change, passes after it, and that the related suite has no new errors.

Add retries to an external call

The main scenario is agreed, but the existing timeouts and idempotency mechanism need to be studied. Unit tests exist, while the behavior of the real provider is checked manually in a test environment. An error may temporarily disrupt the function; release is protected by a flag.

Assessment: certainty 2, verifiability 2, error cost 2. The agent may be assigned the investigation and a limited implementation. After analyzing idempotency, the developer confirms the direction. Before release, the team checks the behavior in the test environment and verifies that the flag disables the change.

Change a payment calculation

Even with a formal contract, error cost is 3. The first matrix rule applies. The agent may find affected locations, prepare test data, and propose a patch. The specialist developer is responsible for the decision, independent tests, edge-case review, and release.

High verifiability allows the scope of the agent’s preparatory work to be expanded, but it does not remove human responsibility for a change with high error cost.

Design a monolith decomposition

Requirements for service boundaries are being clarified, dependencies have been studied only partly, the consequences of the decision are significant, and it is difficult to build full verification before migration.

Assessment: 1, 1 or 2, 3. The agent is suitable for building a dependency map, finding cycles, collecting data accesses, and preparing options. The team makes the architectural decision after investigation.

How to run a pilot on the team's tasks

The pilot must answer two questions:

For which classes of tasks does the agent reduce active developer time at the required quality?
Which properties of the task description and project explain the result?

Token consumption is included in the cost calculation. The team’s goal is to reduce active developer time while preserving the required quality.

Step 1. Build a representative sample

Take completed tasks from a period that reflects the team's workflow. The sample must represent the main categories in realistic proportions: fixes, features, tests, refactoring, integrations, documentation, and research. Include tasks with high error cost within their actual categories and report rare high-error-cost tasks separately.

If half of the time is spent supporting a legacy system, a pilot on new CRUD methods will produce a distorted result. If the sample contains one task of each type, random luck will look like a pattern.

Before the pilot, set a minimum number of comparable repetitions for each category. A category that does not reach its minimum receives preliminary status and stays outside the eligible set used for aggregate task-share calculations. Report its observations separately. Do not derive a stable percentage from a small set of rare high-error-cost tasks.

Build the baseline from comparable tasks. For each pilot task, select a recent manual task from the same category, a similar module, and a predefined complexity group, or compare medians within that group. Matching must account for the affected area, number of dependencies, available tests, and checking cost. If there is no reliable history, perform some new tasks through the current manual process and measure them the same way. Do not use incomparable tasks as evidence of savings.

Step 2. Calibrate assessors and score the cards

Before the main pilot, two assessors independently analyze one small shared sample. They assign levels to the three parameters, record their evidence, and compare their answers. For each parameter, the team calculates a simple share of matches:

Assessor agreement =
  cards with the same assessment
  / all cards in the calibration sample

Calculate the share separately for certainty, verifiability, and error cost. When assessors disagree, they clarify the criterion, add an example, or name a mandatory artifact. Then they repeat calibration on a new small group of cards. Start the pilot after reaching a predefined agreement threshold. On a small sample, this indicator reflects the uniformity of assessments within the team. Check the stability of the conclusions on subsequent groups of cards.

After calibration, two people independently assess the cards in the main pilot. One developer may know about a hidden integration while another considers the task isolated. Their disagreement can surface hidden context before the agent changes the code.

If assessors have not agreed on a level, choose the more cautious mode and record which fact is missing. Preserve the initial assessment: it must not be changed after a successful result to make the forecast look more accurate.

Step 3. Set boundaries and the stop criterion

Before launch, specify:

which directories, modules, and files may be changed;
which commands or configurations may be run;
which actions require confirmation;
which data and environments are forbidden;
on which event the agent must stop;
who accepts the result.

Examples of stop conditions: the need to change a public contract, discovery of a data migration, leaving the permitted module, an unstable test, no access to the required configuration, or a conflict between requirements and current behavior.

Step 4. Record the unmet criterion first

For a defect fix, reproduce the failure before the patch. For a new feature, run an acceptance scenario that does not yet pass. For documentation, save the list of mandatory facts and sources. For a refactoring, record the original symbol, the area of its usages, and compatibility checks.

This sequence shows that the check distinguishes the states before and after the work. If a test was already green before the change, its passing again does not confirm the agent’s result.

The Test Results window records the configuration name, number of tests, run time, and the difference between the expected and actual results. The screenshot confirms one run; the repeat check after the fix must be saved separately.

Step 5. Perform tasks in the assigned mode

Follow the selected mode. Do not let a low-certainty task drift into a long autonomous session after launch. A task with high error cost must not enter a merge without an assigned review.

For collaboration, define the scope of each checkpoint in advance: one hypothesis, one module, one test scenario, or a limited diff. Checkpoints are also part of active developer time.

Step 6. Measure the full cycle

For every attempt, record:

time spent preparing the task description and context;
active time communicating with the agent;
autonomous machine time;
waiting time that the developer actually used for other work;
time spent checking evidence and the diff;
time spent on manual fixes;
number of retries;
rework outcome: accepted as returned, accepted after minor rework, accepted after substantial rework, or rejected;
checks performed and skipped;
defects found before merge and defects found during a predefined post-merge observation window;
model cost, including retries;
the reviewer’s subjective workload on a short scale, for example from 1 to 5.

The main time metric:

Active developer time =
  task description
  + active interaction
  + checking
  + manual fixes
  + recovery after defects

Autonomous machine time is stored separately. If the agent worked for twenty minutes while the developer was solving another task, those twenty minutes must not be added in full to human costs. If the developer watched the process the whole time and answered questions, that is active work.

Step 7. Review evidence independently of the agent's summary

The accepting developer reviews the saved evidence:

the original criterion was in fact unmet;
it was met after the change;
the configurations run and the scope of the checks are named;
skipped checks are visible;
the final diff matches the boundaries;
there are no forbidden changes;
the remaining risks are clear to the developer accepting the result.

The agent’s phrase “task completed” does not confirm the result.

The Auto Review result stores the checked area and the findings for the changed code. The report complements the results of tests, inspections, and diff review listed in the acceptance criteria.

Step 8. Compare with the baseline within each category

Compare local fixes with local fixes, documentation with documentation, and integration tasks with integration tasks. Within a category, use matched pairs of similar complexity or predefined complexity groups. An overall average across different work classes hides the causes of the result.

For each eligible category, calculate the median active developer time, the share of results accepted without substantial rework, retry frequency, and post-merge defects. State alongside them the number of observations, matching method, and complexity range. Report categories below the minimum number of repetitions as preliminary observations outside aggregate shares.

Step 9. Analyze forecast errors

There are two useful error types:

the task was considered suitable, but the agent created more work;
the task was considered unsuitable, but a limited mode produced a good result.

For the first case, find the missed factor: an implicit requirement, a weak test, a dynamic dependency, boundaries that were too broad, costly checking, or insufficient project context. In the second case, record which artifact reduced uncertainty or strengthened verifiability.

After several cycles, the team will obtain rules for its recurring tasks. These rules must be reviewed after changes to the architecture, test infrastructure, model, toolset, or release requirements.

How to calculate task shares

Use four separate indicators. In every aggregate formula below, eligible tasks means tasks from categories that reached their predefined minimum number of comparable observations.

1. Forecast share of autonomous delegation

Forecast share =
  eligible tasks that fall under matrix rule 6
  / all eligible assessed tasks

It shows the potential based on the cards before launch. The autonomous mode includes only tasks with high certainty, high verifiability, and low error cost. Rule 5 concerns collaborative work.

2. Confirmed share

Confirmed share =
  accepted eligible tasks from autonomous mode,
  where active developer time is below the baseline
  and there is no disqualifying defect
  / all eligible pilot tasks

The team defines a disqualifying defect before the pilot. It may be a rollback, an incident, a data violation, a missed mandatory check, or manual rework comparable to a new implementation.

The strict denominator shows what part of the overall stream can already be delegated with confirmed value. Success can also be calculated separately only among launched autonomous tasks.

3. Share of tasks for collaborative work

Collaborative-work share =
  accepted eligible tasks from rules 3, 4, and 5,
  where active time is below the comparable baseline,
  there is no substantial rework
  and there is no disqualifying defect
  / all eligible pilot tasks

Set quality criteria and the list of disqualifying defects before launch, as with the confirmed share. Report collaborative work separately from autonomous delegation so the team can see where the agent works autonomously and where short checkpoints reduce developer effort.

4. Effort-weighted share

Ten small fixes and one week-long integration produce the same eleven tasks but affect workload differently. Therefore, alongside the count-based share, calculate a weighted share:

Weighted confirmed share =
  sum of baseline human time for confirmed eligible tasks
  / sum of baseline human time for all eligible sampled tasks

The weight comes from baseline human time. Agent generation time does not affect the weight. The indicator answers the question of what part of the previous human workload is covered by successfully delegated tasks.

Do not combine these four values into one index. It is more useful for a manager to see separately the autonomous-mode share, collaborative-work share, number of observations, and categories in which active developer time decreased.

Metrics to keep on the dashboard

A pilot dashboard can be assembled from eight groups of indicators:

Coverage: number of assessed tasks and distribution by category.
Modes: forecast autonomous share; confirmed autonomous and collaborative shares; assignment counts for preparatory assistance. Preparatory assistance has no success-share metric because the method does not treat it as delegation.
Acceptance: share of results accepted without substantial rework, reported separately from the confirmed autonomous share.
Human time: median active time and change relative to the baseline.
Checking: median review time and a list of performed and skipped checks.
Retries: how many reruns were required before acceptance.
Quality: defects found before merge, defects found during the predefined post-merge window, rollbacks, and incidents.
Economics: model cost per accepted task and per saved hour of active time.

Tokens can be stored as technical cost detail. Their growth shows an increase in cost, but cannot be used to judge pilot success. An agent with a larger context may spend more tokens and reduce checking. A more economical agent may return a patch that the team deletes.

Task assessment card

Task:
Category:
Module or system:
Complexity group:
Expected result:
What must remain unchanged:

ASSESSOR 1
Name or role:
Assessment date and criteria version:
Certainty: 1 / 2 / 3
Verifiability: 1 / 2 / 3
Error cost: 1 / 2 / 3
Evidence and rationale:

ASSESSOR 2
Name or role:
Assessment date and criteria version:
Certainty: 1 / 2 / 3
Verifiability: 1 / 2 / 3
Error cost: 1 / 2 / 3
Evidence and rationale:

RECONCILED ASSESSMENT
CERTAINTY: 1 / 2 / 3
Unknown questions:
Who makes product and technical decisions:

VERIFIABILITY: 1 / 2 / 3
How to show the unmet criterion before the change:
Independent acceptance criterion:
Which builds, tests, inspections, and runs are available:
What behavior is checked at runtime:
What will remain unchecked:

ERROR COST: 1 / 2 / 3
Who and what the defect will affect:
Can error propagation be stopped:
Rollback method and time:
What will not be restored after rollback:

MODE FROM THE MATRIX
Matched matrix rule: 1 / 2 / 3 / 4 / 5 / 6
[ ] Autonomous execution by the agent
[ ] Short-step collaboration
[ ] Preparatory assistance

Required controls from the matched rule:
[ ] Build a checking method first
[ ] Human checkpoint after each investigative step
[ ] Mandatory checkpoint and expanded result review
[ ] Independent review
[ ] Safe-release plan
[ ] Steps defined and approved in advance

Permitted files, modules, and actions:
Forbidden actions and environments:
Stop condition:
Checkpoint:
Who accepts the result:

BASELINE
Baseline task, pair, or group:
Baseline active developer time:
Matching method:
Complexity range:

AFTER EXECUTION
Task-description and context time:
Active interaction time:
Checking time:
Manual-fix time:
Recovery-after-defects time:
Active developer time total:
Autonomous machine time:
Waiting time used for other work:
Retries:
Rework outcome: accepted as returned / minor rework / substantial rework / rejected
Checks performed:
Checks skipped:
Defects found before merge:
Post-merge defects:
Post-merge observation window:
Model cost:
Reviewer workload, 1-5:
Outcome: accepted / accepted after rework / rejected

Method limitations

The matrix does not assess model quality separately from the environment. One agent can use project tools, while another is limited to reading files and terminal commands. Their results on the same card may differ.

Assessment depends on the team's knowledge. A developer who has been responsible for a module for many years sees hidden constraints. A new contributor may not know them. The card therefore stores both assessors, their original scores, and the evidence behind each score.

Automated checks confirm only the expectations built into them. High coverage does not help if tests preserve outdated behavior. Significant changes need a criterion connected to a real user or system scenario.

The accuracy of historical time depends on the accounting method. If the baseline was collected from subjective estimates, label the savings conclusion as preliminary. It is better to measure the manual and agent processes in the same way.

The confirmed share describes the studied sample under the current architecture and selected toolset. For another team, calculate it again in its own pilot.

Conclusion

This method identifies recurring tasks where an AI agent can reduce active developer time and still meet the team's acceptance criteria. Certainty, verifiability, and error cost determine the work mode; pilot results then show whether autonomous or collaborative work delivered measurable value against a comparable baseline.

Start with a calibration sample and set the minimum number of observations for each category. If verifiability is the bottleneck, improve reproducible builds, tests, inspections, run configurations, and reproducible debug scenarios with captured runtime evidence.

In JetBrains projects, Explyt can use semantic code information, named run configurations, test results, inspections, and debugger state during the agent workflow. A person still owns the acceptance criteria and the merge decision.

A legacy refactoring that is small only in the ticket

Viktoria — Fri, 21 Aug 2026 07:45:06 +0000

Consider this task in a multi-module payment platform:

Rename PaymentService.process() to authorizePayment() and move its implementation from legacy-payment-core to payment-application.

The repository includes:

a PaymentService interface shared by several modules;
multiple process() implementations;
Spring wiring and framework proxies;
integration tests under a separate Gradle source set;
generated clients;
configuration values that refer to classes by name;
a production run configuration with environment-specific parameters.

A repository-wide search for process returns candidates. It cannot tell you which declaration each call resolves to.

A green compile proves something equally narrow: one configuration compiled one scope of code. It says nothing about whether Spring still wires the intended implementation or whether a string-based lookup resolves at runtime.

This is the gap between an IDE-native workflow and a chain assembled from grep output and terminal logs.

Give Claude Code your JetBrains IDE tools for free

Viktoria — Thu, 20 Aug 2026 11:12:19 +0000

Using Claude Code on a large codebase while keeping a JetBrains IDE open beside it? Give the agent access to what the IDE already knows. Explyt MCP connects Claude Code, Codex, Cursor, OpenCode, and other MCP clients to IDE navigation, refactorings, run configurations, and the debugger. The agent can retrieve project facts directly instead of reconstructing them from extra file reads, grep results, and terminal logs. That can reduce token spend, shorten tool chains, and leave fewer opportunities to act on the wrong symbol or stale runtime assumptions.

JetBrains also provides an MCP server, but its published tool set does not include several Explyt operations: native Move and Delete, a separate Find Usages command, and debugger access beyond IntelliJ IDEA Ultimate. Explyt MCP adds those capabilities to your existing external agent.

Consider a rename in a multi-module Java project. The agent must identify the intended declaration, find its references, refactor it, and verify the result.

The public AgentLens Java coding-agent leaderboard provides a same-model comparison with Claude Code. With Opus 4.7, Explyt recorded a median run time of 256 seconds and 15,621 generated tokens, compared with 312 seconds and 16,534 generated tokens for Claude Code. Both configurations reached the same 81.2 formal-verification score. This comparison evaluates complete agent configurations, so it does not isolate IDE tools as the only cause. It does show the practical pattern behind this article: an agent with IDE-native operations finished the published Java task set with less time and fewer generated tokens in this comparison.

Start with the symbol, not the string

Suppose the task is to rename PaymentService.process. A text search for process will return comments, configuration values, generated files, and unrelated methods with the same name. An agent can rank those matches, but ranking does not establish which declaration the task refers to.

A JetBrains IDE has indexes and a Program Structure Interface (PSI). With access to those services, an agent can identify PaymentService.process as a code symbol and inspect its containing class, module, type information, and place in the hierarchy.

When several modules define a process method, text search returns candidates; symbol resolution identifies the declaration each reference points to.

Follow resolved references

After resolving PaymentService.process, the agent needs to see what depends on it. Direct calls and test references may span several modules. An interface declaration or an override can widen the change further.

Find Usages in the IDE follows resolved references instead of collecting every occurrence of the identifier. This gives the agent a better basis for estimating the scope of the rename.

Static analysis still has boundaries. Reflection, string-based lookup, generated code, templates, configuration files, and external systems may refer to the old name without creating a reference the IDE can resolve. The result also depends on completed indexing and language or framework support.

A clean Find Usages result only means that the current IDE project model contains no more resolved references; runtime lookups may still use the old name.

Let the IDE perform the refactoring

Once the scope is known, the agent should call a native JetBrains refactoring for the resolved symbol. Explyt supports three such operations: Rename, Move, and Delete.

Rename updates references to the selected code symbol and can include related elements recognized by the IDE. Moving a class or file through the refactoring engine can update its package, imports, and resolved references. Before Delete removes an entity, the IDE checks its usages so the agent can see what still depends on it.

Before applying any of these operations, Explyt can show the affected elements, files, references, related changes, and conflicts reported by the IDE. For PaymentService.process, the developer should be able to inspect that scope and reject the operation if it reaches an unexpected module or misses a dependency they know about.

After Rename, the diff should contain edits linked to the selected declaration rather than replacements of every process string. Review Move and Delete results as well: reflection, generated code, configuration references, indexing state, and language support can limit what the IDE detects.

Compile with the project configuration

A rename can pass the refactoring engine and still break the build. Generated sources, module profiles, or framework processors may expose errors only during compilation.

The agent should use the project's build or run configuration rather than invent an approximate shell command. Its report must identify the selected configuration, module, and source location of each compiler error; otherwise, "the build passed" is hard to evaluate.

A successful build does not prove that the renamed method still implements the required behavior.

Run the tests and inspect IDE problems

Explyt can work with JetBrains run configurations instead of reconstructing the test command from a terminal session. For the PaymentService.process rename, the agent can launch the relevant test configuration in the IDE and receive the actual results in the project context. The report should name the configuration, executed tests, failures, and skipped checks.

The built-in Explyt agent can also call IDE inspections. They identify unresolved symbols, nullability problems, unreachable code, API misuse, and framework-specific issues, with source locations for every finding.

Passing tests cover only the paths and requirements encoded in those tests.

If the change involves runtime state, Explyt can continue through the JetBrains debugger: launch an existing configuration, stop at a breakpoint, and inspect the call stack and variables. That is a separate experiment with a stated hypothesis, not a substitute for compilation and focused tests.

Built-in Explyt agent or external MCP client?

The built-in Explyt agent chooses and sequences IDE operations inside its own action loop. Explyt MCP exposes enabled tool groups to an external client, which keeps its model and conversation context and decides when to call each tool. Check the MCP log, selected run configuration, refactoring preview, and final diff to see what the external agent actually used.

JetBrains MCP vs Explyt MCP

JetBrains documents get_symbol_info, search_symbol, rename_refactoring, build_project, execute_run_configuration, and get_file_problems. Explyt MCP adds:

Move and Delete: native IDE refactorings alongside Rename.
Find Usages: resolved references for the selected symbol.
Broader debugger availability: debugger tools across supported JetBrains IDEs; JetBrains documents its MCP debugger commands for IntelliJ IDEA Ultimate.

Explyt MCP is available for free. You can keep Claude Code or another MCP client, connect it to the open JetBrains project, and test these operations without buying an Explyt subscription.

Try it on a task you already know

Give the agent a refactoring task you already understand, then watch how it works.

Does it spend minutes assembling sed commands, scripts, and text replacements, or invoke the IDE refactoring directly?
Does it resolve the symbol, retrieve its usages, and preview affected files and conflicts before changing code?
Does it run the configured build or test task and return source-linked errors, or invent a shell command and leave you searching its logs?
If the failure depends on runtime state, can it set a breakpoint and inspect the actual call stack and variables?
Can you inspect the MCP calls, arguments, results, and failures after the run?

Record elapsed time, generated tokens, files touched, manual corrections, and the final verification result. Compare them with the AgentLens figures above, then decide whether the difference holds for your repository and IDE.

Sources

Explyt release: Explyt 5.17: IDE refactoring and an MCP server for external agents.
Explyt documentation: Explyt 5.17: IDE refactoring and an MCP server for external agents.
Explyt comparison: Cursor Just Entered JetBrains IDEs. Can It Finally See What the IDE Sees?.
AgentLens: Java coding-agent leaderboard.
JetBrains documentation: MCP Server.

Explyt 5.17 Lets It Refactor the Project.

Viktoria — Wed, 19 Aug 2026 07:33:09 +0000

Move, Delete, Rename, run, and debug through JetBrains IDE mechanisms instead of reconstructing project relationships from text alone.

AI coding agents are good at producing patches. The harder problem begins when a change crosses file, package, module, or runtime boundaries.

Move a class through the file system and its package may become wrong. Delete an entity without checking usages and another module may stop compiling. Rename a symbol as text and unrelated strings can change with it.

JetBrains IDEs already resolve many of these relationships. Explyt 5.17 gives agents a direct path to that project knowledge.

The central change in Explyt 5.17 is simple: an agent can ask the IDE to perform and inspect engineering operations that the IDE already understands.

Why Are File Edits Not Enough?
A repository is more than a collection of text files.

Code entities are connected through:

resolved symbol references;
imports and package declarations;
module and source-root boundaries;
run configurations;
debugger state;
framework and language support;
generated code and configuration.
A text edit can produce a clean-looking diff while missing one of those connections. Explyt 5.17 reduces that risk by routing supported refactoring operations through native JetBrains IDE mechanisms.

The IDE contributes indexes, symbol resolution, usage search, conflict detection, and refactoring support. The agent contributes task planning, tool selection, and follow-up verification.

What Changed in Refactoring?
Earlier versions already supported Rename. Explyt 5.17 adds Move and Delete.

Move
Move can relocate supported files, folders, and code entities through IDE refactoring.

For a class moved to another package, the IDE can update the package declaration, imports, and resolved references it recognizes in the open project. Before the operation is applied, the agent receives information about affected locations and conflicts.

This is materially different from moving a file and repairing references afterward with a sequence of searches and edits.

Delete
Delete checks usages before removing a supported entity.

The agent can inspect the dependencies found by the IDE, revise its plan, and handle affected code before deletion. This makes the usage check part of the operation rather than an optional cleanup step after the entity has disappeared.

Rename
Rename continues to operate on code entities and their resolved references rather than every matching character sequence.

Depending on the language, entity type, and IDE support, the operation may also include related elements. Renaming a class, for example, may include its file name.

What Does the Agent See Before a Change?
Explyt can expose the IDE's refactoring preview and conflict information to the agent.

Before Move, Delete, or Rename is applied, the operation can identify:

the elements selected for the change;
files and references included in the refactoring;
conflicts that block or complicate the operation;
related entities the IDE proposes to update.
That preview matters most when a task spans several packages or modules. It gives the agent evidence about the scope before files are written and creates a checkpoint where the plan can change.

inspect scope → review conflicts → apply a bounded refactoring → build → test → inspect the diff

This loop is stronger than treating a successful tool call as proof that the task is complete.

Where Does IDE Analysis Stop?
IDE-backed refactoring improves the evidence available to an agent. It does not guarantee complete dependency discovery.

Results depend on:

language and framework support in the installed JetBrains IDE;
successful completion of project indexing;
the relationships the IDE can resolve;
reflection and other dynamic references;
generated code;
configuration files and templates;
references held in external systems;
text mentions that are not symbol references.
A large refactoring still needs a diff review, a build of affected modules, and relevant tests. If the change touches runtime behavior, reproduce the affected path or inspect it with the debugger.

Can External Agents Use the Same IDE Tools?
Yes. Explyt 5.17 includes an MCP server that runs inside the IDE.

It can connect Claude Code, Codex, Cursor, OpenCode, and other MCP-compatible clients to selected IDE operations in the context of the open project.

The external client keeps control of its own model, prompts, context, and call sequence. Explyt executes the selected IDE tool calls and does not provide inference in this mode.

The MCP server is available free without an Explyt subscription.

This gives developers a practical way to keep their preferred agent while adding structured access to JetBrains IDE capabilities.

Which Tool Groups Can You Expose?
The server configuration lets you enable only the groups required for the current task:

project navigation;
run configurations;
debugger operations;
IDE refactoring.
The client does not receive every IDE capability automatically. Its available tools depend on the selected server configuration.

That boundary supports a narrower setup: a client investigating a failing test may need navigation, run, and debugger tools but no refactoring access. A client reorganizing packages may need navigation and refactoring, followed by a run configuration for verification.

How Do You Inspect MCP Calls?
The MCP server window includes a call log.

For each request, you can inspect:

the tool requested by the external agent;
the arguments sent with the call;
the response returned by the IDE tool;
the step at which an error occurred.
This creates a concrete record of the interaction between the agent and the IDE. If a debugger workflow fails, you can see whether the problem came from the selected tool, its arguments, the run configuration, or the returned state.

Before sharing logs or server configurations, review them for local paths, source code, and other project data that should remain inside the working environment.

What Does Chat Branching Actually Separate?
Explyt 5.17 can create a new chat from a selected message.

The new conversation inherits history up to that point. From there, the two chat histories develop independently. This is useful when one investigation produces several directions:

tests in one chat and documentation in another;
implementation in one chat and code review in another;
competing architecture discussions from the same initial context;
a fresh path after an unsuccessful approach.
The separation applies to conversation history, not to project files.

Both chats continue to work with the same open project and the same file system. A change made from one chat is visible to the other. Chat branching does not create a Git branch, worktree, project copy, or isolated workspace.

Use Git branches or worktrees when parallel alternatives must not modify the same files.

What Else Improved in 5.17?
The release reduces unnecessary background work by refreshing the context panel less often and processing internal project data more efficiently. Internal log growth is limited as well.

It also fixes reliability issues in several paths:

background tools resume work in chats more reliably;
tool calls recover more correctly from incomplete arguments;
absolute Windows paths are handled correctly;
issues involving enterprise models are fixed;
large MCP server responses are handled more reliably.
These fixes require no extra configuration after the update.

A Practical Verification Loop
IDE access is useful when it shortens the distance between a proposed change and evidence about its effect.

For a refactoring task, use this loop:

Wait for indexing to finish.
Ask the agent to identify the target entity and affected scope.
Inspect the IDE preview and conflicts.
Apply one bounded Move, Delete, or Rename operation.
Review the resulting diff, including configuration and generated-code boundaries the IDE may not resolve.
Build the affected modules through the relevant run configuration.
Run focused tests.
For runtime-sensitive changes, reproduce the path and inspect state through the debugger.
Repeat with the next bounded operation.
For an external MCP client, add two checks:

enable only the tool groups needed for the task;
confirm the first request, arguments, and response in the call log.
This keeps the agent's work observable and makes completion depend on project evidence, not on a plausible patch.

Try Explyt 5.17
Update Explyt from Settings | Plugins | Installed, restart the IDE if prompted, open the project, and wait for indexing to finish before the first refactoring.

If you use Claude Code, Codex, Cursor, OpenCode, or another MCP client, open the Explyt MCP Server window, select the required tool groups, start the server, connect the client, and inspect its first call in the log.

Explyt is available from the download page.

Sources
Explyt 5.17: IDE refactoring and an MCP server for external agents - product facts, capabilities, limitations, setup, and release fixes.
Building a Fair Benchmark for AI Agent Memory Systems - editorial reference for article pacing and section structure only; its benchmark topic, claims, and wording were not reused.

#java

Viktoria — Mon, 17 Aug 2026 15:07:53 +0000

Stop Copying Random Code Snippets And Finally Build Your Persistence Layers With Ease And Confidence!

Viktoria — Mon, 17 Aug 2026 11:43:04 +0000

Viktoria

Aug 17

How to Configure an AI Agent for Your Project: Context, Rules, Skills, MCP — A Java/Kotlin Playbook

9 min read

How to Configure an AI Agent for Your Project: Context, Rules, Skills, MCP — A Java/Kotlin Playbook

Viktoria — Mon, 17 Aug 2026 07:54:19 +0000

This article distills two of our April webinars with a developer of the Explyt agent. The takeaways are general enough that you can verify every claim on your own project — and if you work in Java or Kotlin, most of them land even harder, because a strongly typed, tooling-heavy stack is exactly where good context wins or loses.
Press enter or click to view image in full size

Agent output quality = context quality. Everything else is downstream.

There are five customization levers: rules, skills, agent modes, MCP, and AgentIgnore. Each has its own use case.
AGENTS.md is the project's source of truth. The memory bank is long-term memory.

The two main anti-patterns: insufficient context and overloaded context. Opposite fixes, opposite symptoms.
A TDD/SDD approach with an agent yields the best quality. With local models it’s critical.

Evolution: from autocomplete to agentic systems
The chain is short:
StageWhat it doesMain painAutocompletecontinue a line, finish a functioncontext = current fileChat assistantsdialogue, explaining codehuman = proxy, copy-pasteAgentsread/edit files, run commands, MCPone context overloadsAgentic systemsorchestrator + sub-agents, rolessetup complexity
With chat assistants the developer constantly copied code back and forth — hence “lots of manual work and broken context.” Agents live inside the project, see all the code, and have tools. On top sit agentic systems that coordinate several specialized agents and solve the single-context overflow problem.

Where the agent lives: three classes of tools
Console agents

They live in the terminal. They run on a remote server or in CI and parallelize easily via git worktree.
ProsConsNo IDE vendor lock-inLess control during developmentCan run in CI/CDNo language semantics — only LSPParallelism across many agentsNo autocomplete for controlled generationRiskier security-wise (filesystem access)
Examples: Claude Code, Codex, Gemini CLI, Aider, Goose, Amp (can commit on its own). Fully autonomous: AutoGPT, SWE-agent, OpenHands.

Agents inside the IDE
They understand the language better: PSI / indexes, refactorings via the IDE API, semantic search (Find Usages, Go to Declaration). For a Java/Kotlin codebase this is the difference between an edit that compiles and one that’s a lucky text guess. A familiar UI is an underrated adoption factor for teams.
ProsConsLanguage understanding via PSITied to one IDEDeveloper’s familiar UIHarder to parallelizeHarder to run in CI
Examples: Cursor (a VS Code fork), Explyt, Windsurf, Junie, AI Assistant.

Agents in CI/CD
Async PR review, description generation, auto-fixes, changelogs.
ProsConsA single review standardTake the human out of the loop, slow pipelinesUnpredictable token costsFalse positives kill trust in reviewAccess to sensitive infrastructure
Examples: CodeRabbit, PR-Agent (Codium Merge), GitHub Copilot for PRs, GitLab Duo (can hunt for vulnerabilities).

Fully autonomous
They take a task from a Jira/GitHub issue and go solve it without a human: AutoGPT, SWE-agent, OpenHands.

Context: where it comes from and how it breaks
Manual collection — @-attaching files, classes, methods. Sometimes more reliable than automation, especially when you know exactly what the result should look like.
Automatic collection — three industry approaches:
ApproachWhat it givesWhere it breaksLSP serversAST, syntaxno semantics, poor with broken codePSI (JetBrains)dependency graph, inheritance, inspections, understands broken codetied to the JetBrains platformCode RAGembeddings, flexible searchindivisible code gets chunked; knows nothing about compilation
PSI is the only one of the three that sees code semantically: it knows for a fact that a method is inherited, that an annotation applies, that a function is used right here. This is precisely the Java/Kotlin superpower — overrides, generics, Spring bean wiring, annotation processing. PSI also works with broken code, so it can explain to the agent exactly what the compilation error is.
The one rule
Result quality depends on context quality. No matter how good the agent is, on bad context it won’t produce the right solution.
Anti-pattern 1: insufficient context

Symptom. The agent solves the wrong task.
Fixes:
Edit the message you already sent instead of tacking on a correction. If you reply “no, you misunderstood,” the agent sees both the original task and the fix in context and starts to get confused.
Use rules for repeated instructions that migrate from chat to chat.
Pass the full spec via MCP — from Jira, Confluence, GitHub.
Plan before solving — most agents can decompose a task before starting. That gives you a de facto spec immediately.
Anti-pattern 2: overloaded context
Symptom. The agent mixes different tasks or suddenly starts solving a third thing.
Fixes:
One task = one chat. A big task → decompose into subtasks in separate chats.
Chat compression — a feature in most agents. It compresses the context into a summary so you can move to a second related task without carrying the whole history.
Five levels of customization
LevelWhat it isWhen to use itRulesrules injected into the system promptcode style, working around model bugs, shell instructions, MCP prioritizationSkillsa description of how to solve a specific task (frontmatter + resources + scripts)recurring tasks: test generation, migrations, refactoringsAgent modesown model, system prompt, tools, and skills per rolea dedicated agent for testing, debugging, analyticsMCP serversexternal toolsJira, Confluence, GitHub/GitLab, Figma, Playwright, Chrome, secrets, TDD flowAgentIgnoreread/write restrictionslegacy, secrets, TDD
Rules
Text rules added to every chat’s system prompt.
What to write:
the rule’s scope (where it applies, where it doesn’t);
what is strictly forbidden.
A classic case is PowerShell on Windows. Most models learned on bash/zsh and get confused on PS. One dedicated rule about PowerShell specifics fixes it once and for all.

Other common uses:
agent style (autonomous vs. controlled);
corporate code style (Google Java Style, parameterized tests — right at home for Java/Kotlin teams);
working around specific model bugs (e.g., Claude 4.5 Sonnet likes to create lots of reports — you can rein it in);
MCP prioritization — if the agent doesn’t call a connected server, spell out when and why to use it.

Skills
Unlike rules, a skill describes how to solve one specific task.

Structure:
skills/
└── my-skill/
    ├── SKILL.md      # prompt + frontmatter with a description
    ├── scripts/      # scripts the agent can call
    └── references/   # docs and specs

The frontmatter holds the skill’s name and description. From the description the agent decides when to invoke it. Manual invocation — /skill_name.

Important about context. Until the skill is invoked, the agent sees only the frontmatter description — no resources or scripts enter the context. That’s what saves you from pollution.

Recommendations:
one goal per skill (even if it has subtasks);
scope and prohibitions — mandatory;
reference scripts and resources from SKILL.md, or the agent won't "notice" them;
the folder structure (scripts/, references/) is optional, but the agent navigates it better;
top-down (write everything at once) is slow and blind to whether it’s needed. Bottom-up (extract from frequently repeated requests) is more practical;
you can generate skills with the agent itself, but validate them by hand.
Compatibility. The SKILL.md format has become a cross-vendor standard — supported by Claude Code, Cursor, Codex, Copilot, and Explyt. A skill written for Claude Code works in Explyt and vice versa.
Agent modes (roles / sub-agents)
Customizable:
the model (e.g., a lightweight model is enough for a tester agent, a stronger one for an architect);
the system prompt describing the role;
the set of available tools;
the set of skills.
A common mistake is describing the role via the org chart (“architect,” “team lead,” “DevOps”). Better to start from the tasks the agent solves. The model doesn’t understand “architect”; it does understand “generates dependency diagrams and checks for cycles.”
Most agents ship with Plan mode and Code mode out of the box. Many can orchestrate — picking which sub-agent to call for a given step.
MCP servers
Not a complicated thing: most often an MCP is a proxy between the model and a service. It parses the model’s arguments, makes an HTTP request, squeezes the response to fit the context limit, and hands it back.
Useful ones:
GitHub MCP — reading repos, search, issues, PRs, branches, commits, review.
Atlassian MCP — Confluence (CQL search, pages) and Jira (JQL search, statuses). Very valuable — that’s where all the company’s domain logic lives.
Figma MCP — project structure, layers, components, variables. Doesn’t handle motion/animation.
Playwright MCP — turning manual clicks into automated tests.
Chrome MCP — DevTools in the agent’s hands.

If the agent doesn’t call an MCP — write a rule, a skill, or a dedicated agent explicitly meant for that MCP.
AgentIgnore
Same syntax as .gitignore. Vendors use different names: .cursorignore, .codeiumignore; Explyt splits "don't read" and "don't edit" into separate lists.
Scenarios:
SituationWhat to forbidLegacy / stable moduleswriting (reading is fine — for understanding)A module too complex, you don’t trust the agentwritingSecrets, creds, .envreading and writingTDD: don't bend tests to the implementationwriting to the test folder
Important. Via the run-command tool, the agent could in theory bypass AgentIgnore. Good agents put a small checker model next to command execution, but even that can be worked around with a clever script. This applies to every vendor — an open question for the industry.
Documentation for the agent
AGENTS.md
A file in the project root. It goes into the system prompt — the agent always knows about it.
What’s inside:
project structure by folders and modules;
the stack;
build and test commands;
code style;
architectural paradigms;
what is strictly forbidden.
Creation — usually an /init command (often a skill) that walks the project and assembles a first AGENTS.md. After generation, validate it: an error in this file lives in context forever. Version it in the repo as the team's source of truth.
Tip: don’t bloat it. The file is added to every chat and eats context. If AGENTS.md grows to 1000 lines, it's no longer help — it's load.
Memory Bank
The agent’s long-term memory — a folder of Markdown files with a size cap. The logic:
Every N messages, the agent pulls relevant facts and adds them to context.
Every N messages, it saves new facts from the current dialogue.
On overflow, it summarizes and drops the irrelevant.
The goal: with each new chat, the agent understands your project better.
Caveat: the smaller N, the higher the token spend. Some agents use N = 1, which becomes constant extra load on the provider.
Context pollution: what the agent should do, and what you should do
The agent:
Cache tokens with the chosen provider — cached tokens are much cheaper.
Auto-compress the chat when the context fills up (thresholds like 75%, 80%).
Save tool results to a file and give the model only the structure. Especially important for unpredictable MCPs whose response easily overloads context.
You:
Know your provider. Anthropic’s cache lives for 5 minutes — small tasks are faster and cheaper within that window.
Compress the chat manually when you see it’s full. Remember: it’s not strictly positive — quality can suffer.
Turn off unneeded tools. Tool descriptions eat context on their own.
Compress AGENTS.md and rules if they've grown and gone stale.
Project setup checklist
StepTimeWhat to do1. AGENTS.md~10 min/init + manual validation2. AgentIgnore5–15 minlegacy, secrets, TDD dirs3. MCP30 minconnect Jira, Confluence, GitHub; compare community vs. official4. Rules15 minagent style + model bug workarounds5. Skillsas neededcheck existing ones in registries first6. Commit1 mincommit it — a single source of truth for the team
Q&A from the chat
Which tasks simply can’t be solved with an LLM? There’s no hard “can’t” if a human sits alongside and validates. Be careful with tasks demanding strict determinism (vulnerability hunting with complex algorithms) — better to give the agent a specialized tool via MCP. With weak/small models, agentic pipelines struggle.
How do you avoid regressing to a junior when using AI? Validate generated code; don’t accept it blindly. Grow expertise in architecture, review, planning, specs. Technical expertise will be needed at any moment.
Cursor vs. IDE agents — worth switching? Cursor is a VS Code fork; you’d have to change your environment. If you’re used to JetBrains, try plugins: Junie, AI Assistant, Explyt. Switching to Cursor pays off only if IDE agents lack the features you need.
When to use SDD, and when not to overcomplicate? It depends on task complexity relative to the model:
Simple tasks — just dive in.
Medium and complex — SDD/planning.
If the agent drifts — roll back, write a plan or spec, continue from it.
If the agent is on track but doesn’t finish — TDD: fix the validation system and let the agent work until the tests go green.
How to control token consumption? Split tasks. One task, one chat. Make sync points through a plan file that carries progress — so sub-agents with small contexts see the big picture.
Which local models are good on limited hardware? There are solid options in the ~32B range with a good quality/cost balance; larger models are better but need more hardware. Very large models can be quantized, but carefully.
Which local-LLM features work in development (think, web)? Thinking works (for local models the concern isn’t token cost but server load). Web/external sources help. Inside the agent, tools like running inspections are critical.
Can analysts and testers use AI? They can and should. Testing and analytics map well onto agents, but the agent must be customized to the role — dedicated skills, corporate rules, task decomposition.
Wrapping up
For Java/Kotlin teams the throughline is simple: the agent is only as good as the context you feed it, and this stack rewards agents that understand code semantically — PSI, real type resolution, inspections, safe refactorings. Set up AGENTS.md, lock down what the agent may touch, connect the MCPs that hold your domain knowledge, and codify your conventions as rules and skills. Do that once, commit it, and every chat starts smarter than the last.