DEV Community

Cover image for Distill Coding Agent Learnings
Lars Moelleken
Lars Moelleken

Posted on • Edited on

Distill Coding Agent Learnings

Repo: https://github.com/voku/agent-loop
Demo: https://voku.github.io/agent_loop_demo/

Your Coding Agent Doesn’t Need More Memory. It Needs a Governed Loop.

Coding agents repeat mistakes.

The obvious response is to give them more memory:

MEMORY.md
project-rules.md
agent-notes.md
lessons-learned.md
MEMORY_FINAL.md

Soon the agent receives old decisions, temporary workarounds, copied transcripts, abandoned ideas, and rules nobody remembers approving.

It has more context.

It does not necessarily have better context.

At some point, memory becomes landfill.

The problem is not that coding agents forget too much.

The problem is that most workflows fail to distinguish between temporary context, evidence, proposed learning, and approved project guidance.

A transcript is not memory.
A note is not a rule.
A finding is not guidance.
And a successful patch is not automatically a project convention.

Instead of giving the agent one growing pile of context, I built "voku/agent-loop" around a governed workflow:

task
-> candidate plan
-> human approval
-> selective recall
-> implementation
-> verification
-> recorded evidence
-> reviewed learning

The important detail is that the developer does not normally operate this workflow by typing every CLI command.

The coding agent does that through related skills provided by, or adapted from, "agent-loop".

The human describes the task, reviews the proposed scope, approves or corrects it, and decides whether anything should become durable project guidance.

The agent operates the loop.

The human governs the decisions that should not be delegated.

The CLI is not the user interface

When people see a CLI project, they naturally assume the workflow looks like this:

vendor/bin/agent-loop workflow plan PROJECT-123 ...
vendor/bin/agent-loop workflow approve PROJECT-123 --by lars
vendor/bin/agent-loop verify
vendor/bin/agent-loop workflow close PROJECT-123 --status done

That is what happens underneath.

It is not how I use it day to day.

I do not manually type an approval command every time a coding agent proposes a plan.

I say:

Yep, approved.

The coding agent recognizes that explicit approval and records it through the workflow.

A normal interaction looks more like this:

Developer:
Fix PROJECT-123.

Invalid order state transitions must be rejected.
Stay inside the order-state validation and its tests.
Run PHPStan and the focused tests.
Show me the plan before changing code.

Coding agent:
I propose the following work brief:

Goal:
Reject invalid order state transitions.

Scope:
Order state validation and focused tests.

Non-goals:
Do not redesign the order aggregate.

Validation:

  • composer phpstan
  • composer test

Approve this scope?

Developer:
Yep, approved.

Coding agent:

  • records the approved revision;
  • compiles the relevant context;
  • implements the change;
  • runs the declared validation;
  • records the evidence;
  • reviews blind spots;
  • reports the result and remaining risks.

That is the actual interface.

The developer talks to the coding agent.

The coding agent operates the workflow.

The CLI provides deterministic state transitions and inspectable evidence behind the interaction.

Otherwise, we would have built a sophisticated automation system whose main achievement was teaching developers to copy longer shell commands.

Start with approved scope

A coding agent should not begin by reading a ticket and creatively filling in everything the ticket forgot to mention.

It should begin by producing an explicit work brief containing:

  • the goal;
  • permitted scope;
  • non-goals;
  • affected areas;
  • required validation;
  • unresolved assumptions;
  • required human approval.

The human can then answer naturally:

Approved.

Or:

No. Do not change the repository layer.
Keep the scope inside the service and its tests.

Or:

Approved, but the public API must remain unchanged.

The agent records the concrete approved revision internally.

When the plan changes materially, the old revision is no longer sufficient.

The agent must present the updated plan and request approval again.

An approval should apply to a concrete work brief, not permanently to a task ID whose meaning can silently change.

That may sound slightly bureaucratic until an agent performs a technically impressive refactoring nobody requested.

Skills operate the loop

"agent-loop" ships focused operational skills for coding agents working in consuming repositories:

agent-loop-task-start
agent-loop-l2-context
agent-loop-task-progress
agent-loop-review-close
agent-loop-learning-boundary

There is also a broader "agent-loop-workflow" skill that describes the complete workflow and command vocabulary.

The smaller skills are intended for actual coding-agent sessions.

A coding agent loads the skill matching its current stage instead of pulling one giant workflow document into every task.

Their responsibilities are deliberately separated.

agent-loop-task-start
-> inspect the task
-> create a candidate work brief
-> compile initial recall
-> present the scope
-> request human approval
-> record the approved revision

agent-loop-l2-context
-> select relevant project guidance
-> provide bounded context artifacts
-> distinguish supplied context from executed work

agent-loop-task-progress
-> record decisions
-> record checkpoints
-> track scope changes
-> record validation evidence
-> bind evidence to the current plan revision

agent-loop-review-close
-> review blind spots
-> verify the workflow state
-> report remaining risks
-> close only when the required gates pass

agent-loop-learning-boundary
-> capture reusable findings
-> prepare learning proposals
-> prevent automatic promotion into durable guidance

Host repositories can synchronize these skills into their coding-agent environment and adapt them to project-specific conventions.

The result is not a magical agent that understands governance by intuition.

It is an agent with explicit operating instructions and a workflow that leaves evidence behind.

Load less context, not more

The coding agent usually needs two different kinds of context:

  1. project guidance;
  2. code structure.

"agent-recall-compiler" selects task-specific guidance.

"agent-map" provides compact information about the relevant code.

These are separate because they answer separate questions.

Recall:

  • Which rules apply?
  • Which previous decisions matter?
  • What must be validated?
  • Which constraints conflict?

Map:

  • Where is this class defined?
  • Which methods and dependencies are relevant?
  • Which tests cover this area?
  • What is the smallest useful code neighbourhood?

The coding agent can use the related skills to invoke these tools itself.

The developer should not need to compile recall output manually, inspect a symbol index, copy selected Markdown into a prompt, and then tell the agent to continue.

The agent should:

  1. build or refresh the relevant map;
  2. compile task-specific recall;
  3. inspect the bounded output;
  4. load only the necessary source ranges;
  5. report missing or conflicting evidence.

The goal is not to compress the complete repository into the prompt.

The goal is to avoid loading most of it.

Dumping a full AST or several months of transcripts into the context window is not understanding.

It is bulk data transport with unusually confident autocomplete.

Keep working memory temporary

During implementation, the agent may need to track:

  • assumptions;
  • claimed files;
  • checkpoints;
  • open questions;
  • intermediate decisions;
  • validation results;
  • blocked states;
  • scope changes.

That belongs in a task session.

It should be inspectable, closable, and removable.

Temporary observations should not silently become permanent project knowledge.

A workaround that was useful for one task may become actively misleading after the code changes.

A useful system therefore needs at least three levels:

session state temporary working information
finding evidence requiring review
guidance approved durable knowledge

Collapsing those levels into one memory file removes the provenance that makes information trustworthy.

A note from a debugging session is not automatically a project rule.

A finding is not automatically guidance.

And a selected piece of context is not automatically something the agent actually used.

Verification is part of the loop

An agent saying “done” is not evidence.

The work brief already contains the expected validation commands.

The coding agent runs those commands and records the real results against the approved plan revision.

For example:

Approved scope:
Order state validation and its focused tests

Declared validation:
composer phpstan
composer test

Observed changes:
src/Order/OrderService.php
tests/Order/OrderServiceTest.php

Validation evidence:
composer phpstan -> passed
composer test -> passed

Unexpected paths:
none

The exact commands depend on the repository.

They may include:

composer phpstan
composer test
composer cs

The important rule is:

«Validation must be declared before implementation, not invented afterwards to match whatever changed.»

A passing test suite also does not prove that the agent stayed inside scope.

Tests can pass after an unnecessary architectural rewrite just as happily as after a focused bug fix.

That is why code validation and workflow verification are related but separate concerns.

The workflow must also verify:

  • whether the current plan was approved;
  • whether the implementation remained inside scope;
  • whether validation evidence belongs to the current revision;
  • whether required review steps were completed;
  • whether unresolved risk was recorded;
  • whether the task is actually ready to close.

A polished summary is not a substitute for evidence.

Coding agents are already very good at producing polished summaries.

Selected guidance is not necessarily useful guidance

Recall may select a guidance item for a task.

Selection alone proves nothing.

The guidance may turn out to be:

HELPFUL
IRRELEVANT
HARMFUL
NOT_USED
UNKNOWN

"NOT_USED" is particularly important.

A guidance item may have been selected for an investigation but never actually consulted during implementation.

Without recording that distinction, selection statistics quietly become usage statistics.

The metrics then begin describing a workflow that never happened.

The difference is simple:

selection = the system supplied it
usage = the agent consulted it
outcome = whether it helped

The coding agent can record these outcomes during review.

The developer does not need to maintain a spreadsheet documenting which Markdown fragment appeared in which generated prompt.

Recording selection, usage, and outcome separately gives us evidence for improving recall.

Otherwise, we merely count how often a file was copied somewhere.

Learning still requires a human

After the task, the coding agent may record findings.

A finding is evidence.

It is not automatically a new rule.

The "agent-loop-learning-boundary" skill helps the agent determine whether anything from the task deserves to be presented as reusable knowledge.

A finding may lead to a proposal such as:

ADD
REPLACE
DELETE
REJECT
NO_DURABLE_LEARNING

The agent may:

  • collect evidence;
  • identify repeated patterns;
  • find conflicting guidance;
  • prepare a proposal;
  • recommend an outcome.

It must not silently approve its own proposal as durable project knowledge.

A human reviews the proposal before durable guidance changes.

Most tasks should probably end with:

NO_DURABLE_LEARNING

Fixing one bug does not necessarily reveal a reusable project rule.

Sometimes the correct conclusion is simply that the bug was fixed.

That conclusion has its own terminal state:

ACKNOWLEDGED

It is neither approved as a guidance change nor rejected as incorrect.

It records that the result was reviewed and no durable change was needed.

This may look like excessive precision for one status name.

It is not.

When lifecycle states use the wrong verbs, audit trails gradually stop saying what actually happened.

Humans approve decisions, not shell commands

The intended responsibility boundary is:

Coding agent:

  • inspect the task;
  • propose scope;
  • operate agent-loop;
  • compile relevant context;
  • edit the code;
  • run validation;
  • record evidence;
  • review blind spots;
  • report risks;
  • prepare findings and learning proposals.

Human:

  • describe the task;
  • approve or correct the proposed scope;
  • decide whether expanded scope is acceptable;
  • review the resulting change;
  • accept or reject remaining risk;
  • decide whether proposed knowledge should become durable guidance.

The human communicates those decisions in normal language.

For example:

Yep, approved.

Approved, but do not touch the public API.

No, this is too broad. Keep the existing abstraction.

That additional database migration is outside the approved scope.

The remaining risk is acceptable.

That finding is task-specific. Do not promote it.

Looks good. Close it.

The coding agent converts those decisions into explicit workflow state.

Natural-language confirmation does not bypass the gates.

It triggers the agent to operate them.

Requiring the human to type the corresponding CLI command would not improve governance.

It would merely expose internal plumbing that the coding agent is already capable of operating.

What "agent-loop" actually does

"voku/agent-loop" is a PHP 8.3 CLI that coordinates several focused packages:

agent-kanban work and task state
agent-session temporary working memory
agent-recall-compiler selective project guidance
agent-map compact code intelligence
agent-learning governed durable learning
agent-loop orchestration

The CLI provides deterministic operations and inspectable artifacts.

The accompanying skills teach coding agents when and how to use those operations.

Together, they provide:

skills agent-facing workflow behaviour
CLI deterministic workflow operations
artifacts inspectable state and evidence
humans approval and governance

"agent-loop" does not replace maintainers, code review, static analysis, tests, or domain knowledge.

It gives those existing engineering practices an explicit workflow around coding agents.

Getting started

There are two separate parts:

one-time repository setup
-> install and synchronize the workflow

daily task operation
-> talk to the coding agent

Install the package

Install "agent-loop" in an existing Composer project:

composer require --dev voku/agent-loop

Scaffold the local workflow

Create the minimal local workflow structure:

vendor/bin/agent-loop init scaffold

To inspect what would be created without writing anything:

vendor/bin/agent-loop init scaffold --dry-run

The scaffold creates the initial local workflow inputs when they do not already exist:

.agent-loop/init.json
todo/board.md
todo/cards/DEMO-1.md
tasks/DEMO-1.md
session_plan/
infra/doc/agent-learning/findings/

Existing files are preserved.

"DEMO-1" is a clearly marked example task, not a special workflow mode.

You can inspect it with:

vendor/bin/agent-loop board card show DEMO-1

Synchronize the coding-agent skills

Preview the skill synchronization for Codex:

vendor/bin/agent-loop init sync-skills \
--agent=codex \
--dry-run

After reviewing the planned changes:

vendor/bin/agent-loop init sync-skills --agent=codex

These are setup and maintenance commands.

They are not the normal per-task interface.

The synchronized skills teach the coding agent how to operate the workflow in the consuming repository.

Use the workflow through normal conversation

Once the skills are available, start with an ordinary task request:

Take PROJECT-123.

Reject invalid order state transitions.
Stay inside the service and its focused tests.
Run PHPStan and the tests.
Show me the proposed scope before changing code.

The coding agent inspects the task and presents a candidate work brief:

Goal:
Reject invalid order state transitions.

Scope:
Order state validation and focused tests.

Non-goals:
Do not redesign the aggregate.
Do not change the public API.

Validation:

  • composer phpstan
  • composer test

Approve this plan?

You answer normally:

Yep, approved.

The agent then operates the workflow:

record approved revision
-> compile bounded context
-> inspect relevant code
-> implement
-> validate
-> record evidence
-> review blind spots
-> make a learning decision
-> verify
-> report

You can interrupt or redirect it in the same way:

Stop. That migration is outside the approved scope.

The agent must then revise the work brief and request approval for the expanded plan before continuing.

At the end, the agent reports:

  • what changed;
  • which validation commands ran;
  • whether they passed;
  • whether the approved scope was respected;
  • which risks remain;
  • whether any reusable learning was found.

You can respond:

Looks good. Close it.

Or:

The code is fine, but that proposed learning is too broad.
Close the task without promoting it.

The agent records and verifies those decisions through "agent-loop".

You do not manually translate them into CLI commands.

What happens under the hood

Internally, the coding-agent skills may execute operations for:

  • planning;
  • approval;
  • context compilation;
  • task progress;
  • validation evidence;
  • checkpoints;
  • blind-spot review;
  • learning decisions;
  • verification;
  • closure.

That internal command sequence remains valuable because it provides:

  • deterministic state transitions;
  • inspectable artifacts;
  • revision-bound approval;
  • validation evidence;
  • explicit close gates;
  • auditable learning decisions.

But those commands belong behind the agent-facing skill.

They are similar to database queries behind an application form.

Essential to the implementation.

Not something the user should have to execute manually.

The developer should not need to memorize the workflow commands.

The skill should.

The actual lesson

Coding agents do not need to remember everything.

They need:

  • agent-operated workflow skills;
  • explicit scope;
  • revisioned human approval;
  • selective context;
  • temporary working memory;
  • declared validation;
  • recorded execution evidence;
  • human-reviewed learning;
  • intentional forgetting.

More memory hides bad context management.

A governed loop exposes it.

The purpose is not to make developers manually operate a more elaborate CLI.

The purpose is to give coding agents a repeatable process they can operate themselves, while preserving human control over scope, risk, and durable project knowledge.

The agent does the workflow work.

The human remains responsible for the decisions that matter.

And once the workflow can reject stale approvals, irrelevant guidance, unsupported claims, and accidental lessons, the agent does not need to behave as though it remembers the entire repository.

It only needs enough verified context to complete one controlled task.

Top comments (3)

Collapse
 
kgaidev profile image
kgaidev

We build kgai, so I'm biased toward the memory side of this, but the landfill line is right. Most "agent memory" fails at capture quality, and your governed loop attacks exactly that: a finding isn't guidance until a human says so.

The part I keep chewing on is what happens after promotion. Rules don't just accumulate, they expire. The constraint that justified a rule disappears, or two devs promote contradictory guidance on the same codebase. I saw learning-boundary has REPLACE/DELETE proposals: when guidance gets replaced, do you keep the dead rule plus the reason it died, or does it just vanish? In our experience, the "why it stopped being true" is exactly what an agent needs to see to avoid re-learning the same bad rule three months later.

Also curious what triggers a REPLACE in practice. A task hitting conflicting guidance, or periodic review?

Collapse
 
suckup_de profile image
Lars Moelleken

I strongly believe that the learning loop should be integrated into every programming workflow. Even though some people might think that the next model will be better anyway, this helps with the review process because you take away prompts, tasks, findings, and so on from the programming session, which allows you to better understand the changes yourself.

Fixed that now :) <-> The decision engine that compiles recall guidance already blocked on a rejected proposal sharing a target with newly selected guidance, quoting the rejection reason. It now runs the same check against retired proposals ... if someone proposes guidance that re-targets a rule that was retired for cause, in the same task scope, compilation blocks and quotes the retirement reason back. So the "re-learning the same bad rule" case you're describing is now a hard stop, not just an audit trail someone could go read if they thought to look.

P.S.: in my case, most of the stuff that are extracted from the learnings into skills or new rules for static analysis come from semi-manual reviews or from comments like "Hint: This is a learning ..." ... and not from the tool outout itself

Collapse
 
kgaidev profile image
kgaidev

That's exactly the right semantics. A retired rule without its retirement reason is just an absence, quoting the reason back at resurrection time is the whole value. And your P.S. matches what we see with kgai too. The useful stuff comes from the conversation, not from tool output, which is what pushed us to capture in the session itself. The model records the decision as it's made, and a Stop hook at end of turn catches anything it forgot. Enjoyed this thread.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.