DEV Community

Cover image for Self improving harness - Part 1: Team intelligence
Emre Ertugrul
Emre Ertugrul

Posted on

Self improving harness - Part 1: Team intelligence

In my previous post in this series, I talked about agents, workflows, execution and intelligence - the building blocks of a plug and play agentic workflow harness that handles software delivery lifecycle solo or in a team environment.

While considering the intelligence part of the harness that had to do with the md files - agents, workflows, skills, knowledge base etc, it was obvious that the harness would benefit tremendously from some form of self improvement mechanism. In the simplest terms we could tell the agents: If you find something funny while working and know a workaround - record it. Next round, read this knowledge so you don't fall into the same trap.

This mechanism soon proved to be essential for any kind of work the harness automation was going to handle. I monitored that quite often some tool would fail, or the agent didn't actually know how to use it. Other times there would be sandbox quirks and quite occasionally some project had some peculiarity which wasn't documented anywhere inside the repository. So without this mechanism the agents would constantly fall into the same traps on each new job, spending valuable time and tokens.

Before we dive into the self improvement mechanism, let's first understand the knowledge (intelligence) design of Coro.

Shared Intelligence

Knowledge sharing is key in a software team. In fact, a team can only be as effective as their communication is, between the team members. That is the reason for all the daily stand-ups, huddles, retrospectives that we humans do. We need to share information with each other because software is not something you develop one-off and never look back, no - we continuously iterate. We iterate over and over again, sometimes to fix bugs, sometimes to improve or add things. Imagine being handed a repository where you had to run a special build command for it to bootstrap everything, and no one told you about it. You would spend countless hours trying to figure it out and worse, break the code along the way - just because of a few lines of information that were omitted and not documented, not communicated.

AI Agent's don't do daily stand-ups or weekly retrospectives.

Knowledge sharing

That's exactly what's happening in todays world of AI software development. You boot up claude code and start smashing some prompts to get something done and whatever quirks you face along the way gets resolved sometimes between the agent turns but never surfaced for other team members to know.

No other AI or human knows about the quirks your agents face while they are working on your code.

And it has more dimensions. Not documenting special cases and workarounds inside the repository could cost countless tokens, roundtrips and waste of developer and AI resources.

How many teams pay attention to the good practice of documenting each piece of knowledge for others to see?

Coro does this by default.

There layers of Intelligence

Being a plug and play harness, Coro's intelligence has 3 layers: Base, Team, Repository . For each job these 3 layers are curated stacked above each other in a last wins basis and provided as 1 intelligence stack for the agents to consume.

Intelligence layers

Base intelligence

The base layer ships with Coro. It contains the generic agents, workflows,

skills, and memory templates that every installation starts from. These are building blocks of Coro's software delivery lifecycle intelligence. They are read-only. You can start building software in an opinionated workflow fresh off the installation with the base intelligence.

Team intelligence

But face it, each team probably has their own way of working. Team A might have a different workflow or agent description than Team B and that's totally fine with Coro. You can bring your own agents, workflows, skills or knowledge base to Coro and they are handled as a layer above Coro's base intelligence. The way Coro wants you to provide this is a way you are already used: specify a shared team repository that stores all the team intelligence files in source control.

Storing knowledge inside a source control repository gives ability for the team members to review, modify and approve what knowledge update actually becomes permanent, much like a code file.

Team knowledge examples include:

  • an internal package registry needs a specific configuration;

  • the team always uses a particular deployment check;

  • a shared sandbox or CI environment has a known limitation;

  • a language convention applies across the company's services;

  • an internal reviewer or infrastructure workflow has a reusable rule.

Team knowledge can update:

  • memory/;

  • .claude/skills/;

  • agents/;

  • workflows/;

  • .claude/CLAUDE.md.

Repository intelligence

Besides the team intelligence, you often have projects that are built that certain (sometimes peculiar) way, and it doesn't make sense to store project specific knowledge inside a team knowledge. So Coro stores and reads project based knowledge inside the project repository itself under .coro/ folder in each repository.

It is for facts that are true only for that codebase. Examples may include:

  • tests require POSTGRES_URL;

  • the repository uses make verify instead of the usual command;

  • a service has a non-standard directory layout;

  • one deployment needs a project-specific flag;

  • this repository has a local agent or workflow override.

The rule is simple:

If another repository on the same team should learn it, use the team

layer. If only this project should learn it, use the repository layer.

While storing the intelligence, human user may also choose the layer explicitly. Otherwise, the Evaluator picks up the best path:

  • .coro/... routes to the repository;

  • shared intelligence paths route to the team.

Let's see how Coro learns.

Learning starts while the job is running

Each agent in Coro is instructed to make a note of anything "funny" they encounter while they are working and record it in a collection called "insights". Insights are first class citizens that are attached to the Job object, so that they are accumulated by the agents while travelling along the workflow path with the job itself.

A special agent that runs as the last step of the workflow called the Evaluator agent handles the accumulated insights and turns them into knowledge.

Insights turn into knowledge

This avoids three common problems:

  • losing a useful lesson before the end of the job;

  • opening several PRs for closely related insights;

  • turning every temporary workaround into permanent guidance.

Example Insight
Category: toolchain-pitfall
Summary: Integration tests need POSTGRES_URL
Detail: Tests fail during setup when the variable is missing.
Suggestion: Document the required local test environment.
Suggested layer: repo
Enter fullscreen mode Exit fullscreen mode

Evaluator runs at the end of each workflow and first verifies the job's completeness, then reviews and verifies all of the collected insights on that job, ensuring that they are not repeated and are valid. Each insight is also assigned a layer: team or repository.

  • Team insights are PR'd into the shared team repository.

  • Repository specific insights are PR'd into the repository itself.

PR's are now the responsibility of the team. Team members (us, humans!) need to review what intelligence update Coro intends to do. From here onwards, it's just a software update. We need to check it, verify it and merge it. What we merge becomes permanent.

What we get with this design

Per-job self-improvement can easily become noisy after a few runs. If every workaround becomes permanent, memory fills with one-off accidents. If agents rewrite instructions directly, one bad observation can affect every future job. If all knowledge goes into one global file, project-specific facts leak into unrelated repositories or vice versa

Coro avoids that by separating the stages:

  • every agent can observe;

  • the human can curate;

  • the Evaluator verifies and grooms;

  • routing separates team knowledge from repository knowledge;

  • the runner validates and opens a reviewable PR;

  • humans decide what becomes canonical.

Per-job self-improvement asks a small but useful question:

“What did this job learn that the next job should not have to rediscover?”

That is how Coro's knowledge base grows one reviewed lesson at a time.

But what about the code?

That's where it starts getting more interesting. Coro is an open source project - so why can't Coro become a contributer to Coro? Turns out, if done right - it can. Let's talk about that in the next post.

Top comments (0)