DEV Community

Neil Ermak
Neil Ermak

Posted on

My AI agents were choking on their own markdown logs. I gave them a database.

#ai #programming #productivity #opensource

I am an AppSec engineer. For the past year I have been running AI agents as part of my security work: recon, attack surface mapping, hypothesis testing. It worked well until it did not.

The problem

Agents produce logs at every step: what they found, what they tried, what failed. All of that was landing in markdown files. After a few weeks on one project the folder looked like this:

recon.md
subdomains-live.md
endpoints.md
attempts-auth.md
notes-session.md
TODO.md
TODO-v2.md
progress-subagent-3.md
Enter fullscreen mode Exit fullscreen mode

I was using a model with a 256k token context window. That sounds like a lot. But at some point the accumulated logs for a single project stopped fitting in the window at all. The agent had to read the entire archive just to remember where it left off, and there was no room left to do actual work. It would hit the limit and restart, losing intermediate results and repeating attempts that had already failed.

Specific symptoms:

  • Compaction erased progress. After context compression the agent would re-read progress.md, see "invoices module - almost done" and start over. What "almost" meant was anyone's guess.
  • Failed attempts repeated. Some hypotheses the agent tried three times, each time as if it were the first.
  • Subagents did not know about the lead agent's decisions. Scope constraints lived in the main agent's context. The subagent had no idea.
  • "What is left?" had no quick answer. You had to read every file and reconstruct the picture yourself.

A markdown file is good as a document for a human to read. As a state store for a running agent it is not: no structure, no statuses that mean the same thing consistently, and you cannot query it.

What was actually needed

I wrote down the requirements:

  1. Restore full context with the minimum number of tokens - at the start of a new session or when spawning a subagent.
  2. "What is left?" must be one query, not a history read.
  3. A failed attempt gets recorded and found before the agent repeats it.
  4. A decision made at the top of the hierarchy reaches every subagent below without retelling the whole conversation.
  5. Finished work collapses into a summary - the next session reads that, not a hundred notes.
  6. Everything local, no external services. Project data stays on the machine.

I did not find a tool that fit. So I built one: PentaCore, an MCP server in Rust. One binary, one encrypted SQLite file.

It does not plan anything or decide what to do next. It stores what it is given and returns what is asked for.

How it works

PentaCore has two stores.

Notes are the narrative: goals, steps, attempts and their outcomes, facts, decisions, questions, lessons. Notes form a tree through parent and a graph through typed links. Search works by exact words and, optionally, by meaning: the first recall downloads a small embedding model (all-MiniLM-L6-v2, English only, about 90 MB). Without it, search by words still works.

Entities are many things of one kind: hosts, endpoints, files, whatever. Each entity has a status, attributes, and named checks.

The split matters. "Show everything that has not passed check X yet" is a field filter. Its result should not depend on how the agent phrased a note.

The pyramid: goals, steps, checklists

Top level is the goal, steps under it, each step has a checklist:

{"name": "note", "arguments": {"kind": "goal", "title": "Audit api.example.com"}}
{"name": "note", "arguments": {"kind": "step", "title": "Module: invoices", "parent": 1}}
{"name": "checklist", "arguments": {"note": 2, "add": [
  "BOLA on read",
  "BOLA on update",
  "Mass assignment",
  "Role bypass"
]}}
Enter fullscreen mode Exit fullscreen mode

Each endpoint is an entity:

{"name": "upsert_entity", "arguments": {
  "type": "endpoint",
  "key": "GET /v2/invoices/{id}",
  "status": "discovered",
  "attrs": {"module": "invoices", "auth": "user"}
}}
Enter fullscreen mode Exit fullscreen mode

Checks are marked as they are done:

{"name": "mark_check", "arguments": {"id": 41, "checks": [
  {"name": "bola_read", "result": "fail", "detail": "user B reads invoices of user A"},
  {"name": "role_bypass", "result": "pass"}
]}}
Enter fullscreen mode Exit fullscreen mode

Now "what has not been checked yet" is one call:

{"name": "query_entities", "arguments": {
  "type": "endpoint",
  "where": [{"field": "check.bola_read", "op": "is_null"}]
}}
Enter fullscreen mode Exit fullscreen mode

And the overall picture is another:

{"name": "query_entities", "arguments": {"type": "endpoint", "group_by": "check.bola_read"}}
Enter fullscreen mode Exit fullscreen mode

Restoring context after compaction

At the start of a new session, one call:

{"name": "resume", "arguments": {"task": 1}}
Enter fullscreen mode Exit fullscreen mode

The agent gets what it needs to continue that task, not the whole archive. For a subtask this includes the notes handed down to it from above. The reply fits within a size budget (budget_chars), and budget.omitted says how much did not fit, so truncation is never silent.

Subagents get only the context they need

The lead agent creates a step for the subagent and links relevant decisions via uses:

{"name": "note", "arguments": {"kind": "decision", "title": "/internal/* is out of scope"}}
{"name": "link", "arguments": {"src": 1, "dst": 3, "kind": "uses"}}
{"name": "note", "arguments": {
  "kind": "step",
  "title": "Zone: b.example.com",
  "parent": 1,
  "body": "Check TLS and auth on b.example.com."
}}
Enter fullscreen mode Exit fullscreen mode

Here the decision is note 3 and the new step is note 4. A note that a task uses is context for that task and for every task beneath it, so a decision linked from the goal reaches subagents at any depth. The subagent is given only the step id. It calls resume task=4 and gets its assignment and the shared decision, plus any findings you linked to the step with uses. No need to retell the lead agent's whole conversation.

Summaries instead of accumulated history

When a step is done:

{"name": "summarize", "arguments": {
  "task": 2,
  "work": "48 endpoints checked: BOLA, mass assignment, role bypass.",
  "result": "BOLA on GET /v2/invoices/{id}: user B reads invoices of user A. Mass assignment on PATCH via status field."
}}
Enter fullscreen mode Exit fullscreen mode

task_summary returns the whole task as its summaries in tree order - one call, however many notes sit below. Any agent at any level of the hierarchy can get a focused summary of one area of the whole perimeter.

Security

Memory is a prompt injection channel. Everything written to it is later read as context. An unauthorized write becomes an injected instruction.

What follows from that:

  • stdio only by default. The server can only be called by the process that launched it.
  • HTTP cannot be accidentally exposed. Loopback only, refuses to start without a token of at least 32 characters, requires an exact Host header to block DNS rebinding, rejects any request with an Origin header.
  • Input validated at the boundary. Every argument is parsed into a type with length and character set constraints. Unknown fields are rejected.
  • User text never becomes SQL. Bound parameters only. FTS5 operators are escaped. Entity queries are assembled from an allowlist of fields and operators.
  • Data at rest is encrypted. SQLCipher, secure_delete, WAL cleared after purge.

Honest about limits: the SQLCipher key is compiled into the binary. It protects the file from someone who only has the file. It does not protect against someone who has the binary or the source. All known limits are listed in the README.

When PentaCore is not the right tool

  • Task tracker with dependencies for software development: Beads or built-in Tasks in Claude Code.
  • One short task in a single session: the model context is enough.
  • Long-term memory about a user or their preferences: different problem, different tools.

PentaCore is for when an agent works through a large number of similar targets over multiple sessions, survives compaction, and needs an exact answer to "what is done, what was found, what is left."

Try it

Binaries for Linux, macOS, and Windows are in Releases. SHA256SUMS, SBOM, and a build provenance attestation are included.

Claude Code:

claude mcp add pentacore -- /path/to/pentacore
Enter fullscreen mode Exit fullscreen mode

Any other MCP client:

{
  "mcpServers": {
    "pentacore": {
      "command": "/path/to/pentacore",
      "env": { "PENTACORE_PROJECT": "my-project" }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

The repo also has a Dockerfile and docker-compose.yml: read-only root filesystem, all capabilities dropped, unprivileged user. You build the image yourself; there is no published image yet.

Repo: https://github.com/DroBov1Tya/PentaCore

If you find a vulnerability, use Security -> Report a vulnerability in the repo rather than opening a public issue.

Top comments (0)