DEV Community

XT
XT

Posted on • Originally published at blog.neatcontext.com

AI Memory Is Not a Trust Model for Team Knowledge

#devops #git #ai #opensource

AI memory solves a real problem: every new session starts cold.

An assistant can save facts about a project, retrieve them later, and avoid asking the same questions again. That works well for personal preferences and low-risk context.

Team knowledge is different.

When remembered information starts influencing incident response, architecture decisions, or production changes, persistence is not enough. The team needs to know:

  • What was saved?
  • Who checked it?
  • Is it still current?
  • How can someone correct it?
  • Who approved the correction?

A memory database does not automatically answer those questions.

Memory can preserve a guess

Suppose an engineer is investigating an incident and tells Claude:

This is probably the connection pool again.

At that point, it is only a working theory.

An automatic memory system may still extract and store it. Three weeks later, another engineer investigates a slow endpoint in a different service. The assistant retrieves the connection pool note and presents it as relevant context.

The problem is not simply that the assistant might be wrong. Runbooks, wikis, documentation, and people are wrong sometimes too.

The difference is the admission process.

In many AI memory systems, the model decides what is worth remembering, writes it into a separate store, and retrieves it in later conversations. There may be no point where another engineer reviews the extracted statement before it starts influencing future answers.

A better extraction model can reduce mistakes. It does not create a review step.

Team knowledge should use the same trust boundary as code

Engineering teams do not trust code because it was written and stored somewhere.

They trust it because they can:

  1. Open the file.
  2. Read the change.
  3. Review the diff.
  4. Leave comments.
  5. Approve it before it reaches main.
  6. Trace its history later.

Shared domain knowledge should meet the same standard.

That means storing it in a form engineers already know how to inspect and review: files in a repository.

I am building NeatContext around this model. The examples below use NeatContext, but the important part is not the command syntax. It is the review boundary.

NeatContext writes extracted context as plain Markdown:

profile.md
knowledge/
  pool-exhaustion.md
  checkout-service.md
  monitoring.md
Enter fullscreen mode Exit fullscreen mode

Those files can go through an ordinary Git workflow:

 AI conversation
        │
        │  /neatcontext:save <name>
        ▼
 Local context
 profile.md + knowledge/*.md
        │
        │  /neatcontext:export <name> --to contexts/
        ▼
 contexts/<name>/ in the repository
        │
        │  commit, push, open PR
        ▼
 Pull request ── review ──▶ merge
        │
        ▼
 Team repository
        │
        │  pull, import, use
        ▼
 Another engineer's AI session
        │
        │  new findings
        ▼
 Another reviewed diff
Enter fullscreen mode Exit fullscreen mode

Nothing becomes shared context merely because an AI decided to remember it.

It becomes shared context after a person reviews it.

What this looks like in practice

Imagine an engineer has finished investigating checkout timeouts. The actual cause was connection pool exhaustion.

1. Save the result of the investigation

After ruling out the wrong theories and finding the cause:

You: /neatcontext:save checkout-pool-exhaustion

Claude:
Context folder:
~/.neatcontext/contexts/checkout-pool-exhaustion-a1b2c3

Profile path:
~/.neatcontext/contexts/checkout-pool-exhaustion-a1b2c3/profile.md

Knowledge folder:
~/.neatcontext/contexts/checkout-pool-exhaustion-a1b2c3/knowledge
Enter fullscreen mode Exit fullscreen mode

At this point, the context is local. It has not become team knowledge.

2. Export it into the repository 

You: /neatcontext:export checkout-pool-exhaustion --to contexts

Claude:
Exported "checkout-pool-exhaustion" to:
contexts/checkout-pool-exhaustion
Enter fullscreen mode Exit fullscreen mode

The repository might now contain:

contexts/
  checkout-pool-exhaustion/
    profile.md
    knowledge/
      pool-exhaustion.md
Enter fullscreen mode Exit fullscreen mode

The Markdown files are the proposed knowledge change.

3. Open a pull request 

git checkout -b add-checkout-pool-exhaustion-context
git add contexts/checkout-pool-exhaustion
git commit -m "Add checkout pool exhaustion context"
git push -u origin add-checkout-pool-exhaustion-context
Enter fullscreen mode Exit fullscreen mode

A reviewer can now read knowledge/pool-exhaustion.md like any other operational document.

They might notice a stale value:

default_pool_size was increased last week. Update this before merging.

The author fixes it and pushes another commit.

Only after review and approval does the context reach main.

The important part is not that Markdown is inherently more accurate than a database. It is that the team can inspect the knowledge before treating it as trusted input.

4. Another engineer uses the reviewed context

A teammate working on a later checkout issue pulls the repository and imports the context:

$ git pull

You: /neatcontext:import --from contexts/checkout-pool-exhaustion

Claude:
Imported "checkout-pool-exhaustion".

You: /neatcontext:use checkout-pool-exhaustion

Claude:
Connected to checkout-pool-exhaustion.
Enter fullscreen mode Exit fullscreen mode

They can now ask:

You:
Checkout is timing out with the same symptoms as last time.
What should I check first?

Claude:
Check pgbouncer's default_pool_size on billing-postgres first.

A previous incident with these symptoms was caused by pool
exhaustion, and the reviewed context contains the current pool
configuration.
Enter fullscreen mode Exit fullscreen mode

The answer is not more reliable because the model suddenly became better.

It is more reliable because the information supplied to the model has already passed through human review.

5. New findings become another diff

Suppose the second engineer adds a monitoring alert after resolving the incident.

They update the context:

You: /neatcontext:save checkout-pool-exhaustion

Claude:
This will update "checkout-pool-exhaustion":

- knowledge/pool-exhaustion.md
  Adds the monitoring alert created after this incident.

Confirm? (y/n)

You: y
Enter fullscreen mode Exit fullscreen mode

Then export it again:

You:
/neatcontext:export checkout-pool-exhaustion --to contexts --force

Claude:
Exported "checkout-pool-exhaustion" to:
contexts/checkout-pool-exhaustion
Enter fullscreen mode Exit fullscreen mode

The update goes through another pull request:

git checkout -b update-checkout-pool-exhaustion-context
git add contexts/checkout-pool-exhaustion
git commit -m "Document pool exhaustion monitoring alert"
git push -u origin update-checkout-pool-exhaustion-context
Enter fullscreen mode Exit fullscreen mode

The context remains a sequence of reviewable changes rather than an accumulating collection of model-written memories.

What Git adds

The useful property is not merely that the files are local.

Git provides several things a team knowledge system needs:

Question Automatic memory store Git-reviewed context
What enters the shared knowledge? Whatever the extraction process saves Changes approved through a pull request
Can engineers inspect it directly? Depends on the product Yes, as ordinary files
How is incorrect information fixed? Product-specific editing or deletion Edit the file and review the diff
Who changed it? Sometimes unclear Commit history and git blame
How does it reach the team? Often tied to an account or service Through the repository the team already uses
Can a change be reverted? Depends on the product Yes, using normal Git history

A database can technically support auditing, review, editing, and version history. The problem is not the database itself.

The problem is treating automatically extracted information as trusted team knowledge without requiring those controls.

Not every memory needs a pull request

This does not mean every remembered detail should go through code review.

Personal preferences such as formatting style, frequently used commands, or how someone likes responses structured are low-risk. Automatic memory is useful there.

The line should be drawn when remembered information becomes shared operational knowledge: system behavior, incident findings, architectural constraints, deployment rules, service ownership, or assumptions that may affect production decisions.

That knowledge should not become authoritative only because an assistant decided to save it.

It should become authoritative through the same process the team already uses for other important technical changes: a visible diff, a named reviewer, and an explicit approval.

NeatContext implements this workflow using plain Markdown and Git. The plugins are open source:

github.com/XTSoftwareLabs/neatcontext-plugins

How does your team review the knowledge that AI tools reuse across sessions?

Top comments (0)