I Built AgentCheck Because “The Coding Agent Said Done” Wasn’t Enough

emre ordu — Sun, 23 Aug 2026 21:08:42 +0000

I Built AgentCheck Because “The Coding Agent Said Done” Wasn’t Enough

AI coding agents are getting surprisingly good at writing code.

I use them regularly, and they can handle increasingly large tasks: refactoring code, adding features, updating dependencies, modifying configuration, creating migrations, and touching files across an entire repository.

But I kept running into the same problem after the agent finished:

How do I independently verify what it actually changed?

The agent usually gives me a perfectly reasonable summary.

Something like:

Done.

Implemented the requested changes, updated the tests, and cleaned up the affected code.

Useful?

Absolutely.

Enough for me to commit without checking?

Not really.

So I built AgentCheck.

The Problem Happens After “Done”

After a coding agent finishes a task, I still find myself manually checking things like:

Which files actually changed?
Were any files deleted?
Did configuration change?
Were dependencies added or updated?
Was a database migration introduced?
Did anything that looks like a secret appear?
Were related tests changed?
Is the overall change set larger or riskier than expected?

Of course, Git already gives us the raw information.

I can run:

git status
git diff
git diff --stat

Then inspect individual files.

And I still do that.

But once coding agents become part of your normal workflow, repeating the same verification process after every task starts to feel like something that should be structured.

That was the idea behind AgentCheck.

What AgentCheck Does

AgentCheck creates a trusted checkpoint before your coding agent starts working.

Then, after the agent finishes, it compares the current Git-visible repository state with that checkpoint.

The basic workflow is deliberately small:

agentcheck start

Then let your coding agent work.

That can be:

Codex
Claude Code
Cursor
another AI-assisted coding tool
or technically even a human

When the work is finished:

agentcheck

AgentCheck then produces four sections:

Changes
Findings
Risk
Verdict

For example:

AgentCheck

Changes
────────────────────────────
2 modified
1 created
0 deleted
0 renamed

A  packages/example/new-file.ts
M  package.json
M  src/example.ts

Findings
────────────────────────────
⚠ Dependency change detected.

Risk
────────────────────────────
Score: 3 — MEDIUM

Verdict
────────────────────────────
REVIEW RECOMMENDED

The important part is that this result comes from the repository state itself — not from the coding agent's explanation of what it believes it changed.

Why Not Just Ask Another LLM to Review It?

This was one of the main design decisions.

There are already many AI code-review tools, and some of them are very capable.

But that wasn't the problem I wanted AgentCheck to solve.

If one LLM changes my repository, I didn't necessarily want the verification layer to be:

LLM changes code
      ↓
another LLM reviews the first LLM

I wanted a smaller and more predictable layer:

Coding agent
      ↓
Actual Git-visible changes
      ↓
Deterministic checks
      ↓
Human review
      ↓
Commit

So AgentCheck does not use an LLM for its analysis.

The checks are deterministic.

Given the same repository state, AgentCheck should produce the same result.

What Does It Currently Look For?

The first public version intentionally keeps the scope limited.

AgentCheck can currently highlight things such as:

database migration changes
production configuration changes
dependency changes
deleted files
CI/CD-related changes
unusually large change sets
possible secrets
situations where production code changed but related tests do not appear to have changed

These signals feed into a transparent risk score and a restrained verdict.

For example:

0–2   → LOW
3–6   → MEDIUM
7+    → HIGH

The goal is not to say:

This code is correct.

AgentCheck cannot know that.

The goal is closer to:

These are the parts of this change set that probably deserve your attention before you commit.

A Checkpoint Without Touching Your Real Git Index

One technical requirement was particularly important to me:

AgentCheck should not modify the developer's actual Git index, working tree, or history.

The checkpoint implementation uses Git's tree/index model with a temporary alternate index.

Conceptually:

Current repository state
        ↓
temporary Git index
        ↓
git write-tree
        ↓
checkpoint tree

Later, AgentCheck creates another representation of the current state and compares:

checkpoint tree
        ↓
       diff
        ↑
current tree

This means the developer can already have:

staged changes
unstaged changes
tracked deletions
non-ignored untracked files

when the checkpoint is created.

Those pre-existing changes become part of the baseline rather than being incorrectly attributed to the coding agent.

The real Git index remains untouched.

Local-First Was Another Requirement

AgentCheck currently has:

no backend
no account
no source-code upload
no telemetry
no LLM API requirement

The verification happens locally.

That also keeps the workflow simple:

npm install -g @agentcheck/cli

agentcheck start

# coding agent works

agentcheck

There is also a VS Code extension if you prefer reviewing the result inside the editor.

Coding-Agent Independent

I developed AgentCheck primarily using Codex, but I intentionally avoided coupling AgentCheck to any specific coding-agent product.

It doesn't need to understand the agent session.

It doesn't need an agent plugin.

It doesn't need the agent to tell AgentCheck when it is finished.

AgentCheck only cares about the resulting repository changes.

So the same workflow can sit after:

Claude Code
Codex
Cursor
another coding agent

That separation is important to me.

Coding agents will change.

The Git repository remains the source of truth.

CLI and VS Code

AgentCheck is currently available as both a CLI and a VS Code extension.

CLI

Install:

npm install -g @agentcheck/cli

Then:

agentcheck start

and later:

agentcheck

VS Code

The extension exposes the same review model inside VS Code:

CHANGES
FINDINGS
RISK
VERDICT

The VS Code extension is intentionally a thin UI over the same deterministic core rather than a separate analysis engine.

Open Source

AgentCheck is open source under the Apache License 2.0.

GitHub:

https://github.com/emreordu/agentcheck

npm CLI:

https://www.npmjs.com/package/@agentcheck/cli

npm Core:

https://www.npmjs.com/package/@agentcheck/core

VS Code Marketplace:

https://marketplace.visualstudio.com/items?itemName=agentcheck.agentcheck-vscode

What I’m Thinking About for v0.2

The first release was mainly about proving the checkpoint and deterministic verification model.

For the next version, I'm currently exploring things such as:

a stable, versioned review-report model
machine-readable JSON output
more actionable deterministic findings
detailed added/removed/updated dependency information
better finding-to-diff navigation inside VS Code

One thing I'm deliberately trying to avoid is turning AgentCheck into a giant AI code-review platform.

I want the core idea to remain simple:

Independent verification of what actually changed.

I’d Love Feedback From People Using Coding Agents

AgentCheck is still early.

The most useful feedback for me right now isn't:

Add more features.

It's things like:

What do you still manually check after your coding agent says “done”?
Did AgentCheck miss something you expected it to highlight?
Did it produce a noisy finding?
Is the checkpoint workflow awkward in practice?
What deterministic signal would actually help you during review?
Would you use this mainly from the CLI or VS Code?

If you use Claude Code, Codex, Cursor, or another coding agent in real repositories, I'd love to hear how this approach fits into your workflow.

Try it.

Break it.

Tell me what it gets wrong.

Don’t trust “done”. Verify the result.

https://github.com/emreordu/agentcheck

DEV Community: emre ordu

I Built AgentCheck Because “The Coding Agent Said Done” Wasn’t Enough

I Built AgentCheck Because “The Coding Agent Said Done” Wasn’t Enough

The Problem Happens After “Done”

What AgentCheck Does

Why Not Just Ask Another LLM to Review It?

What Does It Currently Look For?

A Checkpoint Without Touching Your Real Git Index

Local-First Was Another Requirement

Coding-Agent Independent

CLI and VS Code

CLI

VS Code

Open Source

What I’m Thinking About for v0.2

I’d Love Feedback From People Using Coding Agents