DEV Community: Jean-Luc Martel

Your AI Coding Agent Just Finished. Now Ask It to Attack Its Own Work.

Jean-Luc Martel — Fri, 21 Aug 2026 01:39:27 +0000

AI coding agents are remarkably good at getting from “here’s what I want” to “here’s a working implementation.”

They are also remarkably good at being satisfied with what they just built.

That second trait is a problem.

One of the highest-leverage habits I’ve picked up when working with coding agents is surprisingly simple:

After the agent finishes a meaningful piece of work, ask it to perform an adversarial review of its own implementation.

Not:

Review your work and make sure everything looks good.

That tends to produce a polite little victory lap.

Instead, change the objective.

Act as an adversarial reviewer. Assume this implementation contains subtle bugs, incorrect assumptions, security issues, race conditions, missing edge cases, or architectural problems. Your job is to find them. Do not defend the implementation. Try to break it.

The difference can be dramatic.

Building and attacking are different tasks

When an agent is implementing a feature, its working objective is roughly:

Find a plausible path to satisfying the requirements.

Once it has found that path, everything it sees is colored by the solution it just constructed.

Humans do this too.

You write a function, run the obvious tests, and your brain quietly becomes the function’s defense attorney.

The code looks reasonable because you know what it was supposed to do.

An adversarial review gives the model a different role:

Assume the implementation is wrong. Find the evidence.

That changes what it searches for.

Instead of asking:

Does this satisfy the happy path?
Does this compile?
Did I implement the requested feature?

it starts asking:

What happens with malformed input?
What assumption did I make that the caller never promised?
What happens under concurrency?
Is this operation actually atomic?
Can this fail halfway through?
What happens when a dependency returns something unexpected?
Did I introduce an authorization bypass?
Did I preserve existing behavior?
Is there a hidden performance cliff?
Are the tests proving the behavior, or merely exercising the implementation?

Same model. Same context. Very different search space.

The prompt I actually want

Something like this works well:

Perform an adversarial review of the implementation you just created.

Assume there are bugs.

Do not explain why the current implementation is good. Your job is to attack it.

Look specifically for:

- incorrect assumptions
- edge cases
- race conditions
- security vulnerabilities
- data corruption risks
- failure/retry problems
- backwards compatibility issues
- performance regressions
- missing validation
- incorrect error handling
- tests that pass without proving the intended behavior

For every issue you find:

1. Describe the failure mode.
2. Explain how it could occur in practice.
3. Rate its severity.
4. Point to the relevant code.
5. Propose a concrete fix.

Do not modify the code yet. First produce the review.

That last instruction matters.

I usually want the review before the repair.

If you immediately ask the agent to “find and fix any problems,” it can silently patch things while skipping the explanation. Separating diagnosis from remediation makes the reasoning inspectable.

It also lets you decide which findings are real.

Because the agent can absolutely invent problems too.

Make it argue against itself

For larger changes, I sometimes push this further and create two explicit roles.

First:

You are the implementation engineer. Complete the feature.

Then:

You are now a senior engineer reviewing this change before production deployment.

You did not write this code.

Assume the implementation engineer was competent but may have made subtle mistakes.

Try to reject this change.

The phrase “You did not write this code” is surprisingly useful.

Obviously the model did write it. We are not performing metaphysical surgery on the transformer.

But role framing affects the kind of analysis the model performs. Removing psychological ownership, even fictitiously, tends to produce a more skeptical review.

For especially important code, you can go further:

Imagine this change caused a production incident three months from now.

Work backwards and identify the most plausible ways this implementation could have caused it.

Now you’re effectively asking for a miniature pre-mortem.

That often surfaces issues a generic code review misses.

Ask for counterexamples, not opinions

One of the easiest ways to make AI review more useful is to demand concrete failure cases.

Bad:

Is this implementation robust?

Better:

Give me five concrete inputs, system states, or event sequences that could cause this implementation to behave incorrectly.

Even better:

For each suspected bug, construct the smallest reproducible scenario that would demonstrate it.

This forces the critique toward falsifiable claims.

For example, instead of:

There may be a race condition here.

you want:

Request A reads balance=100. Request B reads balance=100. Both subtract 80. Both persist 20. The system has processed $160 of withdrawals from a $100 balance.

That is something you can reason about.

And test.

Turn the findings into tests

This is where the workflow becomes particularly powerful.

After the adversarial review, ask:

For every credible issue you identified, write a regression test that fails against the current implementation.

Do not change the production code yet.

Now the loop becomes:

Implement → Attack → Reproduce → Repair → Verify

That is much stronger than:

Implement → Looks good → Ship

And it makes the agent prove its criticism.

If the supposed bug cannot be reproduced, maybe the review was wrong.

If the test fails, you now have both evidence and permanent coverage.

Different reviewers find different classes of bugs

“Review this code” is extremely underspecified.

I get better results by running multiple targeted reviews.

For example:

Security reviewer

Review this implementation as a hostile application security engineer.

Look for ways an attacker could abuse inputs, authentication, authorization, state transitions, serialization, file access, network calls, or resource consumption.

Reliability reviewer

Review this as a distributed systems reliability engineer.

Focus on partial failure, retries, duplicate execution, idempotency, ordering, timeouts, race conditions, stale state, and recovery after crashes.

API reviewer

Review this as the maintainer of clients that depend on this API.

Look for undocumented behavior changes, ambiguous contracts, backwards compatibility problems, surprising defaults, and error semantics.

Performance reviewer

Assume this works correctly at 100 requests per day but fails badly at 10 million.

Find the scaling problems.

These prompts constrain the search.

And constrained searches are often much better than asking a model to vaguely “think harder.”

There is another benefit: specification discovery

The adversarial pass does something beyond finding implementation bugs.

It often discovers that your requirements were incomplete.

Suppose the agent asks:

What should happen if two users update the object simultaneously?

Maybe you never specified that.

Or:

Is deleting this resource supposed to cascade to associated records?

Also unspecified.

Or:

Should this endpoint reveal whether an email address already exists?

Congratulations, your coding agent just wandered into a product/security decision disguised as an implementation detail.

This is one of the more useful properties of adversarial review.

It exposes the negative space around your specification.

The original implementation task asks:

What did the user tell me to build?

The adversarial task asks:

What did the user forget to tell me?

That second question can be much more valuable.

Why not just ask it to “double-check”?

Because “double-check” preserves the original frame.

The model is still trying to validate the solution.

Adversarial review changes the success criterion.

Success is no longer:

The implementation appears correct.

Success becomes:

I found a credible way this could fail.

That small prompt-engineering shift matters.

It is basically the software equivalent of red teaming.

You don't ask the red team to confirm that the defenses look sensible.

You tell them to get in.

Don't blindly accept the critique either

There is an important caveat.

AI-generated criticism is not automatically correct.

A sufficiently determined model can find imaginary bugs with impressive confidence.

So I treat adversarial findings as hypotheses.

The hierarchy is roughly:

Concrete failing test
Reproducible execution path
Clear reasoning from documented behavior
Plausible concern
Vibes wearing a security-engineer costume

The farther down that list a finding sits, the less weight I give it.

This is also why asking the agent to produce reproduction cases and tests is so useful.

It converts prose into evidence.

Make the agent inspect the diff, not just its memory

If your tool supports it, another useful instruction is:

Review the actual git diff and all directly affected code.

Do not rely on your memory of what you intended to change.

Intent is dangerous during review.

The implementation may not match the agent’s mental model of the implementation.

The diff is reality.

For significant changes, I also ask it to inspect neighboring code and call sites. Bugs frequently live at boundaries rather than inside the newly written function.

A workflow worth automating

For meaningful changes, my preferred agent loop is increasingly something like:

1. Understand the task.
2. Inspect the existing code.
3. Propose an implementation plan.
4. Implement the change.
5. Run relevant tests.
6. Perform an adversarial review.
7. Produce concrete failure cases for credible findings.
8. Add regression tests.
9. Fix confirmed issues.
10. Run the full relevant test suite.
11. Review the final diff again.

You can put this directly into an agent instruction file.

The marginal cost is tiny.

The value can be enormous.

The broader lesson

The interesting thing here isn't really the prompt.

It's that AI coding agents become more useful when we stop treating them as a single programmer with a single continuous train of thought.

They can be the implementer.

Then the reviewer.

Then the attacker.

Then the test engineer.

Then the maintainer wondering what lunatic wrote this six months ago.

Those roles optimize for different things.

And one of the cheapest ways to improve AI-generated software is to deliberately make the model disagree with the version of itself that wrote the code.

So the next time your coding agent announces:

Implementation complete. All tests pass.

Don't congratulate it yet.

Tell it to try to destroy what it just built.

How to Let gzip Find the Signal in a Pile of Documents

Jean-Luc Martel — Mon, 03 Aug 2026 23:19:53 +0000

Suppose you have a directory full of text documents.

Most are repetitive, padded with boilerplate, or otherwise low-signal. A few contain the useful material. You could read every file manually, feed them all into an embedding pipeline, or ask an LLM to rank them.

Or you could ask gzip.

The basic idea is simple:

Repetitive text compresses well. Varied text usually does not.

That makes compression ratio a crude but surprisingly useful proxy for redundancy.

It will not tell you which document is best. But it can help you identify which documents contain less repetition and deserve a closer look.

The heuristic

For each document:

Measure its original size.
Compress it individually with gzip.
Measure the compressed size.
Calculate:

compressed size / original size

A lower ratio means the document compressed well, which usually indicates more repetition.

A higher ratio means the document was harder to compress, which may indicate more varied or information-dense content.

In other words:

lower ratio: more redundant
higher ratio: less redundant

The Bash command

Here is a small Bash pipeline that ranks .txt files by compression ratio:

find ./documents -type f -name '*.txt' -print0 |
while IFS= read -r -d '' file; do
  raw=$(wc -c < "$file")
  compressed=$(gzip -n -c -- "$file" | wc -c)

  awk -v file="$file" -v raw="$raw" -v gz="$compressed" '
    raw > 0 {
      printf "%.3f\t%8d\t%8d\t%s\n", gz/raw, raw, gz, file
    }
  '
done | sort -nr

Example output:

0.642      18432      11834  ./documents/research-notes.txt
0.417      30211      12600  ./documents/project-summary.txt
0.091      27102       2467  ./documents/standard-contract.txt

The columns are:

ratio    original bytes    compressed bytes    filename

Because the output is sorted in descending order, the least compressible files appear first.

Those are the files I would inspect first when looking for the possible “gems.”

To find the most repetitive documents instead, reverse the sort:

sort -n

Why `gzip -n`?

The -n flag prevents gzip from storing the original filename and timestamp in its output.

That makes the compressed sizes more comparable across files and across runs.

Without it, a small amount of unrelated metadata can leak into the measurement.

What this is actually measuring

This technique does not measure truth, relevance, writing quality, or semantic importance.

It measures compressibility.

Those things sometimes correlate, but they are not the same.

A document full of repeated boilerplate will usually compress extremely well. A document with more distinct vocabulary, sentence structure, numbers, and ideas may compress less efficiently.

That makes the ratio useful as a first-pass ranking signal.

It is closer to a metal detector than a treasure map.

Important caveats

Small files produce noisy ratios

gzip adds headers and other fixed overhead. For tiny files, that overhead can dominate the result.

You may want to ignore documents below a minimum size:

find ./documents -type f -name '*.txt' -size +1k -print0

Already-compressed formats will mislead you

Running this directly against PDF, DOCX, ZIP, JPG, or other compressed formats mostly measures the compression characteristics of the container format.

Extract the text first.

For example, with PDFs:

pdftotext input.pdf output.txt

Incompressible does not mean valuable

Encrypted data, random identifiers, hashes, minified code, and corrupted text are all difficult to compress.

They may score highly while containing little useful information.

Repetition is not always fluff

Contracts, API documentation, technical specifications, and scientific papers may repeat terminology because precision requires it.

A lower ratio can indicate redundancy, but it can also indicate consistency.

Language and formatting matter

Compression ratios can be affected by:

document length
whitespace
markup
tables
repeated headings
source language
character encoding
templated metadata

For a fairer comparison, normalize the documents first.

For example:

tr -s '[:space:]' ' ' < input.txt

You could also strip HTML, remove headers and footers, or convert everything to lowercase before compression.

Just remember that normalization changes what you are measuring.

A slightly more useful version

For larger collections, I would filter out tiny files and print the percentage saved:

find ./documents -type f -name '*.txt' -size +1k -print0 |
while IFS= read -r -d '' file; do
  raw=$(wc -c < "$file")
  compressed=$(gzip -n -c -- "$file" | wc -c)

  awk -v file="$file" -v raw="$raw" -v gz="$compressed" '
    raw > 0 {
      ratio = gz / raw
      saved = 100 * (1 - ratio)

      printf "%6.2f%% saved\t%8d bytes\t%s\n",
             saved, raw, file
    }
  '
done | sort -n

This sorts the files with the lowest percentage saved first, meaning the least compressible documents rise to the top.

Where this could be useful

This trick can be handy for quickly triaging:

scraped web pages
exported support tickets
meeting transcripts
research notes
log samples
generated reports
document archives
large sets of Markdown files

It is especially useful when you want a fast local heuristic without setting up a database, embedding model, or external API.

Compression as a feature

The broader idea is more interesting than the Bash command.

Compression ratio can be treated as a lightweight feature in a ranking system.

You could combine it with:

document length
vocabulary diversity
duplicate paragraph counts
keyword density
entropy
embedding similarity
recency
source reputation

Compression alone is crude.

Compression plus a few other signals could become a genuinely useful document-triage tool.

Final thought

There are sophisticated ways to rank a pile of documents.

Sometimes, though, a 40-year-old compression algorithm is enough to tell you which files keep repeating themselves.

And that is often a very good place to start.