AI-Generated Tests Can Agree With the Bug

An AI coding agent implements a feature, generates tests, runs them, and opens a pull request with a green build.

That looks like independent verification. It often is not.

If the implementation and the tests came from the same prompt, repository context, and interpretation of the requirement, they can repeat the same mistake. The code is wrong. The tests confidently prove that the wrong behavior works exactly as implemented.

The failure is not that AI cannot write tests. The failure is treating correlated output as independent evidence.

The Test Passed Because It Copied the Assumption

Imagine a billing requirement:

Apply a 20% discount to the order subtotal, then add tax to the discounted amount.

The intended calculation is:

subtotal = 100.00
discount = 20.00
taxable amount = 80.00
tax = 8.00
final total = 88.00

An agent misreads the requirement and applies the discount after tax:

export function calculateTotal(subtotal, taxRate, discountRate) {
  const tax = subtotal * taxRate;
  const discount = subtotal * discountRate;

  return subtotal + tax - discount;
}

For the simple values above, both formulas happen to return 88.00. A generated test passes:

import { describe, expect, it } from "vitest";
import { calculateTotal } from "./billing.js";

describe("calculateTotal", () => {
  it("applies a discount and tax", () => {
    expect(calculateTotal(100, 0.1, 0.2)).toBe(88);
  });
});

The test looks reasonable. It even uses a concrete expected value.

But it does not distinguish the intended formula from the incorrect one.

Now introduce a fixed discount, tax exemptions, jurisdiction-specific rounding, or items with different tax classes. The shared assumption becomes a production billing bug.

Coverage Is Not an Oracle

Code coverage answers questions such as:

• Was this line executed?
• Was this branch visited?
• Was this function called?

It does not answer:

• Was the expected value correct?
• Did the test encode the product requirement?
• Would the test fail for a plausible wrong implementation?
• Did the system preserve a business invariant?

A generated test suite can reach 100% line coverage while asserting the implementation's current behavior back at itself.

This pattern is especially weak:

const result = calculateTotal(order);

expect(result.subtotal).toBe(order.subtotal);
expect(result.tax).toBe(result.taxableAmount * order.taxRate);
expect(result.total).toBe(
  result.subtotal + result.tax - result.discount
);

The test reconstructs the same formula used by the implementation. If the formula is wrong, both sides remain wrong together.

A test needs an oracle: an independent reason to believe the expected result is correct.

Shared Context Creates Correlated Failures

When one agent writes the code and tests in the same session, both artifacts may inherit:

• the same ambiguous requirement
• the same incomplete repository context
• the same mistaken domain assumption
• the same missing edge case
• the same naming-driven interpretation
• the same outdated example

Regenerating the test in the same conversation may not add meaningful independence. The model still sees its earlier reasoning and implementation.

Even using a second model is not automatically enough. If both models receive the same ambiguous prompt and derive expectations from the same implementation, they can converge on the same attractive mistake.

The goal is not merely a different generator. The goal is a different source of truth.

The Dangerous Test Patterns

1. Asserting the implementation against itself

expect(formatPrice(input)).toBe(formatPrice(input));

This extreme example is obvious, but indirect versions are common. A test may call the same helper, parser, mapper, or formula as the production path.

2. Snapshotting unknown behavior

expect(calculateInvoice(order)).toMatchSnapshot();

The first snapshot records whatever the implementation produced. Approving it proves that someone accepted the output, not that the output matches the requirement.

Snapshots are useful for detecting change. They are weak at establishing initial correctness when nobody verifies their meaning.

3. Mirroring private implementation details

expect(cache.internalEntries.size).toBe(1);

The test confirms structure rather than observable behavior. It may pass while users receive stale data, and fail during a safe refactor.

4. Testing only the example from the prompt

If an issue contains one example, an agent will often reproduce it as one happy-path test. That example may not distinguish competing interpretations.

5. Mocking away the failure

paymentGateway.charge = vi.fn().mockResolvedValue({ status: "paid" });

The test proves behavior for a dependency that always succeeds instantly. It says nothing about timeouts, duplicated responses, uncertain outcomes, or retries after the remote side processed the charge.

A quick review question For every generated assertion, ask: > Where did this expected value come from? If the answer is "from the implementation," "from the agent's earlier output," or "from a snapshot we did not inspect," the test is not independent enough for a critical behavior.

Start With Invariants, Not Examples

Examples verify selected points. Invariants describe properties that must hold across many inputs.

For a discount calculation, useful invariants might include:

• a discount cannot increase the pre-tax amount
• a 0% discount changes nothing
• a 100% discount produces no taxable amount
• the total cannot be negative
• changing item order cannot change the total
• rounding happens at the boundary defined by the business rules

Property-based tests can exercise these rules across generated inputs:

import fc from "fast-check";
import { expect, it } from "vitest";
import { calculateTotal } from "./billing.js";

it("never increases the total when the discount increases", () => {
  fc.assert(
    fc.property(
      fc.double({ min: 0, max: 100_000, noNaN: true }),
      fc.double({ min: 0, max: 0.3, noNaN: true }),
      fc.double({ min: 0, max: 1, noNaN: true }),
      fc.double({ min: 0, max: 1, noNaN: true }),
      (subtotal, taxRate, firstDiscount, secondDiscount) => {
        const lower = Math.min(firstDiscount, secondDiscount);
        const higher = Math.max(firstDiscount, secondDiscount);

        const totalWithLowerDiscount = calculateTotal(
          subtotal,
          taxRate,
          lower
        );

        const totalWithHigherDiscount = calculateTotal(
          subtotal,
          taxRate,
          higher
        );

        expect(totalWithHigherDiscount).toBeLessThanOrEqual(
          totalWithLowerDiscount
        );
      }
    )
  );
});

This still does not prove every domain rule. It does make it harder for one convenient example to hide a broken relationship.

Derive Acceptance Tests From a Separate Source

The strongest acceptance tests should come from artifacts that existed independently of the implementation:

• product acceptance criteria
• API contracts
• regulatory rules
• protocol specifications
• previously verified behavior
• production examples approved by domain experts
• compatibility fixtures from another implementation

For the billing example, create a decision table before asking an agent to implement it:

Case: percentage discount before tax
Subtotal: 125.00
Discount: 20%
Tax rate: 8%
Expected discount: 25.00
Expected taxable amount: 100.00
Expected tax: 8.00
Expected total: 108.00

Then convert that independently reviewed fixture into a test:

it("taxes the amount after the percentage discount", () => {
  expect(calculateTotal(125, 0.08, 0.2)).toBe(108);
});

The unusual values are deliberate. They distinguish formulas that coincidentally agree for simpler inputs.

Make the Test Fight the Implementation

Mutation testing changes the code in small ways and checks whether the tests fail.

For example, a mutation tool might replace:

const taxableAmount = subtotal - discount;

with:

const taxableAmount = subtotal + discount;

or change:

if (total >= freeShippingThreshold)

into:

if (total > freeShippingThreshold)

If the test suite stays green, it did not protect that behavior.

A mutation score is not a perfect quality metric. Equivalent mutations and costly test runs are real constraints. But targeted mutation testing is valuable for authorization, billing, retries, idempotency, limits, and other logic where a plausible one-character error matters.

Meta reported a related change-aware approach in its 2026 work on Just-in-Time catching tests. Instead of only generating tests that pass on the proposed code, catching tests are designed to surface regressions before a change lands. In an evaluation of 22,126 generated tests, change-aware methods improved candidate catch generation, and the reported process found confirmed serious failures. The important idea is practical: generated tests should actively try to falsify the change, not merely document it.

Use Differential Testing When You Have Two Implementations

Differential testing runs the same inputs through two independent implementations and compares the outputs.

This is useful during rewrites, migrations, and optimizations:

for (const order of generatedOrders) {
  const legacyResult = legacyCalculator(order);
  const newResult = newCalculator(order);

  expect(newResult).toEqual(legacyResult);
}

The reference implementation does not need to be fast or elegant. It needs to be trusted for the behavior under comparison.

Be careful: if the legacy behavior contains known bugs, encode those exceptions explicitly. Otherwise, differential testing can preserve defects just as snapshots do.

Separate the Roles in the Workflow

A safer AI-assisted workflow creates friction between implementation and validation.

Step 1: Extract the contract

Before generating code, write down:

• inputs and outputs
• invariants
• boundary conditions
• error behavior
• security constraints
• examples that distinguish plausible interpretations

Step 2: Generate the implementation

Give the coding agent the contract, but do not let its implementation become the source of expected values.

Step 3: Generate adversarial tests in a fresh context

Provide another agent or session with:

• the contract
• the code diff
• public interfaces
• known failure classes

Ask it to find counterexamples, not to improve coverage.

A useful prompt is:

Act as an adversarial test engineer.

Given the requirement and code diff:
1. List plausible incorrect interpretations.
2. Create inputs that distinguish them.
3. Identify boundary conditions and invariants.
4. Write tests that would fail for each plausible wrong implementation.
5. Do not derive expected values from the implementation under test.

Step 4: Run targeted mutation testing

Focus on critical decisions rather than mutating the entire repository on every commit.

Step 5: Review the oracle

Reviewers should spend less time asking whether the test is readable and more time asking:

• Why is this expected value correct?
• Which wrong implementation would this test reject?
• Is the test independent of production helpers?
• What happens at the exact boundary?
• Which requirement remains untested?

Step 6: Keep merge controls

A green agent-generated pull request still needs repository protections, required checks, and human review. GitHub explicitly requires human review before merging draft pull requests produced by its coding agent and layers security scanning and branch restrictions around generated changes.

That is not ceremony. It is recognition that generated code plus generated tests is evidence, not authority.

A Practical Review Example

Suppose an agent adds this rate limiter:

export function isAllowed(requestCount, limit) {
  return requestCount <= limit;
}

It also generates:

it("allows requests within the limit", () => {
  expect(isAllowed(3, 5)).toBe(true);
});

it("blocks requests above the limit", () => {
  expect(isAllowed(6, 5)).toBe(false);
});

Both tests pass. Coverage may be complete.

But the critical question is whether the fifth request is allowed or whether limit represents the number of requests already consumed. The tests avoid the exact boundary where the interpretation matters.

Add the distinguishing case:

it("defines behavior at the exact limit", () => {
  expect(isAllowed(5, 5)).toBe(false);
});

Whether false is correct depends on the contract. The key is that a human or independent specification must decide it. The implementation cannot decide its own requirement.

Actionable Checklist

Before trusting AI-generated tests, verify that:

• [ ] Expected values come from a specification, fixture, reference implementation, or reviewed calculation.
• [ ] Tests do not call production helpers to calculate their own expectations.
• [ ] Boundary values are tested explicitly.
• [ ] At least one case distinguishes plausible interpretations of the requirement.
• [ ] Important behavior is expressed as an invariant where possible.
• [ ] Snapshots were inspected for meaning before approval.
• [ ] Mocks include failures, delays, retries, and uncertain outcomes.
• [ ] Generated tests attempt to falsify the change, not only confirm it.
• [ ] Critical logic receives targeted mutation testing.
• [ ] Rewrites are compared against a trusted implementation where appropriate.
• [ ] A reviewer examines the test oracle, not only coverage.
• [ ] Agent-created pull requests remain subject to required checks and human review.

CTA: Pick one AI-generated test from your current project and trace every expected value back to its source. If the trail ends at the implementation itself, replace that assertion with an independent fixture, invariant, or boundary case.

The Takeaway

AI-generated tests are useful. They can produce scaffolding, cover routine branches, explore edge cases, and shorten feedback loops.

But tests generated from the same context as the implementation are not automatically independent validation.

They may encode the same misunderstanding twice:

incorrect requirement interpretation
              ↓
      incorrect implementation
              ↓
     matching incorrect tests
              ↓
          green build

Do not measure confidence only by the number of generated tests or the coverage percentage.

Increase independence instead:

• derive expectations from reviewed contracts
• test invariants and exact boundaries
• use fresh adversarial context
• apply mutation and differential testing
• review where each oracle came from
• keep humans responsible for the merge decision

A test suite is valuable when it can disagree with the implementation.

What is the most convincing green test suite you have seen that still allowed a production bug through?

---

Sources

• The Death of Traditional Testing: Agentic Development Broke a 50-Year-Old Field, JiTTesting Can Revive It
• Just-in-Time Catching Test Generation at Meta
• Risks and mitigations for GitHub Copilot cloud agent

---

Connect with Me

If you found this article helpful, let's connect!

• 💻 GitHub: johnnylemonny

𝗝𝗼𝗵𝗻Follow

Freelance fullstack developer creating fast, modern, and refined applications. I focus on simplicity, performance, and real value for users. I also write tech-related articles here.

Comments

[nyaomaru]

Great article, John! 😸
I really liked the point that AI-generated code and tests can share the same misunderstanding, so a green build isn’t necessarily independent evidence.

The idea of using a different source of truth, like contracts, invariants, boundary cases, or a fresh adversarial context, feels much stronger than simply asking another model to generate more tests.

I especially liked the focus on asking where each expected value actually came from. That feels like the real test of whether the suite can challenge the implementation rather than just agree with it. 🚀

[Reid Marlow]

The leak I kept hitting when splitting the workflow into an adversarial test agent was giving that second agent access to the pull request diff.

Even when the tester prompt says to act as an adversarial tester and find bugs in this change, handing it the diff immediately primes the model on the implementer's variable names, error codes, and edge-case boundaries. It ends up writing tests tailored to the shape of the patch rather than the shape of the domain requirement.

The setup that actually caught mismatches for me was keeping the diff completely out of the tester's context. The tester gets only the issue description and the repo at the merge-base commit. It has to write a test that fails on main first, proving that the test is testing the missing requirement, before the candidate patch is merged to run against it. If the test cannot fail against the pre-patch commit, the test is usually either vacuous or asserting an internal helper that did not exist yet.