An agent adds a feature, generates a pile of unit tests, and gets the coverage number over the threshold. Everything is green. Then the application fails when it talks to the database or another service.
In my experience, too many agent-generated tests seem to exist to satisfy that threshold. They exercise the implementation, repeat similar cases, and make the diff larger. Reviewing them can take more thought than generating them did.
I still value unit tests for meaningful logic. For application code that mainly coordinates databases and APIs, I increasingly want the strongest checks around those interactions: local integration tests with real dependencies where practical, controlled HTTP simulations for failure cases, and a small set of end-to-end checks against a deployed environment.
That also means pruning the test suite. Every generated test should earn its place by protecting a behavior, exposing a distinct failure, or making failures easier to diagnose.
Other developers are seeing the review problem
A December 2025 ExperiencedDevs discussion describes AI-generated test suites becoming a substantial part of pull requests and difficult to review. Responses mention duplication, implementation-coupled assertions and tests that simply preserve what the code already does. Others welcome the additional tests and suggest improving their structure. These are practitioner reports, not evidence that every agent produces poor tests.
Simon Willison’s January 2026 advice is more optimistic: give agents a good existing test suite to imitate. He also identifies duplicated setup as a recurring problem and recommends fixtures and parameterized cases where appropriate.
The disagreement about test balance predates agents. Kent C. Dodds argues for putting much of the effort into integration tests, particularly in the JavaScript application context. Ham Vocke’s Practical Test Pyramid emphasizes fast lower-level checks and removing higher-level tests that add no extra confidence. Both positions make us justify the value of a test rather than count it.
My takeaway is to make that judgment explicit in review. A unit-test quota and an integration-test quota can both create busywork.
What useful unit tests still give me
Consider a reservation that expires at a particular instant. The requirement says it is invalid at the expiry time, not only after it.
A small test can check the moment before expiry, the exact boundary and the moment after. Pass a controlled clock into the logic, and those cases can run without a database, HTTP server or deployed application.
That is useful protection. It describes a rule independently of how the method happens to implement it. The expected results should come from the requirement; an agent should not derive them by copying the current comparison operator.
I want the same kind of focused tests for calculations, state transitions, parsing and combinations of business rules. The more cases a rule has, the more valuable it can be to exercise them cheaply and get a precise failure.
I am less interested in a test that mocks every collaborator, invokes a method, and verifies that the implementation called the same collaborators in the same order—unless that interaction order is itself a requirement. Such a test may constrain a refactor while leaving the actual business outcome unchecked.
Put real dependencies around the boundaries
For a database-backed service, a mocked repository cannot show whether the query works with the database schema, whether a constraint is enforced, or whether the transaction behaves as intended.
Testcontainers runs dependencies in containers, so a local integration test can use a real PostgreSQL instance. I would run the application’s migrations, exercise the public operation, and assert what another read observes. That gives the test a chance to detect a mapping or persistence mistake that a repository stub would conceal.
Use the production database engine and relevant version/configuration where practical. A local container still does not reproduce every deployment setting or operational condition.
WireMock has a different role. It simulates an HTTP service and can verify requests, return controlled responses and inject faults. That makes it useful for cases such as a timeout, a malformed response or a retry policy. It is still a test double: passing against the stub shows that the application handles the simulated interaction.
For an illustrative order workflow, I would choose checks like these:
| Risk | Test I would use | Assertion that matters |
|---|---|---|
| Incorrect expiry boundary | Focused unit test | Reservation is invalid at the exact expiry instant |
| Incorrect persistence or mapping | Integration test with PostgreSQL in Testcontainers | Created order can be read back with the required values |
| Incorrect HTTP request or failure handling | Integration test with WireMock | Correct request is sent; timeout produces the specified application outcome |
| Consumer and provider disagree | Verified contract test | Provider satisfies the interaction the consumer requires |
| Deployment wiring is broken | End-to-end test in a deployed test environment | Authenticated user completes the selected workflow and observes the expected result |
These are suggested tests for an example application, not results from a new benchmark.
Check the contract as well as the deployed path
A hand-written stub can agree perfectly with a client while both disagree with the provider. The expected interaction needs an independent basis and a way to detect drift.
For services you control, a consumer-driven contract approach can help. Pact pairs consumer tests with provider verification: the consumer’s required request and response are checked against the provider. For a third-party API, use its documented contract and validate important interactions against its supported sandbox where available.
I also want a few end-to-end checks against a deployed test environment, exercising the entry point, authentication, service wiring and state changes that a user depends on. Those checks cover the deployed paths they exercise. They do not establish compatibility for every possible request or replace focused contract verification.
Keep the broad tests selective. There is little value in repeating every calculation boundary through a browser if a fast test already catches those mistakes. Use the deployed test to check the connections that the focused test cannot see.
Prune generated tests by the protection they add
Yes, I think pruning agent-generated tests should be normal maintenance. I would start with an audit, not a bulk deletion based on similar names or shared line coverage.
For each candidate, ask:
- What behavior does it protect? Name the requirement, regression or failure mode. “Increases coverage” is not a behavior.
- What would make it fail? Check that the assertion would notice a plausible defect, rather than simply confirm the mock’s configured response.
- What does it add to the remaining suite? Consider input boundaries, assertions, dependency realism, execution speed and diagnostic value.
- Can it be simplified safely? Parameterize equivalent cases or share straightforward setup. Keep distinct boundary cases and readable expectations.
Two tests that execute the same lines may check different properties. A unit test and a deployed test may exercise the same business operation while protecting different risks. Similar coverage alone does not make either redundant.
For critical logic, try a targeted mutation or reintroduce a known defect on a disposable branch before and after the cleanup. PIT can automate changes such as altering a comparison boundary and check whether the tests detect them. Surviving mutations need interpretation; some do not change observable behavior. Mutation testing is another diagnostic, not a new score to maximize blindly.
Here is the audit prompt I would give an agent:
Review these tests without changing them yet. For each test, identify the requirement, the defect it would detect, the assertion that detects it, and any distinct boundary or dependency it exercises. Propose keep, simplify, merge or delete. For every deletion, name the retained protection and explain why no unique failure or useful diagnostic would be lost. Do not weaken assertions, lower coverage thresholds, or remove regression cases merely to make the suite smaller.
Then review the proposal and make bounded changes. Preserve the old revision so a lost check can be recovered.
Unit tests remain valuable when they give fast, specific evidence about a real rule. Integration tests matter when the risk sits between components. End-to-end tests matter when the deployed path itself can fail. With agents producing more of all three, I want the review question to be concrete: what could break, and which test would tell us?
Top comments (1)
That 'assert what another read observes' distinction is easy to lose if the assertion reuses the ORM session that created the order. I'd make the readback use a fresh connection and cleared identity map after the transaction finishes, then check the stored values. Otherwise a test can inspect its in-memory object and miss a missing commit. That gives the PostgreSQL integration check a job the unit suite genuinely cannot do.