Shubham Anandani

Posted on Oct 11 Originally published at imshubh.com on May 15

Flaky Tests Are a Concurrency Bug: Fixing Shared State Under Parallel Execution

#java #testing #concurrency #softwareengineering

I’ve watched teams chase “test runner bugs” while the real culprit was design. Flakiness is often a concurrency issue: shared mutable state under parallel execution. JUnit 5 can run tests concurrently, so fixtures that share mutable state can make failures appear intermittent. Creating isolated data per test makes the behavior easier to reproduce and diagnose.

The failure mode

You enable parallel test execution and see intermittent failures: “record not found” or “unexpected duplicate.” You re-run and it passes. Blaming the test runner is easy. The problem is shared state: two tests reading or writing the same fixture or relying on an implicit order.

What I learned

Default to per-test fixtures

Use per-test initialization to avoid shared mutable state. In JUnit 5, keep your test instance lifecycle at PER_METHOD and recreate fixtures for each test. If the fixture is expensive, consider a template with parameterization that still isolates state.

Example setup

import org.junit.jupiter.api.*;

@TestInstance(TestInstance.Lifecycle.PER_METHOD)
class EvidenceServiceTest {
    private InMemoryStore store;
    private EvidenceService service;

    @BeforeEach
    void init() {
        store = new InMemoryStore(); // new store per test
        service = new EvidenceService(store);
        store.seedProfile("alice", syntheticProfileAlice());
    }

    @Test
    void returnsMissingWhenTimeout() {
        // ...
    }

    @Test
    void returnsUnverifiableOnExceptions() {
        // ...
    }
}

Avoid static mutable fields, especially in helpers

Static caches and helpers make tests faster locally but brittle in parallel. If you truly need a cache, scope it per test, or guard it with thread-safe constructs and clear it explicitly. Better: don’t cache at all in tests unless you benchmarked the need.

Make isolation visible with name-bound fixtures

If a test seeds data, bind it to a unique key per test. Use random UUIDs or test names in keys to prevent accidental cross-test reuse. With Testcontainers, create per-test containers when state isolation is critical; share containers only for read-only tests.

Parallel execution deserves explicit design

JUnit 5 lets you run tests in parallel. That doesn’t mean every test should be parallel. Annotate classes that require single-thread execution, but treat that as a last resort. Prefer making tests parallel-safe by design.

JUnit configuration (illustrative)

junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = concurrent
junit.jupiter.execution.parallel.mode.classes.default = concurrent

Deterministic time and waiting

Tests that sleep and “hope the system is ready” fail under parallel runs. Replace sleeps with explicit synchronization: await a condition, poll with a small backoff, or use latches. Keep timeouts small and deterministic.

Embrace “given-when-then” with isolated builders

Build input data per test with dedicated builders. Don’t reuse a mutable “default profile” object across tests; copy or reconstruct it. Records help: immutable inputs prevent accidental mutation.

Diagnose with logging and seeds

When a test fails intermittently, print seeds and keys that identify data. If you see the same key in two tests, you found shared state. If you see non-deterministic ordering in a collection, sort inputs or assert set equivalence rather than list order.

Per-test container with unique data

class StoreTests {
    private PostgreSQLContainer<?> database;

    @BeforeEach
    void setUp() {
        database = new PostgreSQLContainer<>("postgres:alpine")
            .withDatabaseName("test_" + UUID.randomUUID())
            .withUsername("user")
            .withPassword("test-only");
        database.start();
    }

    @AfterEach
    void tearDown() {
        database.stop();
    }
}

Trade-offs and failure modes

Trade-offs

Isolation vs speed: Per-test fixtures are slower than shared ones. Optimize later: start with correct behavior and only share read-only fixtures.
Parallelism vs determinism: Parallel runs surface concurrency bugs but may require more careful test design. The benefits outweigh the cost.
Container lifecycle vs developer ergonomics: Per-test containers are robust but slower. Consider class-level containers for read-only tests with clear resets.

Failure modes

“Hidden globals”: A helper modifies a static map. Under parallel execution, tests interleave and fail intermittently.
“Order assumptions”: Tests rely on iteration order of sets or maps; parallel runs expose different orders.
“Implicit sleeps”: Tests use fixed sleeps; faster or slower environments cause out-of-order expectations.

Try it yourself

Switch to the PER_METHOD lifecycle and remove static mutable fields in tests. Add unique keys per test to seed data. Enable parallel execution and watch for failures that indicate hidden shared state.
Replace sleep-based readiness with explicit condition checks. Add timeouts and backoff policies.
Document which tests are parallel-safe and why; make non-parallel tests the exception and note the reason.

DEV Community