DEV Community

MSakai
MSakai

Posted on

The pytest fixture that quietly leaks state between your tests

#pytest #python #testing #tdd

The suite is green. You run it with -p no:randomly off, or add a test in the middle, and two unrelated tests turn red.

@pytest.fixture(scope="module")
def config():
    return {"retries": 3, "debug": False}

def test_debug_mode(config):
    config["debug"] = True
    assert some_feature(config) == "verbose"

def test_default_mode(config):
    assert some_feature(config) == "quiet"   # fails if it runs second
Enter fullscreen mode Exit fullscreen mode

What's actually happening

scope="module" means the fixture body runs once per module, and every test in that module receives the same object. Not a copy. The dict test_debug_mode mutated is the dict test_default_mode receives.

This is invisible while your tests happen to run in a forgiving order, which is exactly what makes it expensive to debug later.

The rule that decides scope

Scope is about cost of setup, and it is only safe when the object is read-only or self-cleaning.

Scope Use when
function (default) Anything mutable. Start here.
class / module Setup is expensive and the object is immutable or reset between tests
session Genuinely global and expensive: a container, a test database, a browser

If you cannot say "nothing in this module mutates it", it belongs at function scope.

Three fixes, in order of preference

1. Drop the scope. The cheapest fix is usually the right one. Building a dict costs nothing.

@pytest.fixture
def config():
    return {"retries": 3, "debug": False}
Enter fullscreen mode Exit fullscreen mode

2. Keep the expensive part shared, hand out a copy.

@pytest.fixture(scope="module")
def _base_config():
    return load_config_from_disk()      # the expensive bit, done once

@pytest.fixture
def config(_base_config):
    return copy.deepcopy(_base_config)  # each test gets its own
Enter fullscreen mode Exit fullscreen mode

3. Make it immutable so the mutation fails loudly.

from types import MappingProxyType

@pytest.fixture(scope="module")
def config():
    return MappingProxyType({"retries": 3, "debug": False})
Enter fullscreen mode Exit fullscreen mode

Now config["debug"] = True raises TypeError in the test that caused the problem, instead of failing a different test five minutes later.

Prove it before it bites you

Install pytest-randomly and let order-dependence surface in CI rather than in a Friday afternoon debugging session:

pip install pytest-randomly
pytest            # shuffles test order every run, prints the seed
pytest -p no:randomly   # reproduce the old behaviour when you need to
Enter fullscreen mode Exit fullscreen mode

A suite that only passes in one specific order isn't a passing suite. It is a suite you haven't finished writing yet.

The takeaway

Widening a fixture's scope is a performance optimisation. Treat it like any other optimisation: only after you've measured, and only when you can prove nobody mutates the shared object.

These posts come out of material I build for my Udemy courses — 25 of them now, mostly drill-based, across Go, Python, TypeScript, testing and Three.js. If this was useful, the full list is at udemy-c1f90.web.app. The links on that page carry a coupon I refresh each month, which usually lands around half the list price.

Top comments (0)