pytest fixtures that actually scale: patterns from 2 years of Python CI pipelines

I spent two years watching the same test suite failures in CI.

Not logic failures — the test logic was fine. Infrastructure failures. The fixture that created a database table but didn't clean it up. The S3 mock that shared state across test files and produced a passing suite locally and a flaky one in CI. The fixture that took 4 seconds to set up because it was creating a full schema, and ran 200 times across the test suite.

The failures had the same root cause every time: fixtures optimized for "it works" rather than for isolation, speed, and reliability under CI's parallel execution model. Once I understood the patterns that caused the problems, I could see them appearing in every codebase I touched.

These are the four patterns I use now. They don't require any new dependencies — just a clear model of how pytest's scope and teardown machinery works.

---

The scope problem (and why function scope is your default)

pytest fixtures have four scope levels: function, class, module, and session. The default is function. The mistake I see most often is upgrading to module or session scope too aggressively to "speed things up."

The performance gain is real. A session-scoped database connection is faster than recreating it 200 times. The hidden cost is test isolation — the assumption that each test starts with a clean state.

Session-scoped fixtures share state across the entire test run. If test A modifies shared state and test B reads it, you have an ordering dependency. The tests pass when run in isolation and fail when run together — or worse, pass in one order and fail in another. CI parallelism makes this non-deterministic.

The rule I follow:

• Expensive, read-only setup → session scope (loading a config file, establishing a connection pool, reading fixture data from disk)
• Stateful setup → function scope (anything that creates, modifies, or deletes records)
• Explicit reset instead of scope upgrade → always

If you want module or session scope for performance but need clean state per test, yield + teardown is the right tool:

@pytest.fixture(scope="module")
def db_connection():
    conn = create_connection()
    yield conn
    conn.close()  # cleanup after module completes

For stateful resources, function scope + fast setup is better than session scope + flaky isolation:

@pytest.fixture(scope="function")
def clean_table(db_connection):
    """Truncate and repopulate test data for each test."""
    db_connection.execute("TRUNCATE test_records")
    db_connection.execute("INSERT INTO test_records VALUES ...")
    yield db_connection
    # teardown implicit — truncation at start of next test handles it

---

Layered fixtures: build complexity from simple pieces

The most common fixture anti-pattern I see: a large fixture that sets up everything a test might need, even if the test only needs part of it.

# Anti-pattern: monolithic fixture
@pytest.fixture
def full_environment(aws_credentials, s3_bucket, dynamodb_table, sqs_queue, test_user):
    # 40 lines of setup
    yield FullEnv(s3=s3_bucket, db=dynamodb_table, ...)

The problem: every test that uses full_environment pays the full setup cost, even if it only needs S3. And when the SQS setup fails, it breaks tests that don't use SQS at all.

The alternative: layer fixtures from narrow to broad, and let pytest handle composition:

@pytest.fixture
def aws_credentials(monkeypatch):
    """Minimal AWS credential mocking. Every AWS test needs this."""
    monkeypatch.setenv("AWS_ACCESS_KEY_ID", "testing")
    monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "testing")
    monkeypatch.setenv("AWS_DEFAULT_REGION", "us-east-1")


@pytest.fixture
@mock_aws
def s3_client(aws_credentials):
    """S3 client. Depends on credentials, nothing else."""
    return boto3.client("s3", region_name="us-east-1")


@pytest.fixture
@mock_aws
def test_bucket(s3_client):
    """Pre-created S3 bucket. Depends on s3_client."""
    s3_client.create_bucket(Bucket="test-bucket")
    return "test-bucket"

Now a test that only needs S3 requests test_bucket. A test that needs DynamoDB requests dynamodb_table. They get exactly what they need, nothing else. pytest resolves the dependency tree automatically.

The benefit: failures are isolated. If SQS setup breaks, only SQS-dependent tests fail. The rest of the suite continues.

---

The mock boundary problem

With moto (or any mock library), the scope of the mock matters as much as the scope of the fixture.

The @mock_aws decorator intercepts boto3 calls at the function level. If you apply it to the fixture, the mock is only active inside that fixture's setup code:

# Bug: mock_aws only active during fixture setup, not during the test
@pytest.fixture
@mock_aws
def s3_client(aws_credentials):
    client = boto3.client("s3")
    client.create_bucket(Bucket="test-bucket")  # this works
    return client
    # mock_aws context exits here

def test_upload(s3_client):
    s3_client.put_object(...)  # boto3 calls here are NOT mocked — real AWS

The correct pattern is to manage the mock context in the fixture using with or to pass the context manager into the test explicitly:

@pytest.fixture
def s3_mock():
    """Provides an active mock_aws context for the duration of the test."""
    with mock_aws():
        yield


@pytest.fixture
def s3_client(aws_credentials, s3_mock):
    """S3 client within the active mock context."""
    client = boto3.client("s3", region_name="us-east-1")
    client.create_bucket(Bucket="test-bucket")
    return client


def test_upload(s3_client):
    # s3_mock is active for the full duration of this test
    s3_client.put_object(Bucket="test-bucket", Key="key.txt", Body=b"content")
    response = s3_client.get_object(Bucket="test-bucket", Key="key.txt")
    assert response["Body"].read() == b"content"

The s3_mock fixture holds the mock_aws() context open for the entire test. s3_client depends on s3_mock, so pytest ensures the mock is active before setting up the client.

This pattern eliminates the "works locally, fails in CI when test ordering changes" class of failures.

---

The conftest.py architecture: centralize shareable fixtures

If you have AWS fixtures spread across multiple test files, you're re-creating the same setup code everywhere. Any changes — adding a new region, changing the mock scope, adding cleanup — need to be made in multiple places.

The solution is a conftest.py at the appropriate level. pytest automatically discovers conftest.py files and makes their fixtures available to all tests in the same directory and below.

For a typical project structure:

project/
├── conftest.py          # Project-wide fixtures: aws_credentials, s3_mock, sqs_mock
├── tests/
│   ├── conftest.py      # Test-suite-wide fixtures that need the project root
│   ├── test_ingestion/
│   │   ├── conftest.py  # Module-specific fixtures: pre-populated tables, test data
│   │   └── test_*.py
│   └── test_processing/
│       ├── conftest.py
│       └── test_*.py

The root conftest.py contains the infrastructure fixtures — AWS mocking, database connections, session-scoped expensive setup. The module-level conftest.py files contain domain-specific setup — test data, pre-populated tables, service-specific clients.

This layering means:

• Adding a new test file in test_ingestion/ automatically gets the AWS mocks
• Changing the mock configuration is a one-file change
• Module-specific fixtures don't pollute the namespace of other modules

# conftest.py (project root)
import boto3
import pytest
from moto import mock_aws


@pytest.fixture(scope="function")
def aws_credentials(monkeypatch):
    monkeypatch.setenv("AWS_ACCESS_KEY_ID", "testing")
    monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "testing")
    monkeypatch.setenv("AWS_SECURITY_TOKEN", "testing")
    monkeypatch.setenv("AWS_SESSION_TOKEN", "testing")
    monkeypatch.setenv("AWS_DEFAULT_REGION", "us-east-1")


@pytest.fixture(scope="function")
def aws_mock(aws_credentials):
    """Active mock_aws context for the full test."""
    with mock_aws():
        yield


@pytest.fixture(scope="function")
def s3_client(aws_mock):
    return boto3.client("s3", region_name="us-east-1")


@pytest.fixture(scope="function")
def dynamodb_client(aws_mock):
    return boto3.client("dynamodb", region_name="us-east-1")


@pytest.fixture(scope="function")
def sqs_client(aws_mock):
    return boto3.client("sqs", region_name="us-east-1")

---

Putting it together: a CI-reliable test

A test that uses all four patterns — function scope, layered fixtures, correct mock boundary, conftest architecture:

# tests/test_ingestion/conftest.py
@pytest.fixture
def ingestion_bucket(s3_client):
    """Pre-created bucket with test prefix structure."""
    bucket_name = "test-ingestion-bucket"
    s3_client.create_bucket(Bucket=bucket_name)
    s3_client.put_object(
        Bucket=bucket_name,
        Key="raw/2026-03-24/event_001.json",
        Body=b'{"event": "test", "timestamp": 1711238400}'
    )
    return bucket_name


# tests/test_ingestion/test_processor.py
def test_processes_event_file(s3_client, ingestion_bucket):
    """Test processes a single event file from S3."""
    # Arrange: file already in bucket via fixture

    # Act
    result = process_s3_event(
        bucket=ingestion_bucket,
        key="raw/2026-03-24/event_001.json",
        s3_client=s3_client
    )

    # Assert
    assert result.status == "processed"
    assert result.event_count == 1

    # Verify side effects in S3
    response = s3_client.list_objects_v2(
        Bucket=ingestion_bucket, Prefix="processed/"
    )
    assert response["KeyCount"] == 1

This test:

• Has a clean, isolated S3 environment (function scope)
• The mock covers the full test execution (mock boundary correct)
• Domain-specific setup is in the module conftest (layered fixtures)
• The common AWS mocking is inherited from the root conftest (conftest architecture)

If you run this test 50 times in parallel in CI, it produces the same result every time.

---

The free conftest.py drop-in

If you're starting from a broken LocalStack setup or rebuilding from scratch, the patterns above are packaged into a drop-in conftest.py covering S3, DynamoDB, SQS, SNS, Lambda, and Secrets Manager. Free download.

Get the free conftest.py → (Kit landing page — coming soon)

The full version with all fixtures, plus 24 more production-ready Python scripts for automation pipelines, is in the Python Automation Cookbook.

Python Automation Cookbook — $39 one-time. 25 production-ready Python scripts.

---

Tags: #python #testing #pytest #devtools #productivity
Series: Python AWS Testing Without the Subscription Tax — Part 2

---

pytest 8.x is the current stable release as of early 2026. moto 5.1.22 was released March 8, 2026. All fixture patterns above are tested against both.