Why I stopped hardcoding test data and built an AI-powered pytest fixture generator

#python #pytest #testing #ai

The problem I kept hitting

Every project. Every codebase. The same thing:

def test_user_registration():
    user = {
        "name": "John Doe",
        "email": "john@test.com",
        "age": 30
    }
    result = register_user(user)
    assert result["id"]

It works. It passes. And it tests exactly one data path — the happy path with boring English data.

What about:

name = "张伟" — does your system handle Unicode?
age = 0 or age = 150 — boundary values?
email = "user+tag@sub.domain.co.uk" — valid but unusual format?

You never test these unless you manually write them. Nobody does.

The "use Faker" answer doesn't work either

from faker import Faker
fake = Faker()

user = {"name": fake.name(), "email": fake.email()}

Faker is great. But it's random — which means your CI pipeline fails randomly too. And it doesn't understand context. It won't generate adversarial edge cases. It doesn't know your schema.

What I actually wanted

I wanted fixtures that:

Are AI-generated in dev — rich, varied, surprising
Are deterministic in CI — same data every run, no API key required
Understand my schema — not just random values
Work as a pytest plugin — zero boilerplate

So I built FixtureForge.

How it works

pip install fixtureforge

Basic usage — no API key needed:

from fixtureforge import forge

@forge(schema={"name": str, "email": str, "age": int})
def test_user_creation(fixture):
    assert "@" in fixture["email"]
    assert isinstance(fixture["age"], int)

This runs in deterministic mode by default. Seed is 42. Same data every time.

Add AI by setting any provider key:

@forge(
    schema={"name": str, "email": str, "age": int},
    prompt="Edge cases: Unicode names, boundary ages (0, 1, 150), malformed-but-valid emails"
)
def test_user_edge_cases(fixture):
    # Now you get: 张伟, age=0, email=user+tag@sub.domain.co.uk
    assert register_user(fixture)

DataSwarms — bulk generation

from fixtureforge import DataSwarm

swarm = DataSwarm(
    schema={"product": str, "price": float, "qty": int},
    count=500,
    seed=42
)

def test_bulk_processing(swarm):
    results = process_orders(swarm.data)
    assert len(results) == 500

CI/CD integration

# .github/workflows/test.yml
- name: Run tests
  env:
    FIXTUREFORGE_MODE: deterministic
    FIXTUREFORGE_SEED: 42
  run: pytest

No AI key in CI. Auto-detects and falls back to deterministic mode. Tests are stable.

Multi-provider support

# OpenAI
forge(provider="openai", model="gpt-4o-mini")

# Anthropic
forge(provider="anthropic", model="claude-haiku-4-5")

# Groq (fast + free tier)
forge(provider="groq", model="llama3-8b-8192")

ForgeMemory — context across tests

from fixtureforge import ForgeMemory

memory = ForgeMemory()

@memory.remember("created_user")
def test_create_user(fixture):
    assert fixture["id"]  # stored

@memory.recall("created_user")
def test_update_user(fixture):
    assert update_user(fixture["id"], name="Updated")  # same fixture