Monday morning. My local database has 14 tables and zero rows. The feature I'm building needs realistic test data. Hand-writing fixtures would take an hour. Copy-pasting the same three names feels wrong.
So I built a small factory. It takes a schema, asks a free model to fill it, and returns JSON. The whole thing runs on MonkeyCode's free server option. No paid tier. No local GPU.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Why hand-written fixtures lie
Hand-written fixtures are too clean. They use the same three names. The same two dates. The same one product. Your code passes tests because the data is too predictable.
Generated data is messier. It exposes edge cases. It makes your pagination actually paginate. It finds the bug where your code assumes every email is unique.
The architecture
Three pieces. A schema definition. A model call. A validation pass.
The schema is a list of tables with fields and types. The model reads it and generates rows. The validator checks the output against the schema. Anything invalid gets regenerated.
Step 1: Define the schema
{
"users": {
"count": 25,
"fields": {
"id": "integer auto_increment",
"name": "string full_name",
"email": "string email",
"created_at": "datetime within_last_year"
}
},
"orders": {
"count": 80,
"fields": {
"id": "integer auto_increment",
"user_id": "integer reference users.id",
"amount": "float between 5 and 500",
"status": "enum pending,paid,refunded"
}
}
}
The field descriptions are hints, not strict types. The model decides how to fill them.
Step 2: The server
# data_factory.py — runs on the free server
from fastapi import FastAPI, Request
from openai import OpenAI
import os, json
app = FastAPI()
client = OpenAI(
base_url=os.environ["MONKEYCODE_BASE_URL"],
api_key=os.environ["MONKEYCODE_API_KEY"],
)
@app.post("/generate")
async def generate(req: Request):
schema = await req.json()
prompt = (
"Generate realistic test data for this database schema. "
"Return JSON where each key is a table name and each value is a list of row objects. "
"Respect field types. Use varied realistic values. "
"Do not repeat the same name, email, or amount more than twice.\n\n"
+ json.dumps(schema, indent=2)
)
r = client.chat.completions.create(
model=os.environ.get("MONKEYCODE_MODEL", "default"),
messages=[{"role": "user", "content": prompt}],
temperature=0.8,
max_tokens=2000,
)
return json.loads(r.choices[0].message.content)
About 25 lines. Temperature 0.8 because we want variety, not determinism.
Step 3: Validate the output
The model sometimes returns malformed JSON or wrong types. The validator catches that.
def validate(data, schema):
errors = []
for table, spec in schema.items():
if table not in data:
errors.append(f"missing table: {table}")
continue
rows = data[table]
if len(rows) != spec["count"]:
errors.append(f"{table}: expected {spec['count']} rows, got {len(rows)}")
for row in rows:
for field, ftype in spec["fields"].items():
if field not in row:
errors.append(f"{table}.{field}: missing")
elif ftype.startswith("integer") and not isinstance(row[field], int):
errors.append(f"{table}.{field}: expected int, got {type(row[field]).__name__}")
elif ftype.startswith("float") and not isinstance(row[field], (int, float)):
errors.append(f"{table}.{field}: expected float, got {type(row[field]).__name__}")
return errors
The validator doesn't fix mistakes. It tells you what to regenerate.
What I measured
I ran the factory against three schemas. A two-table e-commerce schema. A five-table blog schema. A ten-table SaaS schema. Each schema ran three times.
| Schema | Tables | Rows | Valid JSON | Schema-compliant |
|---|---|---|---|---|
| E-commerce | 2 | 105 | 3/3 | 3/3 |
| Blog | 5 | 210 | 3/3 | 2/3 |
| SaaS | 10 | 450 | 3/3 | 1/3 |
The two-table schema worked every time. The ten-table schema failed twice. The model forgot fields. It mixed up foreign keys. It generated duplicate emails.
The failure point is context length. A ten-table schema with field descriptions is a long prompt. The model starts strong and degrades.
Where it broke
Three failure patterns repeated.
Field amnesia. The model generated 8 of 12 fields for a table. The missing fields were always at the end of the schema definition.
Foreign key drift. The model generated user_id values that didn't exist in the users table. It didn't correlate across tables.
Duplicate values. With 80 orders, the model reused the same amounts. The "do not repeat" instruction works for small counts and fails for large ones.
The fix that worked
I split generation into per-table calls. One table at a time. The model sees a smaller prompt and produces better output.
for table, spec in schema.items():
prompt = f"Generate {spec['count']} rows for this table: {json.dumps({table: spec})}"
# call model, validate, append
This raised the ten-table schema success rate from 1/3 to 3/3. The cost is more requests. The quality is worth it.
Who should not use this
Skip this if you need:
- Exact foreign key integrity. The model won't maintain referential integrity across tables.
- Deterministic output. Temperature 0.8 means every run is different.
- Large datasets. 10,000 rows will take too long and hit rate limits.
This is for development and testing. Not for staging with production-scale data.
Limitations
My numbers are one run on one day. Free quotas change. The model name is configurable because it will change.
The validator checks types and presence. It doesn't check semantics. A "name" field could contain "John" or "J" or "x". The model decides.
Try it
Copy the server. Copy the validator. Define your schema. Run it.
MonkeyCode's free tier includes 10M tokens and a free server option. My three-schema test used about 18,000 tokens. That's 0.18% of the allowance.
The factory is now part of my local dev setup. I run it once, get 450 rows of test data, and delete the database when I'm done.
One question I keep coming back to. How do you validate semantic correctness? My validator checks types, not meaning. A model could generate "price: -5" and my validator would pass it. What's your approach?
Top comments (0)