Hand-writing test data is slow. Free model endpoints can generate it fast. But raw model output is not test data. It needs validation. This tutorial builds a two-stage pipeline. Stage one generates records. Stage two validates them. Only valid records become fixtures.
Why Synthetic Test Data?
Test data must cover edge cases. Hand-written data misses them. Model-generated data covers more ground. It also produces garbage. Duplicates, wrong types, missing fields. You need a filter.
Pipeline Design
The pipeline has two stages. The generator calls a free model endpoint. It asks for JSON records. The validator checks each record against a schema. It rejects invalid records. It writes the survivors to a fixture file.
Stage 1: Generate Records
You need a free model endpoint. MonkeyCode provides free model access. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Set two environment variables.
export MODEL_URL="https://your-endpoint.example/v1/chat"
export MODEL_TOKEN="your-token"
The generator sends a prompt. The prompt specifies the shape. It asks for ten user objects. It demands valid email format.
PROMPT = """Generate 10 user objects as a JSON array.
Each object must have: id (integer), name (string), email (string), role (string).
Vary the names and email domains. Use valid email format."""
The response is a JSON array. Or it is text with code fences. You must handle both.
import json
import os
import httpx
MODEL_URL = os.environ["MODEL_URL"]
MODEL_TOKEN = os.environ["MODEL_TOKEN"]
def generate():
response = httpx.post(
MODEL_URL,
headers={"Authorization": f"Bearer {MODEL_TOKEN}"},
json={"messages": [{"role": "user", "content": PROMPT}], "max_tokens": 500},
timeout=30,
)
response.raise_for_status()
text = response.json()["choices"][0]["message"]["content"]
if text.startswith("```
"):
text = text.strip("`")
text = text[text.index("\n"):]
return json.loads(text)
```
## Stage 2: Validate Every Record
Define a Pydantic model. It enforces types and defaults.
``{% endraw %}{% raw %}`python
from pydantic import BaseModel, ValidationError
class User(BaseModel):
id: int
name: str
email: str
role: str = "user"
```
Loop over the raw records. Catch validation errors. Print the rejected records. Keep the valid ones.
``{% endraw %}{% raw %}`python
def validate(records):
users = []
for record in records:
try:
users.append(User(**record))
except ValidationError as e:
print(f"Rejected: {record} -> {e}")
return users
```
This is the core gate. A record that fails validation is not a fixture. It is noise.
## Stage 3: Run and Inspect
Run the script.
``{% endraw %}{% raw %}`bash
python generate_fixtures.py
```
You should see output like this:
``{% endraw %}{% raw %}`plaintext
Generated 10 records, kept 8
Rejected: {'id': 'abc', ...} -> id: value is not a valid integer
Rejected: {'id': 4, 'name': 'Alice', 'email': 'not-an-email', 'role': 'admin'} -> email: value is not a valid email address
```
The kept records land in `users.json`. Inspect them.
``{% endraw %}{% raw %}`bash
cat users.json
```
Verify that the data looks realistic. Check for duplicates. Add a uniqueness check if needed.
``{% endraw %}{% raw %}`python
def check_duplicates(users):
ids = [u.id for u in users]
if len(ids) != len(set(ids)):
print("Duplicate ids found")
```
## Stage 4: Deploy to a Free Server
You want this pipeline to run on demand. Use MonkeyCode's free server option. Create a server instance. Copy the script.
```bash
scp generate_fixtures.py requirements.txt user@server:~/
```
Install dependencies on the server.
```bash
pip install httpx pydantic
```
Run the script on the server. Verify the same output.
For automation, use a cron job. Run it daily. Regenerate fixtures with fresh data.
```bash
crontab -e
0 6 * * * cd ~ && python generate_fixtures.py >> cron.log 2>&1
```
Check the log after the first run.
## Stage 5: Measure the Pipeline
A good pipeline has a high pass rate. Track it over time.
```bash
grep "Generated" cron.log | tail -10
```
If the pass rate drops, the model changed. Or the prompt drifted. Investigate.
## Limitations
Model-generated data has bias. It reflects the model's training data. It is not a substitute for real production data. Use it for unit tests, not for load tests.
This pipeline does not handle streaming. It assumes a complete JSON response. If your endpoint streams, buffer the full payload first.
Who should not use this? Teams with strict data privacy. Sending real user data to a free endpoint is dangerous. Use synthetic data only.
## Conclusion
A free model endpoint can be a data factory. You need a validation gate. The two-stage pipeline turns raw output into usable fixtures. Run it on a free server. Keep the pass rate visible.
If you want a free endpoint and a free server to try this, MonkeyCode's free options are a reasonable start. The pipeline itself works with any OpenAI-compatible API.
Top comments (0)