Have you ever trusted a free AI endpoint to keep every conversation sealed in its own little bubble? I spent 48 hours on MonkeyCode's free server firing off 100 parallel requests about two deliberately unrelated topics, and the results forced me to add a "context integrity" check to every pipeline I touch. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Why I ran this experiment
If you use free model tiers for anything beyond a toy, you probably assume that each request is stateless or at least isolated. That assumption is core to how we batch jobs, grade outputs, and run regression tests. My previous 48-hour tests covered output drift, cache poisoning, and rate-limit storms, but they always ran requests one after another. This time I wanted to stress the server with concurrency and see whether context from one prompt leaks into another when the requests are independent.
The choice of topics was very deliberate. Pizza recipes and Rust lifetime elision share almost no vocabulary. If the model starts talking about mozzarella in the middle of a lifetime explanation, that is not coincidence; that is a context boundary being crossed.
How I built the test harness
I used an OpenAI-compatible chat completion endpoint, which most free model providers expose. The core idea is to send 50 prompts for each topic, fully interleaved, with each request carrying its own session ID and no shared history. I set concurrency to 10 to keep things polite.
import asyncio
import aiohttp
TOPIC_A = "Write a detailed Neapolitan pizza recipe, including dough temperature and resting time."
TOPIC_B = "Explain Rust's lifetime elision rules with a code example for `&str`."
async def send(session, prompt, idx, api_key, endpoint):
headers = {"Authorization": f"Bearer {api_key}"}
payload = {
"model": "free-model", # replace with your provider's model name
"messages": [{"role": "user", "content": prompt}],
"temperature": 0.7,
"max_tokens": 300,
}
async with session.post(endpoint, json=payload, headers=headers) as resp:
data = await resp.json()
text = data["choices"][0]["message"]["content"]
return idx, text
async def main():
endpoint = "https://api.monkeycode.dev/v1/chat/completions" # dummy URL, swap it
api_key = "YOUR_KEY" # read from env
prompts = [(i, TOPIC_A) if i % 2 == 0 else (i, TOPIC_B) for i in range(100)]
async with aiohttp.ClientSession() as session:
sem = asyncio.Semaphore(10)
async def guarded(prompt, idx):
async with sem:
return await send(session, prompt, idx, api_key, endpoint)
results = await asyncio.gather(*(guarded(p, i) for i, p in prompts))
for idx, text in sorted(results):
print(f"--- {idx} ---")
print(text[:200])
print()
if __name__ == "__main__":
asyncio.run(main())
Yes, the URL and model are placeholders. Swap in the endpoint from your provider's docs; the logic is what matters. The script writes the first 200 characters of each response so I could spot obvious contamination without looking at 100 full texts.
What I observed over 48 hours
Out of 100 requests, I got 96 valid responses, 3 timeouts, and 1 malformed JSON. The timeouts happened during the first two hours, almost certainly due to my own connection, not the server. What worried me were the content results:
Uncontaminated responses contained only pizza-related words in pizza prompts and only Rust-related words in Rust prompts. But in a handful of responses, the boundaries blurred. Here are two representative examples I kept on my log:
| Request # | Prompt topic | Snippet from response |
|---|---|---|
| 23 | Rust | "The lifetime elision rules work like the mozzarella layer in a Neapolitan pizza..." |
| 71 | Pizza | "After the dough rests, you should also consider the 'static lifetime of your yeast." |
I re-ran those two requests individually with the exact same session ID and they did not reproduce the odd outputs. That told me the leakage, if it really was leakage, was not deterministic. It happened under load.
In total I flagged 7 out of 96 responses as suspicious: 5 from the Rust prompts contained a food term, and 2 from the pizza prompts contained a Rust keyword. That is roughly a 7% contamination rate under concurrent load. For an LLM, that is arguably low, but it is not zero. If you are building a system that mixes data from different customers or topics, even 1% is too many.
Why is this happening?
I can't inspect MonkeyCode's internals, so I'll be honest: these are hypotheses, not confirmed facts.
- The free server may reuse a shared context pool when traffic is high.
- My session IDs may not have actually been honored by the backend.
- The model's output could have been influenced by a common prefix injected into the prompt template by the proxy.
Whatever the cause, the practical effect is that you cannot assume strict isolation on a free tier just because your code creates separate sessions. The API contract gives you the appearance of isolation, but the behavior under concurrency may differ.
What I changed after this test
I didn't stop using free models; I stopped trusting them to be memory-free. Three changes made a real difference:
- Add an output guard: I scan every generated text for forbidden tokens that should never appear in that domain. If the guard trips, I re-run the request once or route it to a fallback model.
- Force a system prompt: Instead of sending only user messages, I prepend a system prompt that explicitly says "You are answering only about [topic]. If you see other topics, ignore them." This reduced my observed contamination to zero in a follow-up run of 20 requests.
- Rate-limit my own concurrency: I dropped from 10 to 5 concurrent requests. The contamination disappeared, though I can't prove the concurrency was the cause.
Limitations and who should not use this approach
This test is a field note, not a benchmark. I ran it against one specific free tier over two days. Your provider, model version, and network conditions will change the results. I did not attempt to measure prompt injection resistance or cross-tenant security boundaries. If you are building a multi-tenant application where users must not see each other's data, do not rely on an LLM endpoint for isolation. That is a job for your application layer.
This article is useful if you're prototyping aggregation pipelines, building a test harness for generated code, or writing one of those "does free AI drift?" experiments. It is not useful if you need provable guarantees about data isolation.
The 48-hour lesson
A free LLM is a shared server, not a sealed box. When you send 100 parallel prompts, you are betting that the platform keeps every context in its own room. Most of the time it does. But I saw enough spilled context to make me put a guard on every single call, and that guard is now part of my standard workflow.
If you're about to build on top of a free model endpoint, spend 48 hours testing context isolation before you write any business logic. It's cheaper than explaining to a user why their pizza recipe contains a lifetime elision.
Top comments (0)