Your Free AI Will Forget, Stall, or Queue. Probe All Three.

Free AI fails in patterns. Three of them. It forgets context, stalls after idle, and queues under bursts. Each failure looks different. Each needs a different fix. You can classify all three in ten minutes. This is the probe I use.

You found a free model and a free server. The demo prompt answers instantly. Then a real task arrives with a 20k-token context. The answer comes back wrong. No error. No warning. Just quiet drift. Silent wrong answers are the worst failure mode. Errors are loud. Drift is expensive.

MonkeyCode is an open-source AI coding assistant. It offers a free model tier with a 10M-token budget and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I still treated the free tier like any other dependency. I tested it before trusting it.

The probe has four stages. Warm-up measures the handshake. Burst measures the queue. Needle measures memory. Idle recovery measures the nap. Together they classify your free tier.

Here is the full script. Save it as free_tier_probe.py.

#!/usr/bin/env python3
"""free_tier_probe.py — classify how your free AI tier fails."""

import concurrent.futures
import json
import time
import urllib.request

ENDPOINT = "http://localhost:8080/v1/chat/completions"
MODEL = "default"
TOKEN = ""
CANARY = "PLUM_42_7F3A"
FILLER = "The quick brown fox jumps over the lazy dog. " * 2000


def chat(prompt, max_tokens=200):
    body = json.dumps({
        "model": MODEL,
        "messages": [{"role": "user", "content": prompt}],
        "max_tokens": max_tokens,
    }).encode()
    request = urllib.request.Request(
        ENDPOINT,
        data=body,
        headers={
            "Authorization": f"Bearer {TOKEN}",
            "Content-Type": "application/json",
        },
    )
    start = time.perf_counter()
    try:
        with urllib.request.urlopen(request, timeout=120) as response:
            payload = json.load(response)
        elapsed = (time.perf_counter() - start) * 1000
        text = payload["choices"][0]["message"]["content"]
        return {"ok": True, "ms": round(elapsed, 1), "text": text}
    except Exception as error:
        elapsed = (time.perf_counter() - start) * 1000
        return {"ok": False, "ms": round(elapsed, 1), "error": str(error)}


def needle_prompt():
    question = f"\n\nMemorize this code: {CANARY} = 0x7F3A.\nWhat is the value of {CANARY}?"
    return FILLER + question


def main():
    print("== warm-up ==")
    print(chat("Reply with OK."))

    print("== burst ==")
    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as pool:
        results = list(pool.map(
            lambda _: chat("Write a one-line Python f-string."),
            range(5),
        ))
    for index, result in enumerate(results):
        print(index, result["ok"], result["ms"])

    print("== needle ==")
    result = chat(needle_prompt(), max_tokens=60)
    print(result)
    if result["ok"]:
        print("canary found:", CANARY in result["text"])

    print("== idle recovery ==")
    time.sleep(60)
    print(chat("Reply with OK again."))


if __name__ == "__main__":
    main()

It assumes an OpenAI-compatible chat endpoint. Point ENDPOINT at your server. Set MODEL to the free model. Set TOKEN if your setup requires auth. Then run it.

python3 free_tier_probe.py

The needle stage is the important one. It stuffs roughly 20k tokens of filler into the context. Then it asks for one code value buried inside. If the model returns the wrong value, context handling is degrading. That is the silent failure most demos never show.

Illustrative output looks like this. Your numbers will differ. Run it yourself.

== warm-up ==
{'ok': True, 'ms': 2140.3, 'text': 'OK.'}
== burst ==
0 True 3120.1
1 True 4890.7
2 False 120000.0
3 True 5110.2
4 True 4988.9
== needle ==
{'ok': True, 'ms': 18700.4, 'text': 'The value is 0x0000.'}
canary found: False
== idle recovery ==
{'ok': True, 'ms': 41200.8, 'text': 'OK.'}

What do the numbers mean? Warm-up under three seconds means the server is awake. Burst failures above one in five mean parallel work will hurt. A needle miss means long context is a gamble. Idle recovery over thirty seconds means your server sleeps.

Each pattern points to a different fix. Queueing? Add retries with backoff. Stalling? Send a keep-alive ping. Forgetting? Chunk your context before you send it. Do not apply one fix to all three.

The free tier handles short, sequential, single-user tasks well. A small diff review. A quick refactor. A documentation draft. It breaks on long context, parallel bursts, and strict latency. The probe tells you which camp you are in before your deadline does.

Who should not use this approach? Teams with production latency SLOs. Teams with real-time user traffic. Teams where one silent wrong answer costs real money. For them, a free tier is a liability, not a saving. The probe will say so plainly.

The probe's value is not the pass or fail. It is the pattern. A tier that forgets is different from one that stalls. You fix forgetting with chunking. You fix stalling with a ping. You cannot fix what you have not measured.

MonkeyCode's free model and free server are open to try. Point this probe at them before you commit. Ten minutes of measurement beats a week of surprises.