Your Free Server Has Noisy Neighbors. Here's the Probe.

Your app was fast yesterday. Today it crawls, and you have no idea why. You check your code, but nothing changed. You check your database, but nothing changed. Your logs are silent, and your users are angry.

The problem is not yours. The problem is your neighbor.

Free servers are shared. That is the trade. You get free compute, and your neighbor gets the same. When they run a big batch job, you feel it. Your latency spikes, your timeouts grow, and your users notice.

I learned this the hard way. My agent felt snappy in testing, then it hit production. The p95 doubled overnight, and I blamed my own code. I was wrong.

So I built a probe. It answers one question. Is the slowdown mine or my neighbor's? This tutorial shows you how to build it.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The free server is a shared environment, which makes it a perfect place to test this probe.

The probe

The idea is simple. Send a tiny request on a timer, then measure its latency. Track the statistics over time, and when the numbers shift, you know the environment changed.

A tiny request matters. It uses almost no tokens, it does not trigger your own rate limits, and it isolates the network path and the server queue. That is the signal you want.

Here is the probe.

# neighbor_probe.py
import os
import time
import statistics
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["MONKEYCODE_BASE_URL"],
    api_key=os.environ["MONKEYCODE_API_KEY"],
)

PROMPT = "Reply with the single word: ok."

def probe_once():
    started = time.time()
    response = client.chat.completions.create(
        model=os.environ.get("MONKEYCODE_MODEL", "default"),
        messages=[{"role": "user", "content": PROMPT}],
        max_tokens=5,
    )
    return time.time() - started

def collect(interval=5, count=20):
    samples = []
    for i in range(count):
        samples.append(probe_once())
        time.sleep(interval)
    return samples

Verification: run it. You should see twenty latency numbers. They will not be identical, and that is the point.

The statistics

Raw numbers are noise. You need summary statistics. Median handles outliers, p95 shows the worst case, and the gap between them is jitter.

def report(samples):
    latencies = sorted(samples)
    p50 = statistics.median(latencies)
    p95 = latencies[int(len(latencies) * 0.95) - 1]
    jitter = p95 - p50
    print(f"p50: {p50:.2f}s  p95: {p95:.2f}s  jitter: {jitter:.2f}s")
    return {"p50": p50, "p95": p95, "jitter": jitter}

Run it twice. Once in the morning, once at peak hours. If the p95 doubles while your code did not change, the environment moved.

Verification: you see three numbers. The p95 is always larger than the p50, and the jitter is their difference.

The baseline

One measurement means nothing. You need a baseline. Collect samples at a quiet time, store the p95, and that becomes your reference point.

{"p50": 0.42, "p95": 0.61, "jitter": 0.19, "collected_at": "2026-08-21T06:00:00Z"}

Now run the same probe during your busy hours and compare the new p95 to the baseline. This is the decision rule I use.

def detect_shift(current_p95, baseline_p95, threshold=2.0):
    ratio = current_p95 / baseline_p95
    if ratio >= threshold:
        print(f"Shift detected: {ratio:.1f}x baseline p95")
        return True
    print(f"Normal: {ratio:.1f}x baseline p95")
    return False

A two-times shift means something changed. It might be your traffic, or it might be your neighbor's batch job. The probe does not tell you which. It tells you the environment moved, and that is enough to stop blaming your code.

Verification: run the probe during a quiet hour, then run a heavy load test against the same server. Watch the p95 climb as the probe catches the shift.

The continuous monitor

A single comparison is a snapshot. You want a trend. Run the probe on a schedule, append each window's p95 to a log, and plot it over time.

# monitor.py
import json
import time
from neighbor_probe import collect, report

LOG = "neighbor_log.jsonl"

while True:
    samples = collect(interval=2, count=15)
    stats = report(samples)
    with open(LOG, "a") as f:
        f.write(json.dumps(stats) + "\n")
    time.sleep(60)

Every minute you get one line, and every hour you get sixty. Now you can see the shape of the noise. Is it constant, or is it spiky? Does it match your business hours, or someone else's?

That last question is the gold. If the noise follows your own traffic, fix your own code. If it follows a schedule you do not control, it is your neighbor. You cannot fix that, but you can design around it.

Verification: run the monitor for an hour and open the log. You should see p95 values that move. If they are flat, your server is quiet. Enjoy it.

Designing around the noise

The probe tells you the environment is noisy. Then what? You design for degradation.

First, set your own timeouts below the server's. If the server takes ten seconds, you timeout at eight. You fail fast, you retry later, and your user sees a graceful message instead of a hanging spinner.

Second, cache aggressively. Repeated requests do not need the model. Cache the common ones, and your latency drops to zero. The neighbor's noise never touches you.

Third, queue the work. Batch jobs can wait. Run them when the probe says the environment is quiet. This is the pattern I keep returning to. Evidence before action.

Limitations

This probe has limits. It measures the network path from your machine, not the server's internal queue directly. Your distance to the server adds noise of its own.

The two-times threshold is a heuristic, not a statistical test. It works for my workloads, but it might not work for yours. Tune it with your own data.

The probe cannot identify the neighbor. It cannot tell you who is causing the noise. It only tells you the environment changed, and that is usually enough.

Do not use this for production SLAs. You need the vendor's official monitoring for that. Do not use it for capacity planning, because the probe is too small to measure throughput. Do not use it on a single-tenant server, since there is no neighbor to detect.

Who should use it? Teams shipping on shared free tiers. You need to know when the environment is against you, and this is the cheapest way to find out.

The takeaway

Shared servers are a bargain with a hidden cost. The cost is noise, and the noise is invisible until you measure it.

Your code is not always the problem. Sometimes it is your neighbor. The probe tells you which, and that knowledge changes how you debug. It changes where you spend your time.

The free server at MonkeyCode is shared, which makes it a real test bed for this probe. Run it, watch the p95 move, and then decide if the free tier fits your product.

Measure the crowd. Then trust the server.

MonkeyCode provides free models that can run this workflow.