48 Hours of Cold Starts: Field Notes From a Free Server That Kept Dozing Off

Every tutorial tells you to start with a free server and a free model. But what happens between requests, when nobody is watching and the instance quietly goes to sleep? I spent 48 hours logging every wake-up, and the pattern changed how I budget latency for small AI tools. These are the field notes: what I tried, what broke, and what I would repeat.

Hour 0: The Setup

I built a tiny endpoint that classifies a support ticket as billing, bug, or feature using a free model call. The goal was an internal triage bot for a side project, not a product, which matters because it set my expectations before the first timeout. The whole service is about forty lines of FastAPI:

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class Ticket(BaseModel):
    text: str

@app.get("/health")
def health():
    return {"ok": True}

@app.post("/classify")
def classify(ticket: Ticket):
    # ask_free_model is pseudocode for your provider's chat completion call.
    label = ask_free_model(
        f"Classify this ticket as billing, bug, or feature: {ticket.text}"
    )
    return {"label": label}

I ran the experiment against MonkeyCode's free server and free model access, so the only thing I spent was attention. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Then I pointed a logger at the health endpoint and walked away for two days.

Hour 2: The First Wake-Up Tax

My first request came back in 11.4 seconds. The second one, sent a minute later, took 1.8 seconds. Same model, same prompt, same server, completely different universe, and the only variable was the silence in between.

That gap is the hidden variable in every free-server story, so I decided to measure it instead of guessing. The logger is deliberately dumb: every five minutes it records a timestamp, a latency, and a status. No retries, no backoff, no cleverness, because cleverness would have polluted the data.

# cold_start_logger.py — run it inside tmux, not a terminal tab
import json, time
from datetime import datetime, timezone
from urllib.request import urlopen

def probe(url: str, timeout: int = 30) -> dict:
    start = time.perf_counter()
    try:
        with urlopen(url, timeout=timeout) as resp:
            body = resp.read()
        return {"status": resp.status,
                "latency_s": round(time.perf_counter() - start, 3),
                "bytes": len(body)}
    except Exception as exc:
        return {"status": type(exc).__name__,
                "latency_s": round(time.perf_counter() - start, 3),
                "bytes": 0}

def run(url: str, interval: int, hours: float, log_path: str):
    deadline = time.time() + hours * 3600
    with open(log_path, "a") as fh:
        while time.time() < deadline:
            record = {"ts": datetime.now(timezone.utc).isoformat(),
                      **probe(url)}
            fh.write(json.dumps(record) + "\n")
            fh.flush()
            print(record)
            time.sleep(interval)

if __name__ == "__main__":
    run("https://your-free-server.example/health",
        interval=300, hours=48, log_path="cold_starts.jsonl")

Hour 6: What Broke First

I ran the logger in a terminal. Then I closed the terminal, because that is what humans do when they think a job is running. The logger died at hour six, and the gap in the log is the most honest data point of the whole experiment.

The fix: tmux

tmux new -s coldstart
python cold_start_logger.py --url https://your-free-server.example/health --interval 300 --hours 48 --log cold_starts.jsonl
# detach with Ctrl-b d, reattach with tmux attach -t coldstart

If you take one command from this article, take that one. A logger that dies with your laptop is not a logger.

Hour 12: The Pattern

Once the log had a few hundred rows, I stopped reading individual lines and started comparing each request with the gap since the previous one. That comparison is the whole trick.

The gap analysis

import json
from datetime import datetime

rows = [json.loads(line) for line in open("cold_starts.jsonl") if line.strip()]
rows.sort(key=lambda r: r["ts"])

GAP_THRESHOLD_S = 300  # 5+ minutes of silence counts as a cold start
cold, warm = [], []

for i in range(1, len(rows)):
    prev, cur = rows[i - 1], rows[i]
    gap = (datetime.fromisoformat(cur["ts"]) -
           datetime.fromisoformat(prev["ts"])).total_seconds()
    bucket = cold if gap >= GAP_THRESHOLD_S else warm
    if cur["status"] == 200:
        bucket.append(cur["latency_s"])

def pct(values, p):
    ordered = sorted(values)
    return ordered[min(len(ordered) - 1, int(len(ordered) * p))]

print(f"cold p50={pct(cold, 0.5):.2f}s p95={pct(cold, 0.95):.2f}s n={len(cold)}")
print(f"warm p50={pct(warm, 0.5):.2f}s p95={pct(warm, 0.95):.2f}s n={len(warm)}")

In my run, the cold p95 was several times the warm p95, while the medians looked almost friendly. That is the trap: averages hide the wake-up tax because most requests are warm, and the ones that matter are the ones that are not.

Hour 24: The Pre-Warm Experiment

Could I pay the wake-up tax before a human noticed it? I added a cheap health ping two seconds before the real request, and I want to be precise about what happened.

Pre-warm helped when the instance was merely dozing after a short gap. It did nothing when the instance had been recycled entirely, because some cold starts are not wake-ups at all; they are brand new containers that must boot, load the runtime, and answer from scratch. Pre-warm shrinks the first kind and cannot touch the second.

def warmed_call(base_url: str, payload: dict, timeout: int = 30):
    # Pre-warm is not a retry: one cheap request to leave idle,
    # then the real request once, no matter what the ping returned.
    # post_json is whatever HTTP client you already use.
    try:
        urlopen(f"{base_url}/health", timeout=5)
    except Exception:
        pass
    return post_json(f"{base_url}/classify", payload, timeout=timeout)

That distinction between dozing and recycled is the most useful thing I learned, and it explains why some retry logic feels magical while other retry logic just doubles your bill.

Hour 36: The Decision Table

By hour 36 I stopped asking whether the free server was fast and started asking whether my workload cared. The answer is a table, not a benchmark:

Workload shape	Free server + free model	Reach for paid
Batch job, no human waiting	Yes	No
Internal tool with seconds of slack	Yes, with pre-warm	No
User-facing request under a 2s budget	Only with queue and fallback	Yes
Bursty traffic after idle periods	No, cold starts amplify bursts	Yes
Prototype or weekend experiment	Yes	No

Who should not use this approach: anyone with a hard latency SLO, a real-time UI, or a contractual model-availability requirement. Free access is a budget, not a contract, and treating it like one is how outages start.

Hour 48: What I Would Repeat

If I ran the experiment again, I would keep four things:

• Log every request with a timestamp and the gap since the previous one, because the gap column is the hidden variable.
• Run the logger in tmux, not in a terminal tab, because my own impatience was the first failure.
• Separate dozing from recycled instances, because pre-warm only fixes one of them.
• Write the decision table before writing the code, because it forced me to define "good enough" before the first outage.

I would not repeat the keep-alive ping every thirty seconds. It worked, but it turned a free resource into a constant trickle of requests, and it taught me nothing that the gap analysis had not already shown.

Your numbers will be different, and that is the point. Providers change idle policies, model availability, and rate limits without notice, so treat this as a method rather than a benchmark of any specific service. If you run the same 48-hour log on your stack, I would genuinely like to see your cold p95 versus warm p95 — send the numbers, not the opinions.