Most "down" alerts on a free-tier deployment are not bugs. They are a sleeping process, and a single probe cannot tell the difference. One request to a cold instance and one request to a broken instance look identical from the outside: same timeout, same 502 from the platform edge, same silence. The fix is not a better alert. It is a second request, fired a few seconds later, with the two timings compared.
This post is a probe ladder for that comparison, plus a decision table for deciding whether the free server stays in the stack. It is written for the solo-founder case: one person, one deploy, one bill that should stay at zero.
Why one probe always lies
A free server that sleeps usually answers the next request quickly, because the first one woke it. A genuinely broken deploy answers both requests the same way. So the useful signal is the delta: second_latency - first_latency, computed per run and aggregated across runs.
There is a second signal worth capturing at the same time. A cold instance has a fresh process identity. If your health endpoint returns booted_at and pid, a changed pid between two probes proves a restart, and an unchanged pid with a huge first latency proves a warm-up cost instead.
That distinction decides your product behaviour. Cold start means "show a warming state and keep the user." Crash means "fall back and stop calling." Getting it backwards costs you either a user or a quota.
Step 1: make the server state observable
Two endpoints, both trivial, both cheap. The point of /healthz is to be answered without touching any dependency, so it stays fast even when the app is unhealthy.
# app.py (FastAPI sketch)
import os, time
from fastapi import FastAPI
app = FastAPI()
BOOTED_AT = time.time()
@app.get("/healthz")
def healthz():
# No DB, no model call, no disk write. Must stay cheap.
return {"ok": True, "pid": os.getpid(), "booted_at": BOOTED_AT}
@app.get("/warm")
def warm():
# Touch exactly the resources the first real request will need.
db.execute("select 1")
return {"ok": True}
Keep /warm honest. If your first user request loads a config file and opens a DB pool, /warm should do both. A warm-up route that warms nothing is worse than no warm-up route, because it makes the probe report "warm" while the user still waits.
Step 2: classify, do not measure
Numbers alone do not tell you what to do at 3am. The probe below returns one of four verdicts and an exit code your scheduler can branch on. Thresholds come from a JSON file you fill in from your own runs — the values shown are placeholders, not measurements.
#!/usr/bin/env python3
# coldstart_probe.py -- classify a free-tier deployment in two requests.
import json, statistics, sys, time, urllib.error, urllib.request
CONFIG = json.load(open("probe_config.json"))
BASE = CONFIG["base_url"].rstrip("/")
COLD_MS = CONFIG["cold_ms"] # above this first-response time -> cold candidate
DEAD_MS = CONFIG["dead_ms"] # above this -> treat as dead
RUNS = CONFIG.get("runs", 5)
TIMEOUT_S = CONFIG.get("timeout_s", 10)
def hit(path):
url, t0 = BASE + path, time.monotonic()
try:
with urllib.request.urlopen(url, timeout=TIMEOUT_S) as r:
body = r.read()
return r.status, (time.monotonic() - t0) * 1000, body
except urllib.error.HTTPError as e:
return e.code, (time.monotonic() - t0) * 1000, b""
except Exception:
return 0, (time.monotonic() - t0) * 1000, b""
def classify(s1, ms1, s2, ms2):
if s1 == 200 and s2 == 200:
if ms1 > COLD_MS and ms2 < COLD_MS:
return "COLD" # woke up on the second request
return "WARM"
if s1 != 200 and s2 == 200:
return "COLD" # first call timed out while booting
if s1 != 200 and s2 != 200 and min(ms1, ms2) > DEAD_MS:
return "DEAD"
return "DEGRADED" # answered, but not with what we expect
Then loop RUNS times over /healthz followed immediately by /healthz again, plus one /warm, and print a JSON summary: median first latency, median second latency, verdict counts, and the fraction of runs that changed pid. Exit 0 for WARM, 2 for COLD, 3 for DEGRADED, 4 for DEAD. A scheduler that only understands zero and non-zero is a scheduler you will mute within a week.
Step 3: derive thresholds from your own data, not from this post
Run the probe on a schedule for a few days before you trust any branch. Use the observed distribution, not a round number someone typed into a blog: set COLD_MS above your p50 warm latency and below the fastest cold start you have actually seen, and set DEAD_MS above your slowest successful cold start. If those two windows overlap, your free server has no separable states and the ladder will misfire — that is itself a finding worth writing down.
Run the loop at a low frequency with jitter. Do not hammer a free endpoint every ten seconds and then describe the result as a load test; you are measuring your own impatience. Check the platform's terms on automated requests before you schedule anything, and count probe requests against whatever allowance the free tier gives you.
Step 4: gate the first AI call on the verdict
The probe is only worth building because it changes what the request path does. When the verdict is COLD, do not fire the expensive call and hope. Either return a small 202 with a retry hint, or serve the last cached answer if the feature allows it. When the verdict is DEAD, skip the call entirely and use the deterministic fallback.
MonkeyCode's free model access and free server option are what make this runnable without a card on file — operator-supplied availability, so verify the current terms, limits, and any allowances on the project's site rather than trusting a number in an article, including this one. The probe makes them survivable, not guaranteed.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
The decision table
| Observed pattern | What it means | Action |
|---|---|---|
| WARM most runs, COLD after idle gaps | Normal free-tier sleep | Keep it; gate first call behind a warming state |
| COLD on nearly every run | Instance never stays up, or limit is being hit | Reduce probe frequency, check quota, consider paid |
DEGRADED with 5xx on /warm only |
Dependency is down, app is fine | Fallback path; do not restart the app |
DEAD with no pid change |
Process hung, not sleeping | Restart; investigate logs before retrying |
| Latency rising week over week | You outgrew free | Move, or cut the feature |
Limitations, and who should skip this
This probe tells you what your deploy is doing. It does not add capacity, remove sleep, or create an SLA. Keep-alive traffic may breach platform terms, so read them first. Thresholds drift as your app grows, and a mis-set COLD_MS will silently downgrade a working deploy.
Skip this if you have paying users with a latency SLO, if a wrong "COLD" verdict would drop a request you already charged someone for, or if nobody is around to read a DEGRADED line. A solo founder shipping today should accept the limits loudly and keep the bill at zero — but only after measuring which limit is actually biting.
If you want to run this end to end, deploy the two endpoints and point the probe at your own free instance; the numbers you get back are the only ones worth designing around.
Top comments (1)
One read telling you a lie is exactly how I cost someone a bad answer on a static deploy. Compared the live file's hash to my build, saw a mismatch, and reported it as not live. It was live; the host served a stale copy for a few seconds. A second read with a cache-buster matched. Your delta idea generalizes: before you classify something as broken, ask what a second, slightly different read says.