Free Server, Free Model: Four Assumptions That Break Your First Deploy (And a Probe Script)

Your demo works on localhost. You push it to a free server. It dies.

Why? Because the free box is not your laptop. It has different rules.

I keep seeing the same four assumptions in review threads. Each one is checkable in minutes. None of them needs a benchmark suite.

This post covers free model access plus a free server option, both operator-supplied availability claims. I will not quote quotas, RAM, or uptime. Those change, and stale numbers are worse than no numbers. Probe your own host instead.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Myth 1: "It runs locally, so it will run there"

Local means a persistent disk, a warm process, and no cold start.

A free host often gives you none of those. The filesystem may be wiped between restarts. The process may sleep when idle.

Here is the test. Write a file, restart, read it back.

# run inside the app or via your host's console
marker="/tmp/probe-$(date +%s)"
echo "$(date -u +%FT%TZ)" > "$marker"
ls -l "$marker"
# now restart the service, then:
cat "$marker" || echo "GONE: filesystem did not survive restart"

If you get GONE, your SQLite file is a lie. Your uploaded files are a lie. Your queue is a lie.

The corrected model: treat local disk as a cache, not a database. Keep truth somewhere that outlives the process.

Where should state actually live?

State	Wrong place	Right place
Session tokens	JSON file on disk	Signed cookie or external store
Job queue	In-memory list	Durable table with a claimed_at column
Uploads	Local volume	Object storage
Model output cache	Process dict	Table keyed by prompt hash
Config	.env baked at build	Runtime env from the host

That last row bites people. If config is baked into the image, a restart cannot fix a bad value.

Myth 2: "A 200 from /healthz means the app is warm"

A health check answers one question. Is the process up?

It does not answer: can this process reach the model endpoint? Does it have memory left? Can it serve a real request in under five seconds?

So measure the thing you actually care about. Cold start, not liveness.

#!/usr/bin/env sh
# coldstart-probe.sh — run from your laptop against the deployed app
set -eu

APP_URL="${APP_URL:?set APP_URL}"
MODEL_URL="${MODEL_URL:?set MODEL_URL}"

probe() {
  start=$(date +%s%N)
  code=$(curl -s -o /dev/null -w '%{http_code}' --max-time 90 "$1")
  end=$(date +%s%N)
  echo "$2 status=$code ms=$(( (end - start) / 1000000 ))"
}

echo "== cold start samples (idle 60s between runs) =="
for i in 1 2 3; do
  probe "$APP_URL" "app-run-$i"
  sleep 60
done

echo "== can the app reach the model endpoint? =="
probe "$MODEL_URL" "model-endpoint"

Run it three times with idle gaps. A sleeping instance shows a fat first sample and thin later ones. That gap is your real user experience.

Why idle gaps? Because a warm host hides everything. You want the cold path.

And check memory from inside the box, not from a dashboard guess.

cat /sys/fs/cgroup/memory.max 2>/dev/null \
  || cat /sys/fs/cgroup/memory/memory.limit_in_bytes 2>/dev/null \
  || echo "no cgroup limit file visible"

If that number is smaller than your model client plus your web framework, you have a design problem, not a hosting problem.

Myth 3: "Retries are free because the model is free"

Free model access removes the bill. It does not remove the request.

Every retry costs a round trip, a cold-start risk, and a slot in whatever concurrency you have. Stack three retries on two layers and one click becomes nine calls.

Do the arithmetic before you add a retry:

attempts = a ^ layers        # a = retries per layer
2 layers, 3 attempts each -> 9 calls per user action
3 layers, 3 attempts each -> 27 calls per user action

So cap it. One retry policy per request path. Log the attempt count next to the latency.

# pseudocode — not executed here
for attempt in range(MAX_ATTEMPTS):
    try:
        return call_model(prompt, timeout=BUDGET_LEFT)
    except TransientError:
        log.warning("model_retry", attempt=attempt, budget_ms=BUDGET_LEFT)
        continue
raise BudgetExhausted("no attempts left inside request budget")

Notice the budget, not the count, drives the loop. A request has a deadline. Retries spend it.

Myth 4: "Restarts are rare, so I can ignore them"

Free hosts restart for reasons you do not control. Deploys, idle sleep, maintenance, memory pressure.

So the interesting question is not whether a restart happens. It is what your app does in the first two seconds after one.

Test it on purpose. Kill the service. Then hit /healthz and a real route.

# simulate: restart the service on the host, then from your laptop
curl -s -o /dev/null -w 'health %{http_code}\n' "$APP_URL/healthz"
curl -s -o /dev/null -w 'real   %{http_code}\n' "$APP_URL/api/chat?q=hi"

If health is 200 and the real route is 500, your readiness signal is fiction. Fix the route, not the check.

A short readiness checklist

1. Restart the service on purpose.
2. Hit the real route within five seconds.
3. Confirm no lost writes on the previous request.
4. Confirm config came from the environment, not the image.
5. Confirm the model call still resolves after the restart.

Five steps. Ten minutes. It finds more bugs than a week of dashboards.

Who should not use this approach

This is a fit for prototypes, internal tools, and low-traffic side projects. That is where free model access plus a free server earns its keep.

It is a poor fit for anything with an SLA. If you need predictable p99, a fixed memory ceiling, or long-running background workers, pay for a host that guarantees them.

Also skip this if you cannot tolerate cold starts at all. No probe script fixes a sleeping instance for a user who expects instant replies.

And if your data has retention rules, check the host's current terms yourself. I am not your compliance source.

The corrected mental model

A free server is a shared, restartable, memory-capped process. A free model endpoint is a network dependency you do not run.

Treat both as unreliable by default. Then design the small parts you control: durable state, one retry budget, a real readiness route, and config from the environment.

I did not publish my probe numbers here. Yours will differ, and mine would age badly. Run the script, keep the output in your repo, and re-run it after every host change.

That file is worth more than any claim in this post, including mine.

If you try it on a free MonkeyCode server with free model access, keep the output and compare it against your next host. The diff is the lesson.