Free-tier numbers are the fastest-rotting data in an architecture doc, so the workshop below treats every "free tokens" or "free server" line as a dated observation instead of a fact. You will leave with three files: a claims ledger, a probe that records what an endpoint actually reports, and a CI gate that fails when a claim goes stale. Total time is 70 minutes, and the artifact keeps working after the session ends.
The claim is not the problem; the missing timestamp is
Free tiers change quietly. A quota page gets rewritten without a changelog entry, an idle-suspension policy shifts from hours to minutes, and a "free" tier gains a card requirement in a footnote. None of those events breaks your build, which is exactly why nobody notices until a demo fails in front of a customer.
The usual fix is "research harder", and it does not scale. The fix that does scale is scheduled re-verification: every claim carries the date you read it, the URL you read it on, and a maximum acceptable age. When the age is exceeded, the pipeline complains before your users do.
The worked example: free model access and a free server option
MonkeyCode is the project we use as the worked example, because its offer combines two claims of different shapes: free model access, which you verify by calling a model, and a free server option, which you verify by deploying something and watching it. Per the operator, the current free tier includes a token allowance in the ten-million range, and the project is distributed as open source; both are claims you should re-read at the source before repeating them, which is the entire point of the exercises below.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Two honest caveats before you start. The scripts here are templates written against a generic HTTP API, and I did not execute them against MonkeyCode's endpoints while preparing this article, so the outputs are yours to produce and yours to trust. And no free-tier number belongs in a contract, a slide deck, or a customer-facing promise without its date attached.
Prerequisites and setup (5 minutes, before the clock starts)
- Python 3.11 or newer; the scripts use only the standard library so the CI image stays small.
- An API key exported as an environment variable, never in
claims.jsonand never in git. - A repository with any CI runner, plus write access to a
claims.jsonfile at the root. - One browser tab on the vendor's official pricing or docs page, not on a blog post about it.
Schedule
| Minutes | Block | Output |
|---|---|---|
| 0-5 | Setup check |
python -V, key exported |
| 5-13 | Exercise 1: claim inventory | 6-10 falsifiable sentences |
| 13-25 | Exercise 2: ledger |
claims.json committed |
| 25-45 | Exercise 3: probe |
observations.jsonl with real rows |
| 45-57 | Exercise 4: freshness gate | a red build on a stale claim |
| 57-67 | Exercise 5: decision table | one row per workload |
| 67-70 | Retro | one claim deleted as unfalsifiable |
Exercise 1: turn marketing sentences into falsifiable ones (8 min)
A claim you cannot test is a claim you cannot rely on. Rewrite each sentence until a script could return pass or fail, then attach the page you read it on.
- Weak: "generous free tier".
- Testable: "the free tier includes a token allowance; the current figure is published on the pricing page".
- Weak: "free server included".
- Testable: "a deployment target exists at zero cost; the docs state whether idle instances are suspended and after how long".
Write down what you cannot verify. Those rows are the valuable ones, because they mark the boundary of what you actually know rather than what you assume.
Exercise 2: encode the ledger (12 min)
claims.json is deliberately boring JSON, because YAML parsing would add a dependency to a gate that has to run everywhere, including on a minimal CI image.
[
{
"id": "free-token-allowance",
"claim": "Free tier includes a token allowance",
"source": "https://example.com/pricing",
"verified_at": "2026-10-19",
"max_age_days": 30,
"probe": "usage"
},
{
"id": "free-server-option",
"claim": "A zero-cost deployment target exists",
"source": "https://example.com/docs/deploy",
"verified_at": "2026-10-19",
"max_age_days": 30,
"probe": "cold-start"
}
]
Replace both URLs with the pages you actually read, and keep the wording vague on purpose: the ledger records the shape of the claim, while the specific number lives in the source you re-read.
Exercise 3: probe what the endpoint reports (20 min)
The probe answers one question: what does the service say about itself right now? It records status, latency, and any usage fields, and it never guesses on your behalf.
#!/usr/bin/env python3
"""Record what an endpoint reports about itself; append-only JSONL."""
import json, os, pathlib, time, urllib.request, urllib.error
from datetime import datetime, timezone
BASE = os.environ["MC_BASE_URL"].rstrip("/")
TOKEN = os.environ["MC_API_KEY"]
OUT = pathlib.Path("observations.jsonl")
USAGE_KEYS = ("usage", "quota", "limits", "balance")
def call(path, timeout=15):
req = urllib.request.Request(
f"{BASE}{path}",
headers={"Authorization": f"Bearer {TOKEN}", "Accept": "application/json"},
)
start = time.perf_counter()
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
return resp.status, round((time.perf_counter() - start) * 1000), resp.read().decode()
except urllib.error.HTTPError as err:
return err.code, round((time.perf_counter() - start) * 1000), err.read().decode()[:300]
def record(kind, **fields):
row = {"kind": kind, "observed_at": datetime.now(timezone.utc).isoformat(), **fields}
with OUT.open("a", encoding="utf-8") as fh:
fh.write(json.dumps(row, sort_keys=True) + "\n")
print(json.dumps(row, sort_keys=True))
def usage_snapshot(body):
try:
data = json.loads(body)
except json.JSONDecodeError:
return {"parsed": False, "bytes": len(body)}
found = {k: data[k] for k in USAGE_KEYS if k in data}
return {"parsed": True, "usage_fields": found, "keys": sorted(data)[:8]}
if __name__ == "__main__":
status, ms, body = call("/models")
record("models", status=status, latency_ms=ms, **usage_snapshot(body))
Run it twice and compare the rows. The first call after an idle window measures a cold path, and the second measures a warm one; merging them into a single latency number is how teams end up with dashboards that are wrong by an order of magnitude.
If no usage field appears in the response, that is a finding rather than a failure. Record {"usage_fields": {}} and note that the allowance can only be checked in a console, which means it cannot be monitored automatically yet.
Exercise 4: fail the build on stale claims (12 min)
#!/usr/bin/env python3
"""Exit non-zero when any recorded claim is older than its max_age_days."""
import json, pathlib, sys
from datetime import date
def check(path, today):
claims = json.loads(pathlib.Path(path).read_text(encoding="utf-8"))
stale = []
for claim in claims:
age = (today - date.fromisoformat(claim["verified_at"])).days
if age > claim["max_age_days"]:
stale.append((claim["id"], age, claim["source"]))
for cid, age, source in stale:
print(f"STALE {cid}: {age} days old, re-read {source}")
print(f"{len(claims) - len(stale)}/{len(claims)} claims fresh")
return 1 if stale else 0
if __name__ == "__main__":
sys.exit(check(sys.argv[1], date.today()))
Wire it into CI as a single step, and schedule the same command weekly so a quiet release branch still produces the nag.
- name: Free-tier claims are still fresh
run: python scripts/check_claims.py claims.json
To prove the gate works, set one verified_at to a date 60 days in the past and watch the build go red; then re-read the real page and set it back to today.
Exercise 5: decide per workload, not per vendor (10 min)
| Workload | Free tier acceptable? | Reason to record |
|---|---|---|
| Prototype with fake data, two developers | Yes | monthly re-verification is enough |
| Internal tool holding real customer names | Only with retention terms in writing | privacy review must cite a document |
| Latency-sensitive request path | Not as the only route | measure p95 cold and warm first |
| Nightly batch enrichment | Yes, with a local budget guard | a cap you enforce beats a cap you hope for |
| Anything with an audit requirement | No | free tiers rarely ship evidence |
Fill the table with your own workloads, and write the deciding sentence in the same row as the verdict. A table that says "yes" without a reason is a table nobody will defend in a review three months later.
Failure modes worth rehearsing
- 200 OK with an empty body. Status alone is not evidence; parse the payload and record the byte count.
-
Timezone drift. Store
verified_atas a plain date and compare dates, never naive datetimes. - Blog posts as sources. A tutorial quoting a quota is a rumour with good formatting; cite the docs page itself.
- Silent policy change. The gate catches this only if a human re-reads the page; a stale row is a task, not a bug.
- Probing too eagerly. One call per exercise is plenty; a freshness gate is not a load test.
Who should not use this approach
If you need a contractual uptime figure, a data-processing agreement, or evidence for an auditor, a free tier of any kind is the wrong substrate and no ledger will fix that. If your team cannot commit to a fifteen-minute re-read each month, the gate will quietly become noise that everyone mutes. And if the integration is a throwaway script you will delete next week, skip the ledger and write the date in a comment instead.
For everyone else the pattern transfers: paid tiers, rate limits, regional availability, and deprecation notices all decay on the same schedule as the free ones.
What you keep
Three files, one table, and a build that goes red before a customer does. The exercises are deliberately small so you can rerun them next month against whatever the documentation says then, including the free-tier numbers you started from here.
If you run this as a team session, the artifact worth comparing afterwards is your claims.json: the rows you deleted because nobody could phrase them falsifiably are usually the most interesting part of the discussion.
Top comments (0)