DEV Community

Avery Lin
Avery Lin

Posted on

Write the Probe Before the Prompt

The kitchen light hummed over a cheap laptop. A solo founder stared at a half-built checkout. Friday night still refused to become a launch.

Chat suggested a complete Node payment service. The files even included a README with badges. None of those badges had ever been run.

This is the usual indie trap now. Vibe output mimics engineering without a spine. Calling the paste a product invites silent failure.

The founder needed a boss that cannot flatter. A human reviewer gets tired and polite. A probe script does not get polite.

The method is small and slightly rude. Write a contract file before any model prompt. Point a shell probe at that frozen contract.

Treat a green probe as the only ship signal. Think of the probe as a night watchman. The model is a painter working in the dark.

The watchman cares about doors, not brushwork. Pretty handlers do not open locked doors. A 200 on the wrong shape still fails.

Freeze the door

The contract lives in the repo as JSON. It names paths, status codes, and required keys. The founder types this file by hand.

{
  "base": "http://127.0.0.1:8787",
  "checks": [
    {
      "name": "health",
      "method": "GET",
      "path": "/health",
      "status": 200,
      "json": { "ok": true, "service": "zine-checkout" }
    },
    {
      "name": "create_order",
      "method": "POST",
      "path": "/orders",
      "headers": { "content-type": "application/json" },
      "body": { "sku": "zine-01", "qty": 1 },
      "status": 201,
      "json": { "id": "*", "sku": "zine-01", "qty": 1, "paid": false }
    },
    {
      "name": "unknown_sku",
      "method": "POST",
      "path": "/orders",
      "body": { "sku": "nope", "qty": 1 },
      "status": 404,
      "json": { "error": "unknown_sku" }
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

That file is not a prompt at all. The model may never edit those keys. If a key moves, the founder moved it.

A star in a JSON value means any scalar. It is a hole for generated ids only. Extra keys in the response still fail the check.

This is the opposite of a vibe spec. The door is smaller than the model's appetite. Small doors are how a solo SKU survives.

Hire a rude watchman

A tiny helper loads the contract without drama. Python is enough on a tired Friday. The helper prints one line per check.

# probe.py — labeled template, run against your own stub
import json, os, sys, urllib.request

def must_match(expected, actual, path="$"):
    if expected == "*":
        if isinstance(actual, (str, int, float, bool)):
            return
        raise AssertionError(f"{path} wanted a scalar")
    if type(expected) is not type(actual):
        raise AssertionError(f"{path} type {type(actual)}")
    if isinstance(expected, dict):
        extra = set(actual) - set(expected)
        if extra:
            raise AssertionError(f"{path} extra keys {sorted(extra)}")
        for key, value in expected.items():
            if key not in actual:
                raise AssertionError(f"{path}.{key} missing")
            must_match(value, actual[key], f"{path}.{key}")
        return
    if expected != actual:
        raise AssertionError(f"{path} {actual!r} != {expected!r}")

def call(base, check):
    url = base.rstrip("/") + check["path"]
    data = json.dumps(check.get("body")).encode() if "body" in check else None
    req = urllib.request.Request(url, data=data, method=check["method"])
    for key, value in check.get("headers", {}).items():
        req.add_header(key, value)
    try:
        with urllib.request.urlopen(req, timeout=5) as res:
            raw, status = res.read(), res.status
    except urllib.error.HTTPError as err:
        raw, status = err.read(), err.code
    body = json.loads(raw.decode() or "null")
    if status != check["status"]:
        raise AssertionError(f"{check['name']} status {status}")
    must_match(check.get("json"), body)
    print(f"PASS {check['name']}")

def main():
    spec = json.load(open("contract.json"))
    base = os.environ.get("CONTRACT_BASE", spec["base"])
    for check in spec["checks"]:
        call(base, check)
    print("ALL PASS")

if __name__ == "__main__":
    try:
        main()
    except Exception as err:
        print(f"FAIL {err}", file=sys.stderr)
        sys.exit(1)
Enter fullscreen mode Exit fullscreen mode

The shell wrapper keeps the ritual stupid. It starts no server and tells no jokes. It only exits zero on a full pass.

#!/bin/sh
# probe.sh
set -eu
python3 probe.py
Enter fullscreen mode Exit fullscreen mode

The first server is an empty hallway. It binds a port and returns honest 501s. The probe should fail in a known way.

// stub.mjs — labeled starter, not a measured benchmark
import http from "node:http";

const port = Number(process.env.PORT || 8787);
const server = http.createServer((req, res) => {
  res.setHeader("content-type", "application/json");
  res.statusCode = 501;
  res.end(JSON.stringify({ error: "not_implemented", path: req.url }));
});

server.listen(port, "0.0.0.0", () => {
  console.log(`stub on ${port}`);
});
Enter fullscreen mode Exit fullscreen mode

Run the stub, then run the probe once. The failure is the first honest artifact. A red watchman is still doing the job.

chmod +x probe.sh
node stub.mjs &
export CONTRACT_BASE=http://127.0.0.1:8787
./probe.sh || true
Enter fullscreen mode Exit fullscreen mode

Then let the intern paint

Now the founder opens a model chat. The prompt is short and almost rude. Implement the contract. Do not add routes.

Do not add extra packages. Do not rewrite the probe. Stop when probe.sh exits zero.

This is where a zero invoice still matters. A solo founder cannot fund a private lab. MonkeyCode offers free model access and a free server option for this loop. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The model fills handlers against frozen JSON. The founder runs the probe on localhost. Red output is cheaper than a fake launch.

Paste only the stub, the contract, and the probe. Leave the README badges in the trash. The watchman does not read marketing copy.

When local checks pass, the same probe travels. Point CONTRACT_BASE at the free server URL. Run the identical script without a rewrite.

# after the stub is copied onto the free server
export CONTRACT_BASE=https://your-free-host.example
./probe.sh
Enter fullscreen mode Exit fullscreen mode

The second environment is not a second product. It is the same door, different hallway. If the probe disagrees, the ship waits.

A Makefile holds the ritual in four targets. Indie memory fades after one long week. Make becomes the Friday checklist without a speech.

PORT ?= 8787
CONTRACT_BASE ?= http://127.0.0.1:$(PORT)

stub:
    node stub.mjs

probe:
    CONTRACT_BASE=$(CONTRACT_BASE) python3 probe.py

shipcheck:
    @test -n "$(REMOTE)" || (echo "REMOTE is empty" && exit 1)
    CONTRACT_BASE=$(REMOTE) python3 probe.py

diff-contract:
    git diff --exit-code -- contract.json
Enter fullscreen mode Exit fullscreen mode

Notice shipcheck never says deploy magic. It curls the remote base and stops talking. Marketing pages do not appear in that target.

diff-contract is the quiet lock on the door. A model that rewrites the spec fails git. The founder, not the intern, owns the hinges.

What the watchman cannot see

Limitations walk in after the first green run. Free model access is not a named brain. Free server capacity is not a published SLA.

Quotas, hardware, and lifetime are not claimed here. Those numbers change and should be checked live. This article does not freeze a vendor card.

The probe cannot see race conditions well. It cannot prove tax logic or card handling. It cannot replace a human on refunds.

Flaky networks will shame a good probe. One timeout is not a product verdict. Retry once, then read the body.

A JSON equality check is a blunt instrument. Nested lists and streaming bodies need extra care. Do not pretend this file is a test suite.

It is a door test for a one-person shop. That is useful and also narrow. Narrow tools keep Friday nights finite.

Who should walk away

Teams with shared on-call should not start here. Regulated ledgers should not either. High traffic stores should not start here.

Multi-tenant secrets do not belong on a free box. A founder selling hope should not either. If the SKU needs an audit, hire one.

The approach fits one person and one SKU. A zine, a waitlist, a webhook is enough. The bill stays zero while the door is tested.

Accept the limits in writing before the chat. No custom domains in the first loop. No payment capture until the probe is boring.

Engineering is the probe, not the chat log. Vibe coding can still paint the hallway. The watchman still owns the keys.

Keep the contract in version control forever. When the model invents a new field, refuse. When the probe stays green, ship the zine.

Solo founders who want that zero-invoice first loop can try the free model path and the free server option, then leave the probe in charge.

Top comments (0)