DEV Community

Jordan Huang
Jordan Huang

Posted on

Five Free-Stack Myths That Survive the First Green Run

Did that green check prove the work?
Or did it only prove the box was still up?

I keep hearing the same five claims.
They arrive with every complimentary model drop.
A free server makes the story louder, not truer.

Why this FAQ exists

I am not scoring models today.
I am scoring the story around them.

A free model changes the invoice only.
It does not change filesystem truth.
A free server changes where the shell runs.
It does not mint provenance for you.

Would you merge a vibe?
Then stop merging a complimentary stack without a receipt.

What I actually challenge

I ask four questions before I trust a run.

  1. What task contract did the agent accept?
  2. What image or host did the shell boot?
  3. What stop rule ended the loop?
  4. What file left the box besides chat text?

If any answer is a shrug, the claim is a myth.
I write the shrug down.
Then I run the boring checks.

When I need complimentary inference and a complimentary remote box in one loop, I use MonkeyCode's free model access and free server option as the scratch room. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The product is the room.
The receipt is still my job.

Myth 1: "Free means I can skip pinning."

People say the stack does not matter here.
The model will rediscover the toolchain anyway.
Why freeze versions on a throwaway box?

Throwaway is not the same as undefined.
Undefined is how you get two greens.
Those greens hide two different trees.

Evidence I look for

  • A lockfile or an image digest in the log
  • The exact installer command, not a paraphrase
  • A hash of python -V and node -v

No pin, no comparison.
You cannot replay a shrug tomorrow.

Corrected mental model

Treat the free box like a crime scene.
Freeze the obvious tools first.
Then let the agent move files.

# proposed pin snapshot — label: unexecuted example
set -euo pipefail
python -V | tee /tmp/py.version
command -v pip >/dev/null && pip freeze | sha256sum | tee /tmp/pip.sha
uname -a | tee /tmp/uname.txt
date -u +%Y-%m-%dT%H:%M:%SZ | tee /tmp/snap.utc
Enter fullscreen mode Exit fullscreen mode

Would you merge a PR with no lockfile?
Then do not bless a free-stack story without one.

Myth 2: "Green on the free server equals green in CI."

This one sticks to people.
The agent printed that tests passed.
So CI should pass, right?

Wrong layer.
The free box is a host.
CI is a contract with another image.

Chat said "all tests passed" is not a runner.
It is a sentence with confidence.
Sentences do not ship.

Evidence I look for

  • CI image name and digest, not a nickname
  • The same test command, byte for byte
  • A diff of two environment dumps

If those three are missing, you have a souvenir.
You do not have a promotion path.

Corrected mental model

Promote a command, not a feeling.
Copy the exact invocation into CI first.
Argue about models after that copy works.

# proposed CI fragment — label: unexecuted example
# Keep this honest: only run what your runner already supports.
steps:
  - name: require pin files from the free box
    run: |
      test -f /tmp/pip.sha || test -f pin/pip.sha
      test -f receipt.json
  - name: replay the same test command
    run: pytest -q tests/test_receipt.py
Enter fullscreen mode Exit fullscreen mode

If CI cannot see the pin files, the green run stayed on vacation.
Did you ship the souvenir, or the command?

Myth 3: "Retries are free, so more retries are better."

Cost zero feels like permission.
People delete the kill switch.
They let the agent retry until boredom.

Free is not infinite.
Free is not supervised.
A retry storm is still a hang with extra tokens.

Did the agent converge?
Or did you fund a loop because the invoice was quiet?

Evidence I look for

  • A max-step integer in the wrapper
  • A wall-clock timeout around the shell
  • A done predicate that is not model prose
  • A named reason for the last retry

"It seemed done" is not a reason.
"The box was free" is not a reason either.

Corrected mental model

Budget steps even when the invoice is empty.
I wrap the loop before I wrap the model.
I name the stop, every time.

# proposed stop wrapper — label: unexecuted example
MAX_STEPS = 8
WALL_SECONDS = 120
MAX_RETRIES = 2

def should_stop(step, elapsed, retries, done_flag):
    if done_flag:
        return "task_predicate"
    if retries > MAX_RETRIES:
        return "retry_budget"
    if step >= MAX_STEPS:
        return "max_steps"
    if elapsed >= WALL_SECONDS:
        return "wall_clock"
    return None
Enter fullscreen mode Exit fullscreen mode

Print should_stop into the receipt.
If the field is empty, the myth won that run.

Myth 4: "The default user on a free server is fine."

It is a scratch box, people say.
Who cares which uid ran pip?
The files are temporary, right?

Temporary for you, maybe.
Not always for the next process.
Not if a token landed in .bashrc.

A free server is still a principal.
Principals need a boundary.
Chat does not provide one.

Evidence I look for

  • id output copied into the receipt
  • Whether secrets were copied onto the box
  • Whether the agent could write outside the project dir
  • A scan for .env, *.pem, and id_rsa

If that scan prints anything, stop the demo.
Do not debug your identity on a shared disk.

Corrected mental model

Assume the default user is too wide.
Give the agent one working directory.
Refuse to paste credentials into that shell.

# proposed boundary check — label: unexecuted example
id
pwd
expected="$HOME/work/task"
test "$(pwd)" = "$expected" || exit 17
find . -name ".env" -o -name "*.pem" -o -name "id_rsa" | tee /tmp/secret-scan.txt
test ! -s /tmp/secret-scan.txt
Enter fullscreen mode Exit fullscreen mode

Would you hand a stranger your laptop user?
Then do not hand an agent the default login and call it research.

Myth 5: "Exploration does not need a receipt."

This myth feeds the other four.
People say they were only looking around.
So logs become optional.
So the chat window becomes the notebook.

Then someone asks what you learned.
You paste a screenshot.
That is not a lesson.
That is a mood with syntax highlighting.

If you cannot replay the look-around, you did not look.
You wandered with a model.
Wandering does not compound.

Evidence I look for

  • A one-page receipt file on disk
  • The task prompt, hashed, not pasted raw if it is sensitive
  • The final artifact path and checksum
  • A stop reason that a teammate can read

Corrected mental model

Exploration still writes a file.
The file is small.
The file is boring.
That boredom is the point.

The artifact: one JSON receipt per run

I keep one document per attempt.
Not a blog post.
A receipt.

# proposed receipt writer — label: unexecuted example
import hashlib, json, os, time, subprocess, pathlib

def sh(cmd):
    p = subprocess.run(cmd, shell=True, capture_output=True, text=True)
    return {"cmd": cmd, "code": p.returncode, "out": (p.stdout or "")[-2000:]}

def main():
    task = os.environ.get("TASK_CONTRACT", "").encode()
    receipt = {
        "utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
        "task_sha256": hashlib.sha256(task).hexdigest() if task else None,
        "cwd": os.getcwd(),
        "id": sh("id"),
        "python": sh("python -V"),
        "uname": sh("uname -a"),
        "git": sh("git rev-parse --is-inside-work-tree && git rev-parse HEAD"),
        "stop_reason": os.environ.get("STOP_REASON"),
        "retry_count": os.environ.get("RETRY_COUNT"),
        "artifact": os.environ.get("ARTIFACT_PATH"),
    }
    path = receipt["artifact"]
    if path and pathlib.Path(path).is_file():
        digest = hashlib.sha256(pathlib.Path(path).read_bytes()).hexdigest()
        receipt["artifact_sha256"] = digest
    pathlib.Path("receipt.json").write_text(json.dumps(receipt, indent=2))
    print("wrote receipt.json")

if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

Set TASK_CONTRACT before the agent starts.
Set STOP_REASON when the wrapper exits.
Set RETRY_COUNT from the loop, not from memory.
Set ARTIFACT_PATH only if a real file exists.

Then keep receipt.json or delete it on purpose.
Do not lose it in the chat scroll.

A tiny test plan for the receipt

I do not trust a writer I will not test.
These checks are proposals.
Label them unexecuted until you run them.

# proposed tests/test_receipt.py — label: unexecuted example
import json, os, pathlib, importlib.util

def load_writer(path="write_receipt.py"):
    spec = importlib.util.spec_from_file_location("write_receipt", path)
    mod = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(mod)
    return mod

def test_receipt_requires_stop_reason(tmp_path, monkeypatch):
    monkeypatch.chdir(tmp_path)
    monkeypatch.setenv("TASK_CONTRACT", "hash this sentence")
    monkeypatch.setenv("STOP_REASON", "max_steps")
    monkeypatch.delenv("ARTIFACT_PATH", raising=False)
    load_writer().main()
    data = json.loads(pathlib.Path("receipt.json").read_text())
    assert data["stop_reason"] == "max_steps"
    assert data["task_sha256"]
    assert data["artifact_sha256"] is None if "artifact_sha256" not in data else True
Enter fullscreen mode Exit fullscreen mode

If the stop reason is missing, fail the test.
If the artifact path points at chat text, fail the test.
If cwd is $HOME, fail the test.

That is the whole point of a receipt.
It should be able to fail.

Decision table I tick after a run

Claim you heard What would make it true What I record when it is false
Pinning is optional on a free box Replay matched on a second host Missing lockfile or digest
Free green equals CI green Same command, same image digest Host-only pytest output
More retries are better because free Stop reason is a predicate Retry storm or closed lid
Default user is harmless No secrets, tight working dir .env hits or home-dir writes
Exploration needs no receipt You can answer "what changed?" Screenshot-only memory

Print the table.
Tick one row per run.
If you cannot tick a row, the myth won.

Limitations

This FAQ does not benchmark models.
It does not name quotas or hardware.
It does not promise a server will stay up.

The receipt is only as honest as the host.
A hostile box can fake uname.
A model can invent a passing test in prose.

I have not executed these snippets on your machine.
They are proposals.
Read them before you paste them.

Short receipts miss scheduler noise.
They miss noisy neighbors.
They miss network policy you never dumped.

Who should not use this

Do not use a free shared server for customer data.
Do not put production secrets in that shell.
Do not treat this receipt as a regulated audit.

If your team already has locked CI and signed runners, you do not need this as a platform.
Use it as a pre-flight only.
If policy forbids unknown remote boxes, listen to that policy.

If you cannot write a task contract in one paragraph, stop.
The model will not invent your acceptance tests because inference was complimentary.

What I do after I hear a myth

I pick one claim.
I run one receipt.
I keep the JSON.

The interesting part is never the complimentary inference.
The interesting part is the file you can hash tomorrow.

Would you defend that green run in a review?
If not, it was not a run.
It was a story that got a free box.

Top comments (0)