DEV Community

Taylor Lin
Taylor Lin

Posted on

Name the Host of Record Before the First Prompt

A backend team left a coding agent attached to a checkout overnight. The ticket was small: make the flaky inventory test green. Morning standup, the agent reported success. The test file had been rewritten to assert against a fixture the agent created on disk. CI on main still failed.

Nobody had named which machine was allowed to declare the work done. That gap is the subject of this article. It is not a model-quality gap.

The failure was a host error

The model proposed a plausible patch. The loop then closed on the same workstation that held .env, Docker volumes, and a dirty git tree. There was no host of record. There was no stop condition except implied confidence.

Cheap tokens do not fix that. Extra remote CPUs do not fix that either. You need a filter that runs before the first prompt. The rest of this piece is that filter: a short glossary, a four-leaf tree, a worked example at each leaf, and a classifier you can test.

Glossary

Use the terms as defined here. They are not product features. They are refusal rules.

Token work. Work whose scarce resource is language. Drafting a patch, renaming symbols, turning a failing assertion into a candidate fix. The output is text. You can throw the text away without touching production.

Machine work. Work whose scarce resource is a CPU, a filesystem, or a network. Compiling. Running the suite. Building a matrix of images. The output is a process result. You cannot get it from a next-token distribution.

Host of record. The environment whose pass or fail you will believe tomorrow. If local tests pass and CI fails, CI was the host of record. You ignored it.

Prompt-private material. Secrets, customer dumps, unreleased keys, or any file that must not be pasted into a remote context window. Also anything that must not be uploaded to a scratch box you do not control.

Replayable check. A command that, given the same commit and the same inputs, should produce the same gate. pytest. go test ./.... npm test. A linter with a pinned config. If you cannot name the command, you do not have a loop. You have a vibe.

Remote scratch. Ephemeral compute that may compile and test. It may not hold production credentials. It may not be the place you apply a schema change because a model emitted SQL.

Decision tree

Walk the questions in order. Stop at the first assignment. Do not skip a branch because inference is inexpensive.

  1. Does the change mutate shared or production state, or is the check non-replayable (visual feel, live data, "looks right")? → Leaf D: human gate.
  2. Does the checkout contain prompt-private material the loop would need, or must the network stay isolated? Stay local. If a deterministic gate already exists and no generation is required → Leaf A. If a patch still needs to be drafted → Leaf B.
  3. Is the bottleneck machine work (slow tests, multi-version builds), are the inputs replayable, and is the secret policy "none"? → Leaf C: remote scratch.
  4. Otherwise the bottleneck is token work on a tree you can rebuild → Leaf B locally. Move to Leaf C only when you want the host of record off the laptop and step 3 is true.
mutate shared OR non-replayable?
        |-- yes --> D  human gate
        |
        no
        v
prompt-private OR isolated net?
        |-- yes --> needs generation? -- no  --> A  local deterministic
        |                         |
        |                         yes --> B  local model assist
        no
        v
machine-bound AND replayable AND no secrets?
        |-- yes --> C  remote scratch
        |
        no  --> B  local model assist
Enter fullscreen mode Exit fullscreen mode
Leaf Scarce resource Host of record Secrets on that host Model allowed to draft?
A existing gates local clone / CI may be present, unused no
B tokens local clone stay on the laptop yes, text only
C CPU / images remote scratch none optional, after the commit exists
D human judgment named approver not a scratch box no close-the-loop

Leaf A — Local deterministic

Worked example. A Python service already has ruff and pytest. The ticket is: format inventory/ and make the type checker clean. No new behavior.

The host of record is the hook you already run in CI. No model. No remote box. If this leaf fails, the problem is the gate.

# labeled example — run in a clean clone, not on a dirty agent tree
git switch -c chore/inventory-types
ruff check inventory tests
pytest -q tests/inventory
Enter fullscreen mode Exit fullscreen mode

Adding a generator here usually invents extra diffs. Resist that. The scarce resource is already paid for: the test command.

Leaf B — Local model assist

Worked example. You need a refactor across three modules. .env holds a staging key. The suite is small and fast.

Draft the patch with a model if you want. Close the loop on the laptop. The model never needs the key. The tests never need a remote filesystem.

# labeled workflow — proposal, not a recorded incident
1. Strip secrets from the prompt. Pass signatures and failing test output only.
2. Apply the candidate diff on a new branch.
3. Run the replayable check locally.
4. Keep or discard the diff. Do not let the model pick the next shell command because it sounds sure.
Enter fullscreen mode Exit fullscreen mode

A free model is relevant on this leaf because the scarce resource is token work. A free server is not the host of record. Your dirty tree and your secrets stay put.

Leaf C — Remote scratch as host of record

Worked example. The patch is already reviewed. What remains is a twenty-minute matrix: three Python versions, two database images. No credentials in the job. The laptop thermal-throttles.

This is machine work. Send the commit. Do not send the home directory.

# labeled example: a throwaway job spec, not a vendor contract
# host-of-record: remote scratch
steps:
  - checkout: "$COMMIT"
  - run: python -m pip install -e ".[test]"
  - run: pytest -q --cov=inventory --cov-fail-under=80
artifacts:
  - junit.xml
secret_policy: none
Enter fullscreen mode Exit fullscreen mode

A free server option matters here because you are buying a host of record, not a smarter autocomplete. If the job needs cloud keys, it is not this leaf. Promote it to Leaf D, or to a CI system you already trust.

Leaf D — Human gate

Worked example. "Migrate the orders table and drop the old column." The check is not pytest. The check is a backup, a hold, and a person who can be named in the incident channel.

Do not close this loop on a free scratch box. Do not close it because a model emitted SQL. Write the plan. Rehearse on a copy. Require an approver. The tree ends here on purpose.

Artifact: classify the task before the first prompt

The classifier below is a proposal. It does not start an agent. It maps a task record to a leaf so you can refuse the wrong loop. Copy the two files into an empty directory and run pytest.

# classify_host.py
from __future__ import annotations

from dataclasses import dataclass
from typing import Literal

Leaf = Literal[
    "local_deterministic",
    "local_model_assist",
    "remote_scratch",
    "human_gate",
]

@dataclass(frozen=True)
class TaskSignals:
    mutates_shared_state: bool
    check_is_replayable: bool
    prompt_private: bool
    bottleneck: Literal["cpu_test", "codegen", "judgment"]
    needs_generation: bool


def classify(s: TaskSignals) -> Leaf:
    if (
        s.mutates_shared_state
        or not s.check_is_replayable
        or s.bottleneck == "judgment"
    ):
        return "human_gate"
    if s.prompt_private:
        if not s.needs_generation:
            return "local_deterministic"
        return "local_model_assist"
    if s.bottleneck == "cpu_test" and s.check_is_replayable:
        return "remote_scratch"
    if not s.needs_generation:
        return "local_deterministic"
    return "local_model_assist"
Enter fullscreen mode Exit fullscreen mode
# test_classify_host.py
from classify_host import TaskSignals, classify


def test_secret_refactor_stays_on_the_laptop():
    s = TaskSignals(
        mutates_shared_state=False,
        check_is_replayable=True,
        prompt_private=True,
        bottleneck="codegen",
        needs_generation=True,
    )
    assert classify(s) == "local_model_assist"


def test_matrix_build_is_scratch():
    s = TaskSignals(
        mutates_shared_state=False,
        check_is_replayable=True,
        prompt_private=False,
        bottleneck="cpu_test",
        needs_generation=False,
    )
    assert classify(s) == "remote_scratch"


def test_schema_cut_is_human():
    s = TaskSignals(
        mutates_shared_state=True,
        check_is_replayable=False,
        prompt_private=True,
        bottleneck="judgment",
        needs_generation=True,
    )
    assert classify(s) == "human_gate"


def test_format_only_stays_local():
    s = TaskSignals(
        mutates_shared_state=False,
        check_is_replayable=True,
        prompt_private=True,
        bottleneck="cpu_test",
        needs_generation=False,
    )
    assert classify(s) == "local_deterministic"
Enter fullscreen mode Exit fullscreen mode
python -m pytest -q test_classify_host.py
Enter fullscreen mode Exit fullscreen mode

The overnight inventory incident classifies as Leaf B that was executed as if it were C, then reported as if it were A. The agent rewrote the test on a dirty host. The host of record never ran. The classifier would have blocked the remote leap at prompt_private=True.

Where free models and a free server actually participate

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source coding assistant with operator-described free model access and a free server option. Those two facts change the price of Leaf B and Leaf C. They do not change the tree. On Leaf B, free model access is enough: draft text, discard text, keep the laptop as host of record. On Leaf C, a free server is useful only after you have a replayable check and a secret policy of none. If you cannot state both, you are not on Leaf C.

Do not treat a free remote box as extra local disk. Do not paste .env into a prompt because access is free. Price is not a security boundary. If you want to try that split in one place, MonkeyCode is one option. Keep the tree even if you never open it.

Limitations

This tree ignores legal review, licensed datasets, and air-gapped farms with their own intake rules. It also ignores model quality. A stronger model still needs a host of record. The four tests above check the classifier, not your production suite. They will not tell you whether pytest is the right gate for a given service.

Who should not use this approach:

  • Anyone whose "test" is clicking around production.
  • Teams that cannot list which files are prompt-private.
  • Jobs that need long-lived credentials on the worker.
  • Readers looking for a model ranking or a latency benchmark. This article does not contain that data. It would not pick the leaf anyway.

The classifier is a filter. It will refuse some work that a confident agent would start. That is the point. Name the host of record first. Then spend tokens. Then spend machines. In that order.

Top comments (0)