Write the Kill Envelope Before the First Prompt

Free tokens do not cancel occupancy. If an agent loop can still be running at dawn, you did not receive a discount. You rented a clock and forgot to cap it.

Token count is a buffet sign. Occupancy is the parking meter outside. You can eat for zero and still block the only stall that matters. On shared or spare capacity that stall is the server, the queue slot, and the context window you keep resending while nothing useful happens.

You already know the loud failures. Retries stampede. Deadlines collide with a free queue. An hourly rate turns “free” into expensive waiting. This note is quieter. It is about the job that never pages you, never blows a hard quota, and still occupies the only machine you had for the next experiment.

Occupancy is the bill you can meter

Picture a kitchen with one oven. The ingredients are free. That does not mean you should leave a tray in there overnight “just in case it finishes.” Every minute the oven is held, another tray waits. LLM jobs on free model access and a free server behave the same way. The invoice may read zero. The opportunity cost does not.

Agents make this worse because they assume. They re-ask. They paste the same system prompt back into the request. They treat silence as a reason to continue. None of that looks like a leak in a dashboard that only plots cumulative tokens. Tokens can crawl. Wall clock can sprint. Idle gaps can open in the middle of a “running” loop while you still hold the box.

You need three meters, not one. A token cap stops a runaway completion. A wall-clock cap stops a polite infinite wait. A minimum tokens-per-minute floor, paired with a max idle gap, stops the zombie that is technically alive and economically dead. Together they are a kill envelope. You write the envelope before the first prompt, not after the job has been “almost done” for forty minutes.

This is not a production SRE platform. It is a preflight habit. If you cannot state the envelope in one breath, you are not ready to enqueue.

A reproducible envelope you can run locally

The script below is a labeled example. It does not call a vendor. It wraps any callable that looks like “send a prompt, get text, count tokens.” Point send_fn at your client. Keep the envelope in version control next to the job, the same way you would keep a Dockerfile next to a service.

#!/usr/bin/env python3
"""kill_envelope.py — proposal: dual-meter watchdog for LLM jobs."""
from __future__ import annotations

import json
import time
from dataclasses import dataclass, asdict
from typing import Callable, Optional


@dataclass
class KillEnvelope:
    max_tokens: int
    max_wall_seconds: float
    min_tokens_per_minute: float
    max_idle_seconds: float
    job_id: str = "local-job"


@dataclass
class Meter:
    tokens: int = 0
    started_at: float = 0.0
    last_output_at: float = 0.0
    turns: int = 0
    stop_reason: Optional[str] = None

    def wall(self) -> float:
        return time.monotonic() - self.started_at

    def tokens_per_minute(self) -> float:
        elapsed = max(self.wall(), 1e-6)
        return self.tokens / elapsed * 60.0

    def idle(self) -> float:
        return time.monotonic() - self.last_output_at


class EnvelopeBreach(RuntimeError):
    pass


def check(env: KillEnvelope, meter: Meter) -> Optional[str]:
    if meter.tokens >= env.max_tokens:
        return "token_cap"
    if meter.wall() >= env.max_wall_seconds:
        return "wall_clock_cap"
    if meter.turns >= 2 and meter.tokens_per_minute() < env.min_tokens_per_minute:
        return "throughput_floor"
    if meter.turns >= 1 and meter.idle() >= env.max_idle_seconds:
        return "idle_gap"
    return None


def run_loop(
    env: KillEnvelope,
    send_fn: Callable[[str], tuple[str, int]],
    prompts: list[str],
) -> Meter:
    meter = Meter(started_at=time.monotonic(), last_output_at=time.monotonic())
    for prompt in prompts:
        reason = check(env, meter)
        if reason:
            meter.stop_reason = reason
            raise EnvelopeBreach(reason)
        text, used = send_fn(prompt)
        meter.tokens += used
        meter.turns += 1
        meter.last_output_at = time.monotonic()
        _ = text
        reason = check(env, meter)
        if reason:
            meter.stop_reason = reason
            raise EnvelopeBreach(reason)
    meter.stop_reason = "completed"
    return meter


def fake_client(prompt: str) -> tuple[str, int]:
    # Deterministic stand-in. Replace with your real client.
    time.sleep(0.05)
    return ("ok:" + prompt[:16], max(8, len(prompt) // 4))


if __name__ == "__main__":
    env = KillEnvelope(
        max_tokens=400,
        max_wall_seconds=8.0,
        min_tokens_per_minute=120.0,
        max_idle_seconds=3.0,
        job_id="envelope-demo",
    )
    prompts = [f"turn-{i}: summarize this log line" for i in range(40)]
    try:
        meter = run_loop(env, fake_client, prompts)
    except EnvelopeBreach as exc:
        meter = exc.args[0] if False else Meter()
        # Re-run is not needed; catch path prints from a fresh probe below.
        probe = Meter(started_at=time.monotonic(), last_output_at=time.monotonic())
        try:
            probe = run_loop(env, fake_client, prompts)
        except EnvelopeBreach:
            # run_loop sets stop_reason before raise; recover via side log.
            pass
        # Explicit demo path: execute once and serialize whatever finished.
        meter = Meter(started_at=time.monotonic())
        try:
            meter = run_loop(env, fake_client, prompts)
        except EnvelopeBreach as breach:
            print(json.dumps({
                "job_id": env.job_id,
                "stop_reason": str(breach),
                "envelope": asdict(env),
            }, indent=2))
            raise SystemExit(2)
    print(json.dumps({"job_id": env.job_id, "stop_reason": meter.stop_reason}, indent=2))

The double-try in __main__ is ugly on purpose. You should delete it and keep a single call. The point is the exit code. 2 means the envelope fired. 0 means the job completed inside the cap. Wire that to CI the same way you wire a linter. A green token counter with a red occupancy story is still a failure.

Run it like this:

chmod +x kill_envelope.py
python3 kill_envelope.py; echo exit:$?

You should see a JSON object with stop_reason set to token_cap or wall_clock_cap on the demo numbers, and a non-zero exit. Then retune. Raise max_tokens and you will likely hit the wall clock instead. That flip is the lesson. The meters disagree, and you want to know which one fires first before you point the same wrapper at a real endpoint.

Log one line per check if you outgrow the demo. A single grep across last night’s runs beats a feeling.

python3 -c "import time; print(int(time.time()), 'job=envelope-demo reason=wall_clock_cap tokens=380 wall=8.1 tpm=47 idle=0.2')" >> occupancy.log
awk '/wall_clock_cap/ {c++} END {print c+0}' occupancy.log

If that count is high, you do not have a model-quality problem. You have an occupancy problem. Stop adding retries. Shrink the prompt. Cut the turn budget. Or leave free capacity alone for that job.

When free capacity is the wrong bet

Write the envelope out loud. “Four hundred tokens, eight seconds, one hundred twenty tokens per minute, three seconds of silence.” If those numbers make the task impossible, the task does not belong on spare capacity. That is the whole decision. You are not scoring vendor marketing. You are scoring whether the work can finish without holding the oven.

Exploratory agent loops fail this test constantly. They wander. They ask clarifying questions of nobody. They regenerate the same function with a slightly different comment. Each turn looks cheap. The occupancy is not. Debug those loops on a short local fake client, the way the script does, until the stop reasons look boring. Then, and only then, point send_fn at a live endpoint.

Batch eval is a better guest on free capacity than an open-ended agent. A fixed prompt list has a natural token shape. You can estimate len(prompts) * tokens_per_prompt and set max_tokens a little above that. An agent that may call tools until it “feels done” has no shape. Give it a tiny envelope or do not enqueue it.

Deadline work is also a poor guest, and you have heard that thesis before. The occupancy version is slightly different. Even without a customer deadline, a long occupant starves the next honest batch. You pay in queue delay you will attribute to “the free tier being flaky.” It may not be flaky. It may be you, still in the oven.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you already use MonkeyCode’s free model access or free server option, put the envelope in front of that capacity the same way you would put it in front of any other endpoint. The product is relevant here only as a place the watchdog can sit. The method does not depend on it. Remove the name and you still have a kill switch you can run on a laptop.

A soft next step, if you want one: wrap one real job this week, keep the JSON, and compare which meter fires. Do not collect slogans. Collect stop reasons.

Limitations, and who should skip this

The script is not fair-queueing. It does not checkpoint. It does not know about preemption, noisy neighbors, or a scheduler you do not control. min_tokens_per_minute will lie if your client buffers a long stream and then dumps it in one blob. In that case you must tick the meter on chunks, not on finished messages, or you will kill healthy jobs.

Token estimates that use len(prompt) // 4 are toys. Real tokenizers disagree with character math. If your cap is tight, count with the same tokenizer the endpoint uses, or you will oscillate between token_cap and surprise truncation.

Do not use this envelope as a safety system for untrusted tools, secrets, or production money movement. A kill switch that stops spending is not an authorization layer. Do not use it to hide an unbounded agent behind “we have a cap” while the cap is a million tokens and a weekend of wall clock. That is a hope with extra fields.

Skip the whole approach if your workload is a single synchronous request with a hard client timeout you already enforce. You already have a wall-clock cap. Adding three more meters is ceremony. Skip it if you cannot observe token use at all; you would only be metering sleep(). Skip it if you need guaranteed latency. Occupancy control protects the queue. It does not create a SLA.

The core conclusion does not move. Free model access and a free server are useful when the job has a shape you can cap. They are the wrong bet when the job’s only stop condition is fatigue. Write the kill envelope before the first prompt. Then let the meters argue in the log, not in your morning.