I Timed the Packer. The Model Waited.

The real bottleneck was prompt packing, not generation. I blamed the remote model for a full week. Does that confession sound a little too familiar?

I kept one graph after all the noise. It plotted packed bytes against later cited bytes. The model waited on my packer almost every run.

This week's feeds keep arguing about vibe coding. I still want a profile before any loud verdict. Can we call it engineering without honest timestamps?

I pointed the client at MonkeyCode's free server and free model access. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The open-source project states a ten-million-token free allowance.

I am not ranking vendors in this short note. I am timing my own packing client instead. Brand names will rot. The file walker will not.

Here is the failure I kept seeing locally. The assistant asked for one tiny numeric patch. My client still shipped half the tree anyway.

Why do we keep doing that to ourselves? We fear a missing import more than latency. Then we rename the wait as model slowness.

Think of a weekend bag versus a steamer trunk. The model asked for a toothbrush. I mailed the garage.

I built a packing-efficiency harness in plain Python. Treat every line as a proposal you can run. Point it at a fixture, never at production secrets.

The harness records five facts on each request. Files scanned. Files packed. Bytes packed. Bytes later cited. Client wait until first byte.

Cited bytes are a messy proxy. I know that already. They still beat a vibes-only dashboard, right?

# pack_profile.py — proposal harness, not a production tracer
from __future__ import annotations

import argparse, json, re, time
from pathlib import Path

IGNORE = {".git", "node_modules", "dist", "__pycache__", ".venv"}
TEXT_OK = {".py", ".md", ".toml", ".txt", ".json"}

def walk(root: Path) -> list[Path]:
    out: list[Path] = []
    for p in root.rglob("*"):
        if not p.is_file():
            continue
        if any(part in IGNORE for part in p.parts):
            continue
        if p.suffix.lower() in TEXT_OK:
            out.append(p)
    return sorted(out)

def pack(paths: list[Path], query: str, allow: set[str] | None) -> str:
    chunks: list[str] = [f"Query:\n{query}\n"]
    for p in paths:
        rel = str(p.as_posix())
        if allow and not any(a in rel for a in allow):
            continue
        try:
            body = p.read_text(encoding="utf-8", errors="replace")
        except OSError:
            continue
        chunks.append(f"\n# FILE {rel}\n{body}")
    return "".join(chunks)

def cited_bytes(prompt: str, reply: str) -> int:
    hits = 0
    for m in re.finditer(r"# FILE (\S+)", prompt):
        name = Path(m.group(1)).name
        if name and name in reply:
            start = m.start()
            nxt = prompt.find("# FILE ", start + 1)
            block = prompt[start: nxt if nxt != -1 else len(prompt)]
            hits += len(block.encode("utf-8"))
    return hits

def main() -> None:
    ap = argparse.ArgumentParser()
    ap.add_argument("--root", required=True)
    ap.add_argument("--query", required=True)
    ap.add_argument("--allow", default="")
    ap.add_argument("--reply-file", default="")  # paste a saved reply for the proxy
    args = ap.parse_args()
    root = Path(args.root)
    t0 = time.perf_counter()
    scanned = walk(root)
    allow = {x for x in args.allow.split(",") if x}
    prompt = pack(scanned, args.query, allow or None)
    packed_paths = [p for p in scanned if not allow or any(a in str(p) for a in allow)]
    pack_ms = (time.perf_counter() - t0) * 1000
    reply = Path(args.reply_file).read_text() if args.reply_file else ""
    row = {
        "scanned": len(scanned),
        "packed": len(packed_paths),
        "packed_bytes": len(prompt.encode("utf-8")),
        "cited_bytes": cited_bytes(prompt, reply) if reply else 0,
        "pack_ms": round(pack_ms, 1),
    }
    print(json.dumps(row, indent=2))
    Path("pack_last.json").write_text(json.dumps(row))

if __name__ == "__main__":
    main()

Run the walker against a throwaway fixture first. Do not aim it at a customer tree. Secrets do not belong in a packing experiment.

python pack_profile.py \
  --root ./fixture_repo \
  --query "fix the off-by-one in total.py" \
  --reply-file ./sample_reply.txt

The fixture stays tiny on purpose. Two modules and one test file. You should watch the packer swallow comments and junk docs.

# fixture_repo/total.py
def total(xs):
    n = 0
    for x in xs:
        n = n + x
    return n  # off-by-one lives in the test, not here

# fixture_repo/total_test.py
from total import total

def test_total():
    assert total([1, 2, 3]) == 6

# fixture_repo/NOTES.md
This file is bait. A hungry packer will ship it.

Illustrative output has this shape only. Do not quote it as a study.

# fat pack, no allow list
{"scanned": 8, "packed": 8, "packed_bytes": 17240, "cited_bytes": 640, "pack_ms": 14.2}

# tight pack, allow=total.py,total_test.py
{"scanned": 8, "packed": 2, "packed_bytes": 1880, "cited_bytes": 580, "pack_ms": 3.1}

See the first row clearly? Eight files packed, almost nothing cited. The second row follows one boring allow list. Same question. Less luggage on the wire.

I kept that pair as the graph. Two dots. One lesson. Stop shipping noise the reply never names.

How did I draw it without a dashboard product? I printed a tiny CSV. Then I plotted it with a short script.

printf '%s\n' 'packed_bytes,cited_bytes,pack_ms' \
  '17240,640,14.2' \
  '1880,580,3.1' > pack.csv
python plot_pack.py

# plot_pack.py — labeled example, not a claimed production chart
import csv
from pathlib import Path

rows = list(csv.DictReader(Path("pack.csv").open()))
print("packed -> cited (pack_ms)")
for r in rows:
    pb = int(float(r["packed_bytes"]))
    cb = int(float(r["cited_bytes"]))
    ms = r["pack_ms"]
    bar = "#" * max(1, pb // 800)
    cit = "." * max(1, cb // 800)
    print(f"{bar}\n{cit}  {pb}->{cb} bytes  {ms} ms")

The plot is not pretty. I still kept it. Pretty charts hide the overpack every time.

Network time still matters on a free remote hop. I did not pretend it vanished overnight. Packing waste moved first on my client, though.

Want a sanity check before you trust one fat wait number? Split DNS, TLS, and first byte. One blob named "generation" will lie to you.

# ttfb_probe.py — proposal only, swap in your real endpoint
import socket, ssl, time

host = "example.invalid"  # replace; do not paste secrets
port = 443

def timed(fn):
    t = time.perf_counter()
    fn()
    return (time.perf_counter() - t) * 1000

dns_ms = timed(lambda: socket.getaddrinfo(host, port, type=socket.SOCK_STREAM))
print({"dns_ms": round(dns_ms, 1)})

I also logged which paths entered the prompt. The list shamed me immediately. README, changelog, a screenshot path. None of them appeared in the reply.

So I added a cheap allow list after the first graph. Tests plus the module named in the query. Nothing else unless the reply asked.

That rule will miss a real import someday. I accept that miss on purpose. I would rather miss once than wait on every run.

Is this still vibe coding if I time it? Maybe it is. I just refuse to vibe the clock itself.

A coding agent on a free server is a flashlight. The packer decides where you point it. A bright light on the wrong wall still wastes the night.

If you work in a secret-heavy repo, skip remote packing. Keep the tree on your side of the door. A free server is not an access-control story.

If you need a signed SLA, this note is not yours. A free server can be busy without warning. I am not promising queue times or hardware.

If your monorepo has no ignore file, stop here. You will profile a disaster you already know. Fix the ignore file before you blame tokens.

Client clocks drift under load. Repeat the run three times at least. Throw away the first warm-up. Caches lie in both directions.

Do not ship this harness into CI as a gate. It measures packing waste on the client. It does not measure correctness of the patch.

The testing debate this week missed this dull layer. People ask if models outgrow the tests. I ask if our prompts outgrow the bug.

A test that never runs because the packer is fat still fails. Slow feedback is a test failure. You just named it generation, didn't you?

I now refuse to tune temperature before I tune includes. Temperature is a spicy leftover knob. Includes are the actual meal on the table.

First-token time is not the same as useful time. A fast hello with a useless tree is still waste. Did your last trace even show the packed byte count?

I keep the packing graph next to the patch. If cited bytes stay tiny, I cut the prompt. If cited bytes jump, I inspect the allow list.

That is the whole method, said plainly. Profile the suitcase. Then talk about the model. Anything else is theater with a spinner.

If you already have a free remote coding endpoint, run the harness on a fixture tonight. Keep the ugly graph. Drop the trunk.