DEV Community

Jordan Huang
Jordan Huang

Posted on

FAQ: Five Myths About 'The Model Already Saw the Repo'

You pointed an agent at a full git clone. Then you asked it for a production patch. The reply sounded local, complete, and weirdly calm. Did that model actually read the tree, though?

I keep seeing the same mix-up in agent threads. Disk and context are two different layers. Confuse those layers and you ship a guess.

Disk is not context

A free server can hold your repository tonight. A free model can answer from a tiny slice. Those two free layers still do not merge. I treat them as separate machines in one loop.

Sometimes I run that split on MonkeyCode. It offers free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The product is optional for the checks below.

The FAQ is about claims I hear on loops. Each myth gets a claim, a check, and a better model.

Myth 1: "The clone finished, so the model knows"

The claim. Developers say the repo is already in the agent. They saw git clone succeed on the host. They skip file reads and dive into edits.

The evidence. A clone only writes blobs onto disk. The model only sees tokens you send. A successful clone proves storage, not model attention. Open the agent trace and count file reads.

The corrected model. Treat the server as a filesystem, not a brain. Treat the model as a sliding window. You must pull files through that window on purpose.

Ask yourself one blunt question right here. Which files were read before the first edit? If the answer is "probably all of them," you are guessing.

# Proposal: inspect what git put on disk.
git rev-parse --show-toplevel
git ls-files | wc -l
git ls-files -z | xargs -0 wc -c | tail -n 1
Enter fullscreen mode Exit fullscreen mode

That command output is inventory, not model memory.

Myth 2: "A recursive listing is a mental map"

The claim. One find or git ls-files call indexes the repo. People assume later turns will just remember paths. So why would you read those files twice?

The evidence. A path listing is only one chat message. Later turns can drop that old tool output. Bare path names are not the file bodies. src/auth.go is a label, not the JWT logic.

The corrected model. Treat those listings as a table of contents. Then read the chapters you will change. Re-read those files after each noisy refactor.

Would you review a PR from filenames only? Then do not let the model do that.

# Proposal: pin the files you will actually feed.
git ls-files '*.go' '*.mod' '*.json' | head
Enter fullscreen mode Exit fullscreen mode

Treat the snippet as a proposal only. I am not reporting a production benchmark here.

Myth 3: "The model is free, so dump everything"

The claim. Inference costs nothing tonight, so paste the tree. Bigger dumps feel safer than a tight slice. Free sounds like an unlimited working set.

The evidence. Free inference still does not mean infinite context. Huge dumps crowd the actual diff out. The model then recalls functions that never entered the window.

I published no quota numbers for any host. You should read your own current product docs. Then measure the prompt you actually send.

The corrected model. Spend the window on the blast radius. Send the interface, the failing test, and the caller. Leave generated folders sitting only on disk.

Feed this much, and no extra souvenir files:

  • README and manifest files
  • The failing test
  • The module under change
  • One neighbor interface

Keep these on disk only:

  • vendor/ or node_modules/
  • build artifacts
  • snapshots and huge fixtures

Is more files the same as more truth? It is not more truth in this window.

Myth 4: "If I didn't paste the secret, it is safe"

The claim. The prompt has no keys, so we are fine. The free server is just a scratch box. People swear the model will not open .env.

The evidence. An agent with shell access can read any reachable file. .env, id_rsa, and cloud creds are ordinary paths. Disk presence plus a tool call is enough.

The corrected model. If the process can cat it, the model can see it. Redact secrets before you clone the tree. Use a throwaway token for any smoke test. Never mount production secrets just for context.

# Proposal: fail closed on obvious secret names.
git ls-files | grep -E '(\.env|id_rsa|credentials|serviceAccount|\.pem)'
Enter fullscreen mode Exit fullscreen mode

Should a free model ever hold your payroll key? No, it should not, not even once.

Myth 5: "Explore the codebase" builds a durable index

The claim. People think one exploration prompt creates a lasting map. They expect later sessions to reuse it for free. People talk about that map like a database.

The evidence. Chat text is not a real code graph. Restart the session and the map dies. The server may still hold every file. The model still starts empty on the next session.

This is adjacent to git status myths, but it is not the same. Git status is only about dirty files. This myth is about imagined durable session memory.

The corrected model. Persist what you care about in git, tests, and a short brief. Write a short AGENT_BRIEF.md file for the next session. Do not trust that vanished exploration turn later.

# AGENT_BRIEF.md (proposal)
- Runtime: record it from `node -v` or `go version`
- Entry points: cmd/, src/main
- Commands: test, lint, build
- Do not touch: vendor/, dist/
Enter fullscreen mode Exit fullscreen mode

Would you onboard a human with one forgotten hallway chat? Then do not onboard a model that way.

Read the trace before the next prompt

Do not argue with the model's summary paragraph. Ask the saved trace three hard questions instead.

# Proposal: count read-like events in a saved trace.
# Replace the pattern with your own tracer format.
grep -c '"tool": "read"' trace.json
grep '"path":' trace.json | sort | uniq -c | sort -nr | head
Enter fullscreen mode Exit fullscreen mode

Which files were opened in that trace? Which files were listed and never read? Which files never appeared in the trace at all?

If the module you patched never showed up, stop. You are about to merge a pure guess.

Artifact: a repo visibility report

Run this on the same host the agent uses. It does not call any model at all. It prints what exists, what looks huge, and what looks secret.

I am labeling this script a proposal. Adapt the paths before you trust it.

#!/usr/bin/env python3
"""Proposal: repo visibility report. Not a model benchmark."""
from __future__ import annotations

import os
import sys
from pathlib import Path

SECRET_HINTS = (".env", "id_rsa", "credentials", "serviceAccount", ".pem")
SKIP_DIRS = {".git", "node_modules", "vendor", "dist", "build", ".venv"}


def iter_files(root: Path):
    for dirpath, dirnames, filenames in os.walk(root):
        dirnames[:] = [d for d in dirnames if d not in SKIP_DIRS]
        for name in filenames:
            yield Path(dirpath) / name


def main() -> int:
    root = Path(sys.argv[1] if len(sys.argv) > 1 else ".").resolve()
    rows = []
    for path in iter_files(root):
        try:
            size = path.stat().st_size
        except OSError:
            continue
        rel = path.relative_to(root)
        rows.append((size, str(rel)))
    rows.sort(reverse=True)
    total = sum(size for size, _ in rows)
    print(f"root\t{root}")
    print(f"files\t{len(rows)}")
    print(f"bytes\t{total}")
    print("top10_by_size")
    for size, rel in rows[:10]:
        print(f"{size}\t{rel}")
    print("secret_name_hits")
    hits = [rel for _, rel in rows if any(h in rel for h in SECRET_HINTS)]
    if not hits:
        print("(none)")
    for rel in hits[:50]:
        print(rel)
    print("suggested_feed")
    prefer = {"README", "README.md", "package.json", "go.mod", "pyproject.toml", "Cargo.toml"}
    for _, rel in rows:
        base = Path(rel).name
        if base in prefer:
            print(rel)
    return 0


if __name__ == "__main__":
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode

Run it like this:

python3 repo_visibility.py .
Enter fullscreen mode Exit fullscreen mode

Then fill this checklist before the first generation:

  1. How many files are on disk?
  2. Which ten files dominate bytes?
  3. Which secret-shaped names exist?
  4. Which three files will enter the prompt?
  5. Which command proves the change besides chat?

If you cannot answer item four, stop. The model does not already have the repo.

Decision table

Situation On disk In context Do this
Fresh clone Full tree Almost nothing Run the report, then read targets
Huge fixture JSON Yes Usually wasted Keep on disk, cite the path
Failing unit test Yes Required Paste the test and the error
.env on the server Dangerous Worse Delete it, rotate the token
Second session Files remain Map is gone Restore from AGENT_BRIEF.md

Print the table next to the agent trace. Argue with the trace, not the vibe.

Who should not use this

Skip the free-server path if the repo is regulated. Skip it if you cannot rotate secrets. Skip it if you need a guaranteed image and SLA. This FAQ does not replace your compliance review.

Do not use the script as a security scanner. It only matches names, not file contents. This script is not an adversarial security scanner. It will miss secrets with cute filenames.

Limitations

I published no latency numbers in this FAQ. I did not name models, quotas, or hardware. Those capability claims go stale very fast. Verify current product pages before you plan capacity.

The report cannot see inside the model window. Only your hidden trace can show that. If the trace is hidden, you are flying blind.

Short sentences help a lot in incident review. They can also hide some real nuance. Binary files, submodules, and sparse checkouts need extra care. You should add those extra checks yourself.

The loop I recommend now

I clone the repository onto the host. I run the visibility report on that clone. Then I pick a tight blast radius. I ask the model to edit that radius only. Then I run the test command I already wrote.

Does the chat transcript still say it is done? I ignore that chat line on purpose. The test command exit code is the story.

If you already have a free server, run the script there first. Paste the five checklist answers, not a vibe summary.

Top comments (0)