How Does an AI Coding Assistant Actually Understand Your Codebase?

You open your editor, type "fix the shipping bug", and the AI goes ahead and changes the right file,
the right function, the right lines — without you telling it where anything is.

How?

Does it read every file? Does it use some kind of search index? Does it dump your entire project
into its memory before it even starts?

Let's break it down properly.

---

The obvious assumption — and why it's wrong

The first thing most people assume is that the AI pre-loads the whole codebase before answering.
Like it reads everything upfront, builds some internal map, and then works from that.

That's not what happens.

%%{init: {'theme': 'dark'}}%%
flowchart LR
    subgraph MYTH["❌ What people assume"]
        direction LR
        A1["Your codebase\n📁 all files"] -->|"pre-loaded upfront"| B1["AI memory\n🧠"]
        style A1 fill:#fee2e2,stroke:#ef4444,color:#991b1b
        style B1 fill:#fee2e2,stroke:#ef4444,color:#991b1b
    end

    subgraph REALITY["✅ What actually happens"]
        direction LR
        A2["Your codebase\n📁 all files"] -->|"reads only what\nit needs, on demand"| B2["AI memory\n🧠 small slice"]
        style A2 fill:#dcfce7,stroke:#22c55e,color:#166534
        style B2 fill:#dcfce7,stroke:#22c55e,color:#166534
    end

The AI has a context window — a fixed amount of working memory. Think of a token as roughly ¾ of a word; modern models handle ~100,000–170,000 tokens (~50,000 words) at once. Everything goes
into it: your conversation, file contents, tool output, and the AI's own responses. There's no separate storage. There's no background indexing running.

If it loaded your entire codebase upfront, it would burn through that budget before you even asked your first question.

So, It doesn't load your entire codebase upfront.

Instead, it explores on demand — reading only what it needs, when it needs it.

---

The tool-driven exploration loop

Rather than a RAG pipeline — where an AI pre-indexes your code into a searchable database and retrieves chunks at query time — modern AI coding assistants use a set of read tools they call during a conversation to pull specific code into context.

The loop has three phases: Discover → Understand → Edit. Each is explained below.

%%{init: {'theme': 'dark'}}%%
flowchart TD
    A["You type a request"]
    A --> B["Step 1 · Discover\nFind candidate files — nearly free"]
    B --> C["Step 2 · Understand\nRead only what's needed — targeted"]
    C --> D["Step 3 · Edit\nChange only the lines that matter"]

---

Step 1: Discovery — finding the needle without reading the haystack

Here's the key insight most people miss: searching is not the same as reading.

grep and glob are OS-level operations. The operating system does the mechanical work —
the AI just decides what pattern to look for.

glob — filename pattern matching

glob("**/*shipping*.ts")

The OS walks the directory tree and matches filenames against the pattern.
It never opens a single file. The AI gets back a list of paths:

src/shipping/shippingService.ts
src/middleware/shippingGuard.ts
src/utils/shippingHelpers.ts

Token cost: essentially zero. No file content was touched.

grep — regex scan across file bytes

grep("calculateShipping", include="*.ts")

The OS scans raw bytes looking for the pattern. It returns only the matching lines + line numbers,
not the whole file:

src/shipping/shippingService.ts:42:  async calculateShipping(order: OrderDto) {
src/controllers/orderController.ts:18:  await shippingService.calculateShipping(req.body)

Token cost: small. Only the matching snippets land in context — not the files themselves.

How does the AI know what to search for?

It uses your words as clues. You say "fix the shipping bug" → it searches for shipping,
order, calculateShipping. Developers name things predictably, and grep exploits that.
The AI's job at this stage is just to form a good search query — the OS does the actual scanning.

%%{init: {'theme': 'dark'}}%%
flowchart TD
    A["You say: 'fix the shipping bug'"] --> B

    subgraph SIGNAL["Signal sources"]
        B["Natural language in your request\n'shipping', 'order', 'delivery'"]
        C["Project structure hints\npackage.json, folder names, imports"]
    end

    B --> D
    C --> D

    D["AI reasons: shipping logic is probably in a file\nnamed shipping or order, or a function\ncalled calculateShipping or getDeliveryRate"]

    D --> E["grep for the function name\nglob for the file name pattern"]

    E --> F["OS returns: file paths + line numbers\nwhere the pattern matched"]

    F --> G["Now the AI knows exactly\nwhich file, which line to read next"]

---

Step 2: Triage — multiple results is the normal case

grep almost always returns multiple matches. That's expected. The AI triages them
using the file path and matched line snippet — no extra file reads needed at this stage.

%%{init: {'theme': 'dark'}}%%
flowchart TD
    A["grep 'calculateShipping' returns\n5 matches across 4 files"] --> B

    subgraph TRIAGE["Triage — reason about the matches"]
        B["Look at file paths + line snippets\nalready in context — no extra reads"]
        B --> C{"What kind of match?"}
        C --> D["shippingService.ts:42\nasync calculateShipping — DEFINITION\n✅ Read this first"]
        C --> E["orderController.ts:18\nawait calculateShipping — CALL SITE\n📌 Read if change affects callers"]
        C --> F["shippingService.spec.ts:91\ncalculateShipping mock — TEST\n🧪 Skip unless fixing tests"]
        C --> G["shipping.types.ts:7\ncalculateShipping: Function — TYPE\n📋 Skip for now"]
    end

    D --> H["read_file shippingService.ts:38–55\nonly 17 lines loaded into context"]

The filename alone carries most of the signal:

Match	Signal	Decision
shippingService.ts:42: async calculateShipping(	Definition, service layer	Read this
orderController.ts:18: await calculateShipping(	Call site, controller	Read if impact matters
shippingService.spec.ts:91: calculateShipping(mock)	Test file	Skip for now
shipping.types.ts:7: calculateShipping: Function	Type declaration	Skip
index.ts:3: export { calculateShipping }	Barrel export	Skip

A senior developer does exactly this with "Find All References" in their IDE —
they scan the result list and instantly know which hit is the definition vs. a test vs. a type, without opening every file. The AI applies the same reasoning.

If results are still ambiguous after triage, there are escalating strategies:

%%{init: {'theme': 'dark'}}%%
flowchart LR
    A["Still ambiguous\nafter triage"] --> B["FindReferencingSymbols\nwho calls this?"]
    B --> C["GetSymbolsOverview\nscan file structure\nwithout reading bodies"]
    C --> D["read_file with\nnarrow line range\nonly the function body"]
    D --> E["If still unclear:\nask the user"]

---

Step 3: Edit — surgical, not wholesale

Once the AI has read the relevant section, it edits using the most targeted tool available:

Tool	Scope
apply_diff	Precise targeted lines — the default for edits
search_and_replace	Find-and-replace a pattern within a file
insert_content	Insert at a specific line position
write_file	Full rewrite — only for new files

The preference is always apply_diff — it changes only the lines that need to change.
The whole file never needs to be rewritten.

---

So... does any of this cost tokens?

Yes. Let's be honest about it.

grep and glob themselves are free — the OS does the work.
But everything that lands in context and every step where the AI reasons costs tokens.

%%{init: {'theme': 'dark'}}%%
flowchart LR
    A["grep output\n5 snippets ≈ 50–100 tokens\n🟢 cheap"] --> B
    B["AI reasons: which file to read?\n🔴 LLM cost"] --> C
    C["read_file chosen file\n200 lines ≈ 1,500 tokens\n🟡 moderate"] --> D
    D["AI reasons: which lines to change?\n🔴 LLM cost"] --> E
    E["apply_diff writes to disk\n🟢 free"]

The real saving is in how much context the AI has to reason over:

Without the grep strategy:
  Read 50 files × 300 lines avg → ~112,000 tokens

With the grep strategy:
  5 grep snippets   → ~100 tokens
  1 file read       → ~1,500 tokens
  Total             → ~1,600 tokens

The LLM reasoning cost happens either way. What changes is the size of the input it's reasoning over.
Smaller input = cheaper per-token cost + faster response.

---

The complete picture

Every tool, every phase — the full loop:

Step 1 · Discover — find candidate files

%%{init: {'theme': 'dark'}}%%
flowchart LR
    A["You type a request"]
    A --> B["grep\nsearch for keywords across files"]
    A --> C["glob\nfind files by name pattern"]
    A --> D["list_files\nbrowse directory structure"]

Step 2 · Understand — read just what's needed

%%{init: {'theme': 'dark'}}%%
flowchart LR
    A["Candidate files\nfrom Step 1"]
    A --> B["GetSymbolsOverview\ntop-level symbols in a file"]
    A --> C["FindSymbol\njump to a specific class or method"]
    A --> D["read_file\nread exact file or line range"]
    A --> E["FindReferencingSymbols\nwhere is this function called?"]

Step 3 · Edit — targeted surgical change

%%{init: {'theme': 'dark'}}%%
flowchart LR
    A["Relevant section\nfrom Step 2"]
    A --> B["apply_diff\nchange specific lines only"]
    A --> C["search_and_replace\nupdate a pattern"]
    A --> D["insert_content\nadd lines at a position"]
    A --> E["write_file\ncreate a new file"]

Three phases. The AI never skips to Step 3 without going through 1 and 2.

%%{init: {'theme': 'dark'}}%%
flowchart TD
    A["You: 'fix the shipping bug'"]
    A --> B["grep / glob\nOS scans — FREE"]
    B --> C["Snippets in context\nAI triages — small cost"]
    C --> D["read_file narrow range\nRelevant section only — moderate cost"]
    D --> E["AI reasons about the change\nLLM cost"]
    E --> F["apply_diff\nOS writes — FREE"]

• The OS does the dumb-but-fast scanning for free
• The LLM does the smart reasoning on a small, targeted result set
• The context window fills incrementally — never all at once

No RAG. No pre-indexing. No magic.
Just grep, triage, read the right bit, change the right lines.