A technical deep-dive into exhaustive repository analysis with LLMs.
Every code intelligence tool I've used works the same way: you ask a question, it finds three relevant chunks using embeddings or keyword search, it answers from those three chunks.
That design made sense in 2021. Inference was expensive, context windows were small, and reading an entire repository in one pass was a non-starter.
None of that is true anymore.
A mid-size C project like Redis contains roughly 10,000 functions. At one LLM call per function, analyzing the whole thing costs an afternoon of inference and somewhere between $2 and $20 depending on your model. That's not a recurring cost — you do it once, you cache the results, and every question afterward is answered from a complete index rather than a lucky search.
The inversion that makes this interesting: instead of choosing which parts of the codebase to read, you just read everything. The hard part is no longer selection. The hard part becomes structuring the output so it's useful.
I built Strata to explore what happens when you take that inversion seriously.
The core idea: layers
The name comes from the architecture. Analysis happens in seven passes, each building on the one beneath, from raw syntax at the bottom to project philosophy at the top.
L6 History & Philosophy ← "What does this project value?"
L5 Security Attention Map ← "Where should a reviewer look first?"
L4 Dead Code ← "What can safely be deleted?"
L3 Documented vs Actual ← "How badly has the architecture drifted?"
L2 Module/Subsystem Rollup ← "What does each module do?"
L1 Function Summaries ← "What does each function do?"
L0 Structure ← "What functions exist? Who calls whom?"
Every pass writes to a content-addressed SQLite cache. Re-runs are free — only changed files re-analyze. Bumping a pass version (e.g., "L1_v2") invalidates exactly that pass without touching the others.
L0: Structure without LLM
The foundation. L0 runs tree-sitter on every source file, extracts the symbol table, builds the call graph, and records git blame attribution. No LLM touches this layer.
Getting this right matters disproportionately because everything above it hangs off it. If L0's call graph is wrong, L4's dead code detection produces garbage, and L5's security paths become fictional.
The tree-sitter query for C function extraction looks like this:
_FUNC_DEF_QUERY_SRC = """
(function_definition
declarator: (function_declarator
declarator: (identifier) @name)
) @func
"""
# In tree-sitter 0.25, QueryCursor is the execution API
def caps(query, node):
return QueryCursor(query).captures(node)
One subtlety: tree-sitter's Python bindings changed their API between 0.22 and 0.25. In 0.22, query.captures(node) was a method on the Query object. In 0.25, Query is just a compiled pattern — you need QueryCursor(query).captures(node) to execute it. This is the kind of thing that silently produces empty results rather than erroring, so watch for it.
The blame optimization matters too. A naive implementation calls pygit2.blame() once per symbol:
# SLOW: one I/O-heavy blame call per symbol
for sym in file.symbols:
author = primary_blame(repo_root, rel_path, sym.line_start, sym.line_end)
Redis has ~30 symbols per file × 324 files = ~10,000 blame calls. Each one opens the git repository, walks the history, and deserializes hunk data. At 1-2 seconds per call, that's hours.
The fix is obvious once you see it — load blame once per file, reuse for all symbols:
# FAST: one blame call per file
blame_cache: dict[str, dict] = {}
for file_path in files:
if rel not in blame_cache:
blame_cache[rel] = blame_file(repo_root, rel) # once
for sym in info.symbols:
author = _blame_range(blame_cache[rel], sym.line_start, sym.line_end)
~10x speedup on L0.
L1: Function summaries at scale
This is where the inference budget goes. One LLM call per symbol, all running concurrently up to a configurable semaphore ceiling.
The key design decision is structured output via instructor + pydantic. The model returns a typed FunctionSummary object, not free text:
class Citation(BaseModel):
file: str
line_start: int
line_end: int
class FunctionSummary(BaseModel):
one_line: str = Field(max_length=180)
preconditions: list[str]
invariants: list[str]
side_effects: list[str]
error_paths: list[str]
surprising: str | None
citations: list[Citation] # EVERY claim must cite a real line
confidence: float = Field(ge=0.0, le=1.0)
static_vs_llm_disagreements: list[str]
Every list item in preconditions, invariants, side_effects, and error_paths requires a citation in the prompt instructions. The validator then checks that cited line ranges actually fall within the symbol's range:
def _validate_citations(summary, file_path, line_start, line_end):
errors = []
for c in summary.citations:
if c.file != file_path:
errors.append(f"citation file mismatch: {c.file!r}")
if c.line_start < line_start - 5 or c.line_end > line_end + 5:
errors.append(f"citation {c.line_start}–{c.line_end} outside {line_start}–{line_end}")
return errors
Rather than rejecting out-of-range citations outright (which would waste the inference), we penalize confidence instead:
if val_errors:
summary = summary.model_copy(
update={"confidence": max(0.0, summary.confidence - 0.2 * len(val_errors))}
)
The static_vs_llm_disagreements field is one of the most interesting outputs. When the model mentions a callee that the static call graph doesn't show, it records it. These disagreements usually mean one of three things: dynamic dispatch, function pointers, or macro expansion. All three are exactly the places a human reviewer should pay attention.
The few-shot examples in the prompt are doing a lot of work. Here's the opening of one:
EXAMPLE 1 — simple allocator wrapper
Function: dictCreate File: src/dict.c Lines: 97–120
Source:
int *dictCreate(dictType *type, void *privDataPtr) {
dict *d = zmalloc(sizeof(*d));
if (d == NULL) return NULL;
_dictInit(d,type,privDataPtr);
return d;
}
Response:
{
"one_line": "Allocates and zero-initializes a new hash table with the given type vtable.",
"preconditions": ["type is a valid non-NULL dictType pointer [src/dict.c:97]"],
"side_effects": ["allocates heap memory via zmalloc [src/dict.c:99]"],
"error_paths": ["returns NULL if zmalloc fails [src/dict.c:100]"],
...
"confidence": 0.97
}
Notice the inline citation format inside list items ([src/dict.c:99]). The model learns to inline citations rather than collecting them only in the top-level citations array.
L2: Map-reduce rollups
L2 summarizes the repository bottom-up. Function summaries → module summaries → subsystem summaries → project summary. The model never sees more than one level at a time.
def _module_prompt(module_path, summaries):
joined = "\n".join(f"- {s}" for s in summaries[:120])
return f"""
Summarize the module '{module_path}' in 2–4 paragraphs based on these function summaries.
Cover: what the module does, its key abstractions, notable design decisions, anything surprising.
Function summaries:
{joined}
"""
Capping at 120 function summaries per module prevents token overflow on large files. The cap is generous enough for real-world C files but worth adjusting for codebases with very large files.
The map-reduce structure means Strata scales to any repository size — you never send the entire codebase to a model in a single context.
L3: Architecture drift
This is the most interesting output in the product.
Every long-lived project has drifted from its own documentation. Strata measures it. L3 reads every Markdown file in the repo, extracts the claimed architecture, and diffs it against what L0 and L2 actually found.
DOC_GLOBS = ["README*", "ARCHITECTURE*", "docs/**/*.md", "doc/**/*.md"]
def _divergence_prompt(claimed, actual_modules, actual_imports, rollup_names):
return f"""
Claimed architecture (from docs):
{json.dumps(claimed, indent=2)}
Actual modules (from code):
{json.dumps(actual_modules[:200], indent=2)}
Identify divergences. Classify each as:
- undocumented_module: code module not mentioned in docs
- layering_violation: import that violates a stated layering rule
- phantom_abstraction: described as pluggable but only one implementation exists
- ghost_component: documented component that no longer exists in code
Return JSON array: [{{"kind": "...", "description": "...", "evidence": "..."}}]
"""
Four divergence kinds. In practice, undocumented_module is the most common — projects grow new subsystems that never make it into the docs. ghost_component is the most interesting — abstractions that were removed but whose documentation lives on.
L4: Dead code with confidence tiers
Reachability over the call graph from real entry points. The key design decision: three tiers instead of a binary dead/alive classification.
Tier 1: no callers, no exports, no reflection indicators → almost certainly dead
Tier 2: unreachable statically but language has escape hatches → uncertain
Tier 3: reachable only from test files → test-only
Overclaiming tier 1 destroys trust in the whole tool. A dead code report where 30% of "dead" functions turn out to be called at runtime via dlopen or function pointers is worse than no report at all. So we surface the tier prominently and let the user decide.
has_escape = (
"dlopen" in (sym.get("source", "") or "")
or name.startswith("_") # convention: private but possibly used by macros
)
tier = 1 if not has_escape else 2
This is conservative on purpose.
L5: Security attention map (not a vulnerability scanner)
L5 traces paths from untrusted-input sources to sensitive sinks through the call graph.
SOURCE_NAMES = frozenset({
"read", "recv", "recvfrom", "recvmsg", # network
"fread", "fgets", "getline", # file
"getenv", "getopt", # process
"readQueryFromClient", "processInputBuffer", # Redis-specific
})
SINK_NAMES = frozenset({
"memcpy", "memmove", "strcpy", "strcat", # memory
"execve", "execl", "system", "popen", # exec
"open", "fopen", "unlink", # path
})
The framing matters. This is explicitly not a vulnerability scanner. The UI and documentation say so clearly. It answers "where should a security reviewer spend their first day?" — which is genuinely valuable and honest. A false-positive-riddled bug finder is a liability. A well-ranked attention map is a tool people will actually use.
L6: History and philosophy
Walk every commit. Chunk by quarter. Summarize each chunk. Reduce to eras.
def _quarter(timestamp: int) -> str:
d = datetime.datetime.utcfromtimestamp(timestamp)
q = (d.month - 1) // 3 + 1
return f"{d.year}-Q{q}"
The interesting outputs:
Churn coupling — files that change together frequently, revealing coupling the architecture doesn't admit to:
def _compute_churn_coupling(commits, top_n=20):
pair_counts: Counter = Counter()
for c in commits:
for i, a in enumerate(c.files_changed):
for b in c.files_changed[i+1:]:
pair = tuple(sorted([a, b]))
pair_counts[pair] += 1
# normalize by minimum individual change count
...
Knowledge silos — subsystems where one author wrote >80% of the code and no one else has significantly touched it. These are bus factor risks.
Implicit values essay — a final LLM call that reads the entire commit history and argues for what the project consistently chose when forced to trade off between speed, simplicity, compatibility, and safety. The prompt asks for evidence from the commit record, not assertions.
The visualization
The static site uses d3-hierarchy's treemap layout. The key design constraint: it must be fully static, deployable to a CDN, with no server. All data is precomputed JSON.
const treemap = d3.treemap<HierarchyDatum>()
.size([width, height])
.paddingOuter(4)
.paddingInner(1)
.paddingTop(18)
.round(true)
Five lenses, each a different color function over the same cell data:
export function cellColor(sym: SymbolCell, lens: Lens): string {
switch (lens) {
case 'churn': {
const age = clamp((NOW - parseDate(sym.last_changed)) / ONE_YEAR, 0, 5) / 5
return lerp('#e63946', '#264653', age) // red=recent, blue=old
}
case 'security':
if (sym.security_paths >= 3) return '#e63946'
if (sym.security_paths >= 1) return '#f4a261'
return '#264653'
case 'dead':
if (sym.dead_tier === 1) return '#6b6b6b'
// ...
}
}
Switching lenses is instant — all data is already in the precomputed JSON, no fetches required.
The cache architecture in full
┌─────────────────────────────────────────────────┐
│ cache table (SQLite) │
│ │
│ key TEXT PRIMARY KEY │
│ sha256(file_hash + pass_id │
│ + prompt_hash) │
│ │
│ pass_id TEXT "L1_v1", "L2_v1", ... │
│ file_path TEXT relative to repo root │
│ symbol TEXT qualified name (nullable) │
│ created_at INTEGER unix epoch │
│ result_json TEXT the LLM output │
└─────────────────────────────────────────────────┘
- Same file, same prompt, same pass version → cache hit, zero cost
- File changes →
file_hashchanges → cache miss for that file only - Prompt edit →
prompt_hashchanges → cache miss for all files in that pass - Pass version bump →
pass_idchanges → same effect as prompt edit, but explicit
Old cache rows are never deleted — they become historical records. Useful for seeing how your summaries improved as you refined the prompts.
What I learned
Retrieval is a crutch, not a constraint. The reason every code intelligence tool uses retrieval is cost, not correctness. Once you remove the cost constraint, the design space opens up significantly.
Citations are load-bearing. The requirement that every claim cite a real source line is not just a correctness measure — it changes the model's behavior in ways that improve overall quality. The model stops making confident assertions about things it can't point to.
The call graph disagreement field is underrated. I added static_vs_llm_disagreements mostly as a correctness measure, but it turns out to be a useful output on its own. When the model says a function calls something the static analysis can't see, that's usually a signal of interesting dynamic behavior.
Confidence surfacing matters. The model's self-reported confidence correlates well with actual accuracy on spot-checks. Surfaces it in the UI, not buried in a tooltip — functions with confidence below 0.6 deserve human review.
The history pass is the one that surprises people. Most engineers have never read a narrative history of the projects they work on. When you give them one argued from the actual commit record, they find things they didn't know: decisions that were made and reversed, coupling that the architecture doesn't admit to, subsystems with a single author who left three years ago.
Running it yourself
git clone https://github.com/harishkotra/strata
cd strata && pip install -e .
cp .env.example .env # fill in your LLM endpoint
git clone https://github.com/redis/redis /tmp/redis
strata analyze /tmp/redis --passes 0 # L0 first, no LLM
strata analyze /tmp/redis --passes 1 # L1, watch progress bar
strata analyze /tmp/redis --passes 2,3,4,5,6
strata export /tmp/redis/.strata.db web/public/data
cd web && npm install && npm run dev
The map loads at localhost:5173. Click any cell. Switch lenses. Read the tour. Ask questions:
strata ask "Which functions are most likely to contain memory bugs?"
strata ask "What is the Redis cluster replication protocol?"
strata ask "Which files changed together most often in the last five years?"
Code & more: https://www.dailybuild.xyz/project/238-strata
Top comments (0)