The Hero Feature: codebase-vis explain

So, I built codebase-vis: a language-agnostic CLI tool that parses your code, clusters related modules using graph theory, and uses an LLM to generate a semantic architecture report.

Here is how I built it and how it works.

The Hero Feature: codebase-vis explain

You can't just dump an entire codebase into an LLM's context window and expect a good architecture breakdown. The model gets confused, loses context, or simply hits token limits.

To solve this, codebase-vis breaks the problem down. It maps your code into a graph, groups highly coupled files into "clusters," and then asks the LLM to explain each cluster one by one.

Here is what the workflow looks like in the terminal (see explain.png for the output):

1. Bring Your Own Key: When you run codebase-vis explain, it prompts you for your API credentials (I highly recommend using Groq with Llama 3.1 for lightning-fast inference). It saves these locally to ~/.codebase-vis/config.json.
2. Graph Loading: The tool loads the previously generated dependency graph (e.g., 122 nodes, 203 edges).
3. Cluster Detection: It identifies isolated "communities" within your code (e.g., 17 clusters across 96 files).
4. Concurrent AI Analysis: It loops through these clusters, extracting a payload of the file structures and dependencies, and hits the LLM to generate a summary for each.
5. Output: It enriches your graph.json with semantic metadata and outputs a highly readable semantic-summary.md detailing the purpose of each cluster.

As you can see in explain.png, hitting an API concurrently for 17 clusters can trigger rate limits. To handle this, the CLI uses a custom TokenBucket algorithm to throttle requests, catching failures gracefully and keeping the analysis moving.
also a --retry flag is also introduced to handle the failed clusters, the failed cluste's JSON data is saved seperately and when hit the retry flag is dumps the json and the retries the cluster.

---

How the pipeline works under the hood

To make the explain command work accurately, the foundation has to be rock solid. Here are the three main pipelines making it happen:

1. The Parsing Engine (Tree-sitter + Worker Threads)

Relying on simple Regex to find imports is a nightmare. Instead, I used Tree-sitter to generate Abstract Syntax Trees (ASTs).

Because parsing hundreds of files synchronously blocks the Node.js event loop, I implemented a custom WorkerPool. The CLI spins up background processes to parse TypeScript, Python, Rust, C++, and Go files concurrently. It extracts module-level dependencies and entities (functions, classes) and pipes them back to the main thread.

2. Structuring the Graph (Graphology + Louvain)

Once I had the raw dependencies, I used Graphology to construct a directed graph.

But massive codebases just look like a giant spiderweb. To chunk the graph for the LLM, I implemented the Louvain community detection algorithm. The tool automatically detects clusters of highly coupled files (like grouping your API routes separately from your database models) and names them based on their common root directories.

3. The AI Handoff

Once the Louvain algorithm finishes grouping the graph, the explain command takes those exact boundaries and feeds them to the LLM. By only feeding the model logically related chunks of code, the resulting semantic-summary.md is incredibly accurate and actually reflects the modular architecture of the system.

---

The Advance Flag

1. codebase-vis explain --concurrency number : number of parallel concurrent requests.
2. codebase-vis explain --rpm number : number of request per minute, by default it is set to 30. only useful when paid api key.
3. codebase-vis explain --reset : to reset the creds.
4. codebase-vis explain --model name : to update model name (currently supports GROQ only)
5. codebase-vis explain --retry : to retry with failed cluster[s].