I Built RepoPilot: An AI-Powered Tool to Understand Codebases and Generate Tests

Arnab Lahiri — Sun, 18 Oct 2026 21:40:28 +0000

How a side project evolved into a developer tool combining AST parsing, dependency graphs, repository-aware AI, and automated test generation.

As developers, we've all been there.

You open an unfamiliar repository. There are hundreds of files, deeply nested dependencies, unfamiliar abstractions, and barely any documentation.

Before you can fix a bug or introduce a feature, you need to answer a few questions:

Where does the actual business logic live?
How are different modules connected?
Which functions depend on this particular service?
How can I generate meaningful tests without spending hours understanding every implementation detail?

AI coding assistants have made parts of this easier. But I wanted to experiment with something more structured.

What if an AI assistant could first understand a repository's architecture and dependencies, then use that context to answer questions and generate tests?

That's how RepoPilot started.

Introducing RepoPilot

RepoPilot is an AI-powered developer tool designed to help engineers explore unfamiliar codebases, understand their structure, and generate context-aware tests.

🔗 Try RepoPilot: https://repopilotapp.com

Instead of manually copying files into an AI chatbot, you can connect a repository and let RepoPilot build the context required for codebase-aware interactions.

Here's what the current version offers:

1. Repository integration

Connect GitHub, GitLab, or Bitbucket repositories, including private repositories through supported OAuth integrations.

2. Codebase indexing and dependency analysis

RepoPilot analyzes the repository's source code, extracts structural information, and builds relationships between files and code entities.

3. AI-powered codebase chat

Ask questions about your repository and receive answers informed by the indexed codebase.

4. Context-aware unit and integration test generation

Select source files and generate tests based on their implementation and related dependencies.

5. Scenario-based test generation

Describe a business scenario in natural language and let RepoPilot generate relevant test cases using repository context.

The goal isn't to replace an engineer's judgment. It's to reduce the repetitive work required to understand a codebase and establish a useful testing foundation.

The Engineering Challenge: Context Is Everything

One of the biggest challenges with AI-assisted development is context.

Large repositories cannot simply be passed into an LLM prompt in their entirety.

Even when a repository fits within a model's context window, providing every file is often inefficient.

It increases token consumption, introduces irrelevant information, and makes it harder to focus on the specific implementation being discussed.

I wanted RepoPilot to retrieve relevant code based on its actual structural relationships.

That led me toward a combination of AST parsing, dependency graphs, and targeted context building.

Step 1: Understanding Source Code with AST Parsing

RepoPilot's indexing pipeline uses TypeScript's compiler ecosystem to analyze JavaScript and TypeScript source files, Python AST for Python source files & Comment-aware source scan for Java.

Rather than treating code as plain text, the parser extracts structural information such as:

Imports and exports
Functions and classes
Method signatures
Source-file relationships
Relevant source-code segments

This provides a structured representation of the repository.

For example, consider:

import { PaymentService } from "./payment.service";

export async function checkout(order: Order) {
  const paymentService = new PaymentService();
  return paymentService.process(order);
}

A text-based search might identify the imported class name.

An AST-based approach gives us a more reliable way to identify the import relationship and the structure of the function containing it.

That information becomes valuable when deciding which related files should be included in an AI request.

Step 2: Building a Repository Dependency Graph

After parsing, RepoPilot constructs a knowledge graph representing relationships within the codebase.

Conceptually:

checkout.ts
    |
    | imports
    v
payment.service.ts
    |
    | imports
    v
payment.gateway.ts

Now, if someone requests tests for checkout.ts, we don't necessarily need to provide the entire repository.

We can start with the selected file and traverse relevant dependencies.

This allows the context-building layer to collect the implementation details most likely to matter.

The graph also creates a foundation for future capabilities such as impact analysis and architectural exploration.

Step 3: Building Context for the LLM

This was one of the more interesting architectural decisions.

I separated the code-analysis pipeline from the AI-generation pipeline.

At a high level, the flow looks like this:

Git Repository
      |
      v
Repository Parser
      |
      v
Dependency Graph
      |
      v
Context Builder
      |
      v
Prompt Builder
      |
      v
LLM Provider
      |
      v
Generated Tests / AI Responses

The context builder is responsible for identifying and collecting relevant code.

The prompt builder then transforms that information into task-specific instructions.

For example, generating unit tests requires a different prompt structure from answering an architectural question.

Keeping these responsibilities separate makes the system easier to extend, debug, and maintain.

It also allows the AI provider to evolve independently of the repository-analysis pipeline.

Step 4: Generating Tests from Real Code Context

One of RepoPilot's primary use cases is test generation.

Consider a function that validates an order, checks inventory, and processes payment.

A useful test suite should consider more than the happy path.

It might include:

Valid orders
Invalid input
Insufficient inventory
Payment failures
Dependency exceptions
Boundary conditions

By supplying the relevant implementation and dependency context, RepoPilot aims to produce more meaningful test suggestions than a generic prompt containing only a function name.

The generated tests still require developer review and execution. AI-generated tests are not automatically proof of correctness.

But they can significantly reduce the effort required to create an initial testing baseline.

Step 5: Scenario-Based Test Generation

I also wanted developers to describe tests in the language of business requirements.

For example:

Generate tests for a checkout flow where payment succeeds but inventory reservation fails.

RepoPilot uses the scenario and available repository context to generate relevant test code.

This is particularly useful when developers know the behavior they want to validate but haven't yet identified every implementation detail.

It also helps connect business-level requirements with actual source-code behavior.

The Technology Stack

Here's a simplified view of the technologies behind RepoPilot:

Layer	Technology
Frontend	Vue 3, TypeScript
Backend	Python, FastAPI
Code analysis	TypeScript Compiler API / ts-morph
Repository intelligence	Dependency graph and context builder
AI integration	Provider abstraction for LLM services
Authentication	Auth0
Source integrations	GitHub, GitLab, Bitbucket
Background processing	Asynchronous jobs and workers

One of my main goals was to keep the architecture modular.

I didn't want the repository parser, context builder, and LLM integration to become one large service that would be difficult to maintain.

What I Learned Building RepoPilot

A few lessons stood out during development.

1. Better context matters as much as better prompts.

Prompt engineering is useful, but identifying the correct source files and relationships is equally important.

2. Deterministic code analysis and generative AI complement each other.

AST parsing provides structural information. LLMs help interpret that information and generate useful outputs.

Neither approach needs to replace the other.

3. AI-generated tests require validation.

Generated code can contain incorrect assumptions, incomplete mocks, or invalid assertions.

Human review and execution remain essential.

4. Security becomes more complicated with private repositories.

Repository credentials must be handled carefully, with scoped access, secure storage, and protection against accidental exposure in logs or prompts.

5. Building a product is different from building a prototype.

Authentication, quotas, failure handling, deployment, observability, and user onboarding quickly become just as important as the AI functionality.

What's Next?

RepoPilot is still evolving.

Some areas I'm interested in exploring further include:

Better semantic retrieval for large repositories
More advanced dependency and impact analysis
Test quality evaluation
Improved support for different languages and frameworks
More intelligent, repository-aware development workflows

I'm deliberately keeping production preview execution out of the initial public release while I validate the security, infrastructure, and operational requirements.

For now, the focus is on repository intelligence and AI-assisted testing.

DEV Community: Arnab Lahiri