Ramu Narasinga

Posted on Apr 20

AstCodeSplitter in claude-context codebase.

#opensource #ast #treesitter #claude

In this article, we review AstCodeSplitter class in claude-context codebase. You will learn:

What is Claude Context?
AstCodeSplitter class.
tree-sitter package.

What is Claude Context?

Claude Context is an MCP plugin that adds semantic code search to Claude Code and other AI coding agents, giving them deep context from your entire codebase.

🧠 Your Entire Codebase as Context: Claude Context uses semantic search to find all relevant code from millions of lines. No multi-round discovery needed. It brings results straight into the Claude’s context.

💰 Cost-Effective for Large Codebases: Instead of loading entire directories into Claude for every request, which can be very expensive, Claude Context efficiently stores your codebase in a vector database and only uses related code in context to keep your costs manageable.

Architecture

Below is the architecture from README.md.

I wanted to learn how the re-indexing is done, I found this in the README.

⚡ Incremental Indexing: Efficiently re-index only changed files using Merkle trees.

Learn more about Claude Context.

AstCodeSplitter class

AstCodeSplitter implements Splitter interface.

export class AstCodeSplitter implements Splitter {

and this interface is defined as shown below:

export interface Splitter {
    /**
     * Split code into code chunks
     * @param code Code content
     * @param language Programming language
     * @param filePath File path
     * @returns Array of code chunks
     */
    split(code: string, language: string, filePath?: string): Promise<CodeChunk[]>;

    /**
     * Set chunk size
     * @param chunkSize Chunk size
     */
    setChunkSize(chunkSize: number): void;

    /**
     * Set chunk overlap size
     * @param chunkOverlap Chunk overlap size
     */
    setChunkOverlap(chunkOverlap: number): void;
}

You will find these three functions defined in the AstSplitter class.

It also has the below private defined that are used in the interface methods implemented.

tree-sitter package

extractChunks function has its first parameter type as Parser.SyntaxNode. Parser is imported as shown below:

import Parser from 'tree-sitter';

So what’s tree-sitter? I read this npm package description and found this below example:

const Parser = require('tree-sitter');
const JavaScript = require('tree-sitter-javascript');

const parser = new Parser();
parser.setLanguage(JavaScript);

// parse some code:
const sourceCode = 'let x = 1; console.log(x);';
const tree = parser.parse(sourceCode);

// and inspect the syntax tree.

console.log(tree.rootNode.toString());

// (program
//   (lexical_declaration
//     (variable_declarator (identifier) (number)))
//   (expression_statement
//     (call_expression
//       (member_expression (identifier) (property_identifier))
//       (arguments (identifier)))))

const callExpression = tree.rootNode.child(1).firstChild;
console.log(callExpression);

// {
//   type: 'call_expression',
//   startPosition: {row: 0, column: 16},
//   endPosition: {row: 0, column: 30},
//   startIndex: 0,
//   endIndex: 30
// }

About me:

Hey, my name is Ramu Narasinga. Email: ramu.narasinga@gmail.com

Tired of AI slop?

I spent 3+ years studying OSS codebases and wrote 350+ articles on what makes them production-grade. I built an open source tool that reviews your PR against your existing codebase patterns.

Your codebase. Your patterns. Enforced.

Get started for free — thinkthroo.com

DEV Community