Lately, I’ve been deep in the weeds of building developer tooling and workflow optimizations. I’ve realized that the tools I build don’t necessarily need to be beneficial for the entire world; they just need to solve a specific friction point for me. This isn’t solipsism exactly, it’s a recognition that scratching your own itch often produces better abstractions than attempting to solve everyone’s problems simultaneously.
Photo by Annie Spratt on Unsplash
General-purpose tools dilute their opinionated effectiveness in exchange for flexibility; they become configuration engines rather than workflow accelerators. By narrowing scope to “works for my brain,” I can make aggressive assumptions about usage patterns, error recovery, and interface design that would be unacceptable in enterprise software.
I’ve been working on Christina, a Git commit message generator, because I wasn’t satisfied with the existing tools. I’d been using OpenCommit for a while, eventually forking it to implement nuanced changes. However, forking a project only takes you so far when you have a fundamentally different vision for performance.
Here’s the thing about Git diffs: they can be absolutely massive. A single dependency update can balloon your package-lock.json to 50,000 lines. A database migration might touch hundreds of files across schema definitions, generated ORM code, and migration metadata. And LLMs? LLMs have finite, expensive context windows. Past a certain point, adding more tokens doesn’t just waste money, it degrades output quality. The model can’t effectively prioritize signal over noise when both are drowning in a sea of lockfile churn.
You can’t just throw a 100KB diff at Gemini 3 Flash and hope for the best. Well, you can, but you’ll burn through your token budget faster than you can say “rate limit exceeded,” and you’ll likely get a vague summary that misses the architectural significance of the changes because the model’s attention mechanism got distracted by repetitive lockfile churn.
The challenge isn’t just about fitting things into a context window. It’s about maximizing signal while minimizing noise, which requires understanding the information entropy of different diff segments. The LLM doesn’t need to see every deleted line from that minified JavaScript bundle you’re removing. It doesn’t need the full contents of Cargo.lock, which is essentially a serialized dependency graph with high redundancy. It needs just enough context to understand what changed and why, enough to reconstruct the semantic intent without drowning in syntactic noise.
This is fundamentally a compression problem. Git diffs are already a form of delta compression, but they’re optimized for storage and patch application, not for semantic summarization by a neural network. We need a secondary compression layer that respects semantic boundaries, an understanding that package.json and package-lock.json carry different semantic weight per line, that test file changes often mirror implementation changes and can be summarized by reference rather than repetition.
The Recursive Dissection Strategy
The core idea is simple: break down a massive Git diff without losing the semantic context the LLM needs to understand the change. But “simple” in theory gets complicated fast in practice because semantic boundaries don’t align with byte boundaries. A function definition might span multiple hunks. A refactor might touch the signature in one file and the call sites in twenty others. The strategy needs to be lossy in terms of information volume but lossless in terms of semantic connectivity.
Level 1: The Greedy File Packer
The engine starts by trying to pack entire files into chunks using a First-Fit algorithm. This isn’t optimal from a theoretical computer science perspective, we could use bin-packing algorithms like Best-Fit Decreasing for better space utilization, achieving packing efficiency, but greedy is O(N) and bin-packing is O(N log N) at best. More importantly, greedy packing maintains file order, which matters semantically in ways that pure algorithmic efficiency ignores.
If you’re changing auth.ts, auth.test.ts, and auth.types.ts, you want them in the same chunk or consecutive chunks. The LLM can infer relationships between related files through proximity in context, this is the "locality of reference" principle applied to token windows. Optimal bin packing might scatter them across chunks to maximize packing density, destroying that contextual adjacency and forcing the model to reconstruct relationships without adjacency cues.
// If this single file fits in our budget
if combined_tokens <= token_limit.get() {
// Add to current chunk (truncate if lockfile)
if is_lockfile
&& file_diff.token_count > TokenCount::new_saturating(LOCKFILE_TOKEN_LIMIT)
{
let truncated = truncate_to_token_limit(
&file_diff.content,
TokenCount::new_saturating(LOCKFILE_TOKEN_LIMIT),
tokenizer,
);
buffer.content_mut().push_str(&truncated);
buffer
.content_mut()
.push_str("\n[... truncated lockfile ...]\n");
} else {
buffer.content_mut().push_str(&file_diff.content);
}
buffer.file_paths_mut().push(file_diff.path);
current_tokens = TokenCount::new(combined_tokens);
} else {
// Flush current chunk and start new one
if !buffer.is_empty() {
chunks.push(DiffChunk::new(
Arc::from(buffer.take_content()),
buffer.take_file_paths(),
current_tokens.unwrap_or_else(|| TokenCount::new_saturating(1)),
));
buffer.clear();
}
// ...
}
This is where we maintain maximum context: the LLM sees complete file headers and all changes within that file together. The file boundary acts as a natural semantic firewall, we assume files are generally cohesive units (modulo some exceptions like generated code), so keeping them intact preserves intra-file relationships like variable scoping and import dependencies.
Level 2: The Hunk-Level Splitter
When a single file’s diff exceeds the token limit, we drop down to splitting by hunks, those @@ -start,count +start,count @@ markers that Git uses to denote changed sections within a file.
This is a natural semantic boundary because hunks represent logically contiguous changes within a file, typically scoped to a function or a cohesive block of code. Git’s diff algorithm (the Myers diff or histogram diff depending on your configuration) already does the hard work of segmenting changes into minimal edit scripts. Splitting here preserves local context: the LLM still sees what function or section of code changed, even if it can’t see the entire file.
The tricky bit is handling the file header metadata. If you’re splitting a diff by hunks, each fragment needs the original file header (diff --git a/file.rs b/file.rs, index lines, mode changes) so the LLM knows what file it's looking at. Without this, you just have anonymous hunks floating in context space. But you only tokenize the header once per file during the fitting calculation, not once per chunk, because that would be wasteful and artificially deflate your capacity.
Another subtlety: hunks include context lines (the lines that haven’t changed but surround the changes). When splitting by hunks, you need to decide whether to duplicate context lines at the boundaries of chunks or truncate them. Christina duplicates up to 3 lines of context at chunk boundaries. This creates intentional overlap, redundancy that costs tokens but prevents semantic breakage when a change logically spans the boundary between two hunks. Without this overlap, renaming a variable might appear in one chunk as the removal and in another as the addition, looking like unrelated changes.
// Find the file header end (up to first hunk or end)
let header_end = content.find("\n@@").unwrap_or(content.len());
let header = &content[..header_end];
let header_tokens = tokenizer.count_tokens(header);
let header_tokens_count = header_tokens.get();
// Check if adding this hunk would exceed limit
if current_tokens + hunk_tokens_count > token_limit.get() {
if !buffer.is_empty() {
chunks.push(DiffChunk::new(
Arc::from(buffer.take_content()),
vec![file_path.clone()],
TokenCount::new_saturating(current_tokens),
));
buffer.clear();
}
// Start new chunk with header + hunk
buffer.content_mut().push_str(header);
buffer.content_mut().push('\n');
buffer.content_mut().push_str(hunk);
current_tokens = header_tokens_count + hunk_tokens_count + 1;
} else {
buffer.content_mut().push('\n');
buffer.content_mut().push_str(hunk);
current_tokens += hunk_tokens_count + 1;
}
Level 3: Smart Line Slicing
It happens rarely, but it happens, you encounter a single hunk that’s too large. Maybe someone committed a minified JSON blob with one line containing 10,000 characters, or there’s a giant SQL migration that inserts thousands of records in a single multi-row INSERT statement, or a base64-encoded file that someone really shouldn’t have committed but did.
At this point, you fall back to line-by-line splitting. This breaks semantic units, which is unfortunate because a single line of code might contain multiple statements or a complex expression. But it’s better than failing entirely or dropping the change on the floor.
There’s a heuristic here about which lines to prioritize if you can only fit a subset. Christina prioritizes:
- Lines starting with + (additions) over - (deletions) because the new state is usually more relevant than the old
- Lines containing keywords like TODO, FIXME, hack, bug through simple regex matching, on the theory that these indicate high-intent changes
- The first and last N lines of the hunk, under the assumption that changes usually have primacy and recency bias in importance
This is a form of semantic triage, acknowledging that when we must lose information, we should lose the least important information first.
Level 4: Binary Search for Byte-Level Precision
And then there’s the truly pathological case: a single line that exceeds your token budget.
BPE (Byte Pair Encoding) tokenizers don’t have a fixed character-to-token ratio. A line of common English prose might compress to roughly 1 token per 4 characters due to frequent subword units in the vocabulary. A line of random hex strings or minified code might approach 1 token per 1–2 characters because the tokenizer can’t find efficient subword compressions for high-entropy strings. You can’t just slice at the character midpoint and expect the token count to halve.
So we binary search for the longest UTF-8-safe slice that fits:
while low <= high {
let mid = (low + high) / 2;
// Ensure mid is at a UTF-8 character boundary
let mut adjusted_mid = mid;
while adjusted_mid > start && !line.is_char_boundary(adjusted_mid) {
adjusted_mid -= 1;
}
// Guard against zero progress
if adjusted_mid == start {
// Take at least one character
adjusted_mid = start + 1;
while adjusted_mid < line.len() && !line.is_char_boundary(adjusted_mid) {
adjusted_mid += 1;
}
}
let slice = &line[start..adjusted_mid];
let tokens = tokenizer.count_tokens(slice);
if tokens <= token_limit {
best = adjusted_mid;
low = mid + 1;
} else {
high = mid - 1;
}
}
This handles multi-byte UTF-8 (emoji, CJK characters, mathematical symbols) by checking is_char_boundary() at every potential split point. The guard clause ensures we always make progress, at minimum, we advance by one complete UTF-8 character, which prevents infinite loops that could occur if we naively adjusted without the decrement guarantee.
The binary search is O(log N) in the line length, with each iteration requiring a tokenizer call. For o200k_base tokenizer, this is relatively fast, but with local models using unoptimized tokenizers, this could be a bottleneck.
There’s also the question of where to split within a line. Christina prefers to split at word boundaries (spaces) when possible, searching backward from the binary-search-determined midpoint to find the nearest space. If no space exists within a 10-character window, it splits at the byte boundary and inserts a [split] marker to indicate discontinuity. This preserves readability better than mid-word truncation.
Overengineered Performance with Buffer Pooling
In Rust, high-frequency string manipulation can lead to significant allocation overhead when using the default allocator. Processing a large diff involves creating thousands of intermediate strings, temporary buffers for truncated content, concatenated hunks, file paths collections. If you allocate and drop these “scratchpads” constantly, you get memory fragmentation, allocator lock contention in multi-threaded contexts, and cache pollution from zeroing memory.
The solution is a thread-local buffer pool using an object pool pattern:
thread_local! {
static BUFFER_POOL: RefCell<Vec<ChunkBuffer>> = const { RefCell::new(Vec::new()) };
}
pub(crate) fn acquire_buffer() -> ChunkBuffer {
BUFFER_POOL.with(|pool| {
let mut pool = pool.borrow_mut();
match pool.pop() {
Some(mut buffer) => {
buffer.clear();
buffer
}
None => ChunkBuffer::new(),
}
})
}
Why thread-local? Because diff chunking is CPU-bound and runs on a threadpool. Thread-local storage (TLS) avoids mutex overhead or atomic operations , there’s no cross-thread contention because each thread has its own pool, leveraging the fact that Rust’s thread model gives us true OS threads with separate stacks and TLS segments.
Each buffer is pre-allocated to 4KB, which is a typical chunk size and aligns well with memory page boundaries on most architectures (though we don’t explicitly align, the allocator typically rounds up). When you return a buffer to the pool via release_buffer(), clear() resets the length to zero but keeps the allocated capacity instead of deallocating. This prevents the allocator from returning the memory to the global heap and potentially unmapping pages.
The pool is capped at 16 buffers per thread (16 × 4KB = 64KB max overhead), which prevents unbounded memory growth in long-running processes. In pathological cases with extremely bursty traffic, we simply drop excess buffers on the floor rather than hoarding memory. This is a classic memory-time tradeoff: we trade 64KB of resident memory per thread for avoiding thousands of allocations per commit.
There’s also the consideration of allocator choice. If you’re using a modern allocator like mimalloc or jemalloc, the benefit of pooling diminishes because these allocators have thread caches and size-class binning that make small allocations cheap. However, on Windows with the default system allocator or in constrained environments, the buffer pool provides consistent performance characteristics across platforms, insulating us from allocator behavior differences.
Heuristics and Noise Reduction
Not all diff content is created equal. Information theory tells us that highly predictable content (like lockfile updates) carries low entropy and thus low information value per byte. The LLM doesn’t need to see the specific contents to understand “updated dependencies”, it just needs confirmation that this is indeed a routine lockfile update, not a malicious injection or a hand-edited dependency resolution.
Lockfiles (package-lock.json, Cargo.lock, yarn.lock, go.sum) are auto-generated noise with high internal redundancy. The LLM doesn't need to see 10,000 lines of dependency version updates to generate "update dependencies" as a commit message. Christina truncates lockfiles to 100 tokens, about 25 lines which is enough to show intent (the file header plus a sample of the changes) without wasting the context budget.
pub const LOCKFILE_TOKEN_LIMIT: u32 = 100;
// Add new file to fresh buffer
if is_lockfile
&& file_diff.token_count > TokenCount::new_saturating(LOCKFILE_TOKEN_LIMIT)
{
let mut truncated = truncate_to_token_limit(
&file_diff.content,
TokenCount::new_saturating(LOCKFILE_TOKEN_LIMIT),
tokenizer,
);
truncated.push_str("\n[... truncated lockfile ...]\n");
buffer.content_mut().push_str(&truncated);
} else {
buffer.content_mut().push_str(&file_diff.content);
}
The 100-token limit isn’t arbitrary , it’s based on the observation that most lockfile changes follow a power-law distribution: a few dependencies change significantly, most change by version bump only. 25 lines typically captures the “interesting” changes (major version bumps, new dependencies) while excluding the long tail of patch updates.
Similarly, deletion-only diffs get special treatment. If you’re deleting entire files, the LLM just needs to know what was deleted, not the full contents of the corpse. Christina detects these cases and heavily truncates, showing only the file paths and the first few lines (in case there’s a header comment explaining what the file was):
// Truncate deletion-only diffs to save tokens
// The LLM doesn't need to see all deleted content to generate "delete file" messages
if parsing::is_all_file_deletions(diff) {
// All files are being deleted - heavily truncate
return self.process_owned(parsing::truncate_deletion_diff(diff, 3));
} else if parsing::is_deletion_only(diff) {
// Only deletions (no additions) - moderately truncate
return self.process_owned(parsing::truncate_deletion_diff(diff, 10));
}
The distinction between all_file_deletions and deletion_only matters. When deleting entire files (a "remove dead code" commit), the content is irrelevant. When deleting content within a file (removing a function but keeping the file), some context helps the LLM understand what functionality was removed without seeing the full implementation.
Binary assets are replaced with [Binary file: path.png], which is still a sensible default for review because raw bytes aren’t human-readable. If you’re using a vision-capable model, though, it can interpret the actual image content meaningfully. A practical approach is to attach the relevant assets when visual changes matter, rather than trying to inline base64 blobs in the patch, but that’s complexity we don’t need yet.
Photo by Tim Gouw on Unsplash
The exact thresholds are provisional. Why 100 tokens for lockfiles? Why 3 lines for deletion previews? Not because those values are special, but because they create a measurable ceiling on noise while still exposing recognizable structure.
These constants exist to make the system observable. Once commit quality, latency, and token usage are measured in real workflows, they become tuning parameters rather than opinions. Until then, they’re deliberately conservative defaults that keep the system predictable.
Parsing as Attack Surface
One thing that’s easy to overlook in developer tooling: diff parsing is a potential attack vector. Git diffs are text with conventions, not a strict format. Treat them like structured data at your peril.
Malicious actors can craft diffs with fake headers embedded in file content to corrupt parsing or cause the tool to hallucinate changes that don’t exist. For example:
diff --git a/real.txt b/real.txt
+some code
+more code with diff --git a/fake.txt b/fake.txt embedded
If your parser uses contains("diff --git") rather than anchored matching, it might treat that embedded string as a new file boundary, causing the chunker to split incorrectly and potentially miss the malicious payload or misattribute changes to the wrong file. This is analogous to HTTP header injection or CSV injection attacks, context-sensitive parsing that fails to distinguish between metacharacters and data.
The parser needs to only treat diff --git as a header when it appears at line start:
for line in diff.lines() {
if line.starts_with("diff --git ")
&& let Some(path) = parse_git_diff_header(line)
{
paths.push(path);
}
}
But anchors aren’t enough. You also need to handle the case of newlines in filenames (yes, Git supports_files with newlines in names, though it’s pathological). diff --git line format is formally diff --git a/
There’s also the question of Unicode normalization. macOS uses NFD (decomposed) form for filenames, while Linux typically uses NFC (composed). A diff generated on macOS might have decomposed UTF-8 in the header, while the filesystem expects composed. If we treat these as different strings, we might fail to correlate diff chunks with working tree files. Christina NFC-normalizes all paths internally, accepting the slight performance cost for consistency.
It’s a small detail, but security often is. The difference between contains("diff --git") and starts_with("diff --git") is the difference between a working tool and a security vulnerability that could be exploited to hide malicious code in generated commit messages or cause the tool to emit misleading metadata.
A Good Enough Engine (Maybe?)
So, does it work? Honestly? I think it might. I’ve run it against synthetic diffs, single-line typo fixes to medium refactoring across a handful of files. It hasn’t crashed on pathological inputs: null bytes, mixed encodings. The commit messages look decent: occasionally inspired, usually plausible, rarely obviously wrong.
When you’re building tools for yourself, ‘good enough’ isn’t a benchmark — it’s a feeling. Does the tool disappear into your workflow? I’m not there yet. Right now I’m optimizing for ‘does this function correctly’ rather than that flow state where you’re just thinking about code, not the commit message you’re writing.
At the time of writing, Christina is still being built. I’m documenting ideas as much as reporting results.
The chunking strategy should preserve context across architectural layers, but that’s theoretical. The buffer pooling should keep it responsive, but I haven’t measured latency under real load. Everything about scale is speculative. Production-grade? Not by my definition, reliable daily driver that doesn’t lose work or misrepresent changes. I’m nowhere near confident yet. It’s a promising prototype that solves a specific friction point in theory.
There’s a temptation to polish before publishing, to wait until everything is measured and proven. But the interesting decisions are visible mid-process, when you’re still uncertain. Once a tool disappears into your workflow, you forget why you chose these tradeoffs. I want to remember, and maybe these choices are useful to someone else wrestling with the same friction
Top comments (0)