ec1980

Posted on May 19

I Kept Hitting Claude Code's 5-Hour Limit After 2 Hours. So I Built This.

#ai #claude #opensource #productivity

If you use Claude Code heavily, you've probably hit the wall.

You're deep into a session, the agent is finally understanding your codebase, and then — usage limit reached. Not after 5 hours. After 2.

Aside from the limit, a major contributing factor is that every coding session re-sends the same project context over and over. Even with prompt caching, the tokens stack up fast on a large repo, and you burn through your window faster than you'd expect.

So I built L-SDF — Latent-Structured Documentation Format.

What Is L-SDF?

L-SDF generates compact index files (.lsdf) that give AI coding agents a structural map of your codebase — before they ever open a source file.

Instead of an agent reading 110K tokens of raw source code to find its way around, it reads an 8K token index first. It only drills into source files when it actually needs implementation details.

The result: significantly reduced input token costs compared with reading source directly, even with prompt caching — which means your usage limit stretches a lot further in navigation-heavy coding sessions.

It works today with Claude Code, Cursor, and Copilot — and because it's a plain file format, it's not tied to any one tool.

The Token Economics

Here's what the savings look like on a typical Python repo:

Metric	Value
Source tokens (21 files)	110,432
L-SDF index tokens	8,241
Compression ratio	13.4×

And in terms of estimated input token session cost (output and other costs not included):

Scenario	Est. 50-turn session cost (input tokens only)
Source code, with caching	$2.03
L-SDF + caching	$0.55
Savings	~73% (~4×)

How It Works

L-SDF uses a hierarchical sigil-based topology to represent your codebase structure. Each directory gets two index files:

INDEX.lsdf — shows what exists (compact structural map)
INDEX.detail.lsdf — shows signatures, schemas, and call edges

The agent reads INDEX.lsdf first to navigate. It only opens INDEX.detail.lsdf when it needs signatures or call edges, and opens source files only when it needs the implementation body. This is the key insight: agents don't need to read your source files to understand where things are.

Here's a simple example. Given this Python file:

# hello.py
import sys

class Greeter:
    def say_hello(self, name: str) -> str:
        if not name:
            raise ValueError("Name must not be empty")
        message = f"Hello, {name}!"
        print(message)
        return message

    def greet(self, names: list[str]) -> list[str]:
        return [self.say_hello(n) for n in names if n.strip()]

L-SDF generates two index files:

INDEX.lsdf:

@hello.py
 ~sys
 @Greeter
  !say_hello
  !greet

INDEX.detail.lsdf:

@hello.py
 ~sys
 @Greeter
  !say_hello(name:s):s
  !greet(names:[s]):[s] > say_hello

In a handful of tokens the agent knows: there's a file hello.py, it imports sys, contains a class Greeter with two methods — their signatures, return types, and that greet calls say_hello. No implementation details, no noise — just the map.

Quick Start

L-SDF is distributed as a global CLI tool. Install it with pipx so the lsdf command is available across all your projects:

# Install pipx if you don't have it (Ubuntu/Debian)
sudo apt install pipx
pipx ensurepath

# Install lsdf-core
pipx install lsdf-core

# Verify
lsdf --help

Then three commands to set it up in any repo:

# Initialize L-SDF in your repo
lsdf init

# Generate index files recursively
lsdf gen . --recursive

# See your token savings estimate
lsdf stats

Contributors: If you have the repo checked out locally and want to work on the source, see the repo README for editable install and test instructions.

What lsdf init Creates

project.lsdf
.lsdf/
  lsdf_instructions.md
  lsdf_spec.md
.lsdfignore

It also detects your stack and skips irrelevant paths (__pycache__, .venv, etc.) automatically via .lsdfignore.

lsdf init automatically appends instructions to CLAUDE.md and .cursorrules so your agents know to use the index files right away.

How It Compares to Other Approaches

vs. Vector embeddings (Cursor's built-in RAG)
Cursor injects context via vector embeddings — powerful but opaque. You don't control what gets sent or at what cost. L-SDF is an explicit, versionable map in your repo. You can inspect it, version it, and it behaves consistently across every agent. The tradeoff is determinism and zero runtime overhead instead of recall flexibility.

vs. Workflow governance tools (like Haxaml)
Tools like Haxaml govern how the agent works through a task (prepare, build, verify, record). L-SDF governs what the agent reads — giving it a structural map so it navigates without opening files. They're complementary, not competitive. You could run both.

vs. AGENTS.md / CLAUDE.md / .cursorrules
These are prompt files — instructions to the agent. L-SDF is the structured data underneath them. lsdf init writes to these files automatically so they point agents at the index.

vs. Graphify
Graphify turns your entire project — code, docs, PDFs, images, videos — into a queryable knowledge graph with an interactive HTML visualization. It's a powerful tool for onboarding, architecture exploration, and cross-repo discovery. L-SDF is more narrowly focused: two compact index files per directory, optimized for low-token coding navigation during active sessions. Graphify's output is rich and explorable; L-SDF's output is minimal by design. They're complementary — Graphify for understanding a codebase, L-SDF for navigating it efficiently while coding.

Index Drift

One concern people raise: what happens when the code changes and the index goes stale?

L-SDF has three layers of defense:

1. Auto-regeneration after structural edits. The agent is instructed to run lsdf gen <dir> after any structural change. You do the same when editing manually.

2. lsdf sync as an enforcement check. Run it in CI or as a pre-commit hook:

lsdf sync . --check

The exit code is non-zero if any index is out of date. Wire this into your CI's required checks and stale indices stop reaching main.

3. Auto-regeneration on push via lsdf init --ci. The strongest enforcement — regenerates the index on every push. Requires write permissions on the branch and may add some commit noise, but gives you the highest confidence that the map is always current.

What's Next

L-SDF currently supports Python repos. The format is language-agnostic and open — generators for Go, Rust, and TypeScript are on the roadmap and contributions are very welcome.

The community has also asked for benchmarks across repo sizes and context thread lengths. Rather than publishing self-generated data, I'd love for early adopters to share their own numbers.