Building a Token-Efficient Retrieval Layer for claude-plugins-official

Building a Retrieval Layer for claude-plugins-official Without Turning Plugin Discovery Into a Token Sink

At 3:00 AM, the incident is not caused by the plugin search returning the wrong answer; it is caused by the search returning too much. Quotas drain, latency cliffs appear, and stale documentation quietly becomes production guidance. The difficult engineering problem in using anthropics/claude-plugins-official as a developer knowledge source is not embedding quality. It is context discipline.

A plugin directory contains descriptions, commands, agents, skills, examples, and configuration files. Naive ingestion concatenates every Markdown file, embeds arbitrary fragments, and sends the top results to Claude Code. Repeated descriptions outrank implementation details, unrelated plugins consume the context window, and obsolete examples are presented as authoritative configuration.

Treat the repository as a versioned catalog with code-adjacent documentation, not as one large document.

Architecture

The retrieval service should have four separate responsibilities:

1. Repository ingestion reads a pinned commit and extracts eligible files.
2. Normalization and deduplication removes repeated catalog text while preserving plugin boundaries.
3. Retrieval selects a small number of chunks using metadata filters and semantic search.
4. Context assembly produces a bounded prompt with source paths and commit identity.

The repository remains the source of truth. The retrieval index is disposable and rebuildable from a commit SHA. Generated summaries must not be stored as though they were upstream documentation.

A useful metadata record includes:

repository: anthropics/claude-plugins-official
commit: "PINNED_COMMIT_SHA"
include:
  - "plugins/**/README.md"
  - "plugins/**/*.md"
exclude:
  - "**/node_modules/**"
  - "**/.git/**"
  - "**/dist/**"
chunk:
  target_tokens: 420
  overlap_tokens: 60
retrieval:
  top_k: 8
  max_context_tokens: 2600
llm:
  base_url: "https://b-lost.com/v1"
  model: "claude-sonnet"
  temperature: 0
  max_tokens: 700
cache:
  static_prefix: true

The model identifier depends on the gateway and account configuration. The important properties are a pinned revision, a bounded context, and a replaceable relay endpoint.

Chunking and deduplication

Chunk by Markdown headings where possible, then enforce a token budget. Character-only splitting loses command names and configuration semantics. Heading-only splitting fails on long reference sections.

The following dependency-light script approximates four characters per token and can run during CI ingestion:

from __future__ import annotations

import hashlib
import re
from pathlib import Path

ROOT = Path("claude-plugins-official")
MAX_CHARS = 1680       # approximately 420 tokens
OVERLAP_CHARS = 240    # approximately 60 tokens

def normalize(text: str) -> str:
    text = text.replace("\r\n", "\n")
    text = re.sub(r"[ \t]+", " ", text)
    text = re.sub(r"\n{3,}", "\n\n", text)
    return text.strip()

def split_sections(text: str) -> list[str]:
    parts = re.split(r"(?m)(?=^#{1,6}\s+)", text)
    return [part.strip() for part in parts if part.strip()]

def chunk(section: str) -> list[str]:
    if len(section) <= MAX_CHARS:
        return [section]

    result = []
    start = 0
    while start < len(section):
        end = min(start + MAX_CHARS, len(section))
        piece = section[start:end]

        # Prefer a paragraph boundary over cutting a command example.
        boundary = piece.rfind("\n\n")
        if boundary > MAX_CHARS // 2:
            end = start + boundary

        result.append(section[start:end].strip())
        if end >= len(section):
            break
        start = max(end - OVERLAP_CHARS, start + 1)

    return result

seen: set[str] = set()

for path in ROOT.glob("plugins/**/*.md"):
    if any(part in {".git", "node_modules", "dist"} for part in path.parts):
        continue

    text = normalize(path.read_text(encoding="utf-8"))
    for section in split_sections(text):
        for body in chunk(section):
            fingerprint = hashlib.sha256(
                re.sub(r"\s+", " ", body.lower()).encode()
            ).hexdigest()

            if fingerprint in seen:
                continue

            seen.add(fingerprint)
            print({
                "source": str(path),
                "text": body,
                "fingerprint": fingerprint,
            })

Replace print with an embedding write or SQLite export. Store source, plugin name, heading, commit SHA, and file hash as filterable fields.

Deduplication must remain conservative. Exact or normalized-text duplicates are safe to remove. Near-duplicate detection may collapse plugins that share installation text but differ in commands or permissions. Preserve path identity and deduplicate repeated bodies only.

Context assembly and caching

Never pass raw search results directly to the model. Enforce the budget and remove repeated sources first:

def assemble_context(results: list[dict], max_chars: int = 10400) -> str:
    blocks = []
    used = set()
    total = 0

    for item in results:
        source = item["source"]
        text = item["text"].strip()

        if source in used or not text:
            continue

        block = f"Source: {source}\n{text}"
        if total + len(block) > max_chars:
            break

        used.add(source)
        blocks.append(block)
        total += len(block)

    return "\n\n---\n\n".join(blocks)

The static prefix should contain system instructions and repository policy. The dynamic suffix should contain the question and retrieved excerpts. If the API supports prompt caching, mark the stable prefix using the provider’s request format. Do not assume that a relay gateway preserves cache metadata; verify it through request logs and response headers.

For 1,000 monthly requests, an uncached request containing a 2,600-token repository prefix plus 700 input tokens consumes approximately 3.3 million input tokens. At a hypothetical $3 per million tokens, that costs about $9.90. If 80 percent of the prefix is cached at one-tenth the uncached rate, the effective billable input falls to roughly 1.22 million tokens, or $3.66. Actual savings depend on cache pricing, minimum cacheable length, and relay behavior.

Caching works only when the prefix is stable. Timestamps, random identifiers, and reordered metadata cause cache misses.

Operational boundaries

Generated catalog text can dominate retrieval; use path filters and fingerprints before generation. Stale instructions survive updates unless every chunk carries a commit SHA and the index is invalidated when the pinned revision changes. Retrieval may recommend a plugin without exposing operational requirements, so the answer layer must distinguish documentation from executable validation.

When benchmarking multi-provider fallback through B-Lost’s relay gateway at https://b-lost.com/v1, compare identical prompts, chunk IDs, cache metadata, latency, and output length. Otherwise, lower token consumption may simply mean less context. Treat embedding-store failures as rebuildable index failures, but treat an unavailable upstream repository as a release-blocking ingestion failure.

The hardest operational trade-off is between retrieval precision and instruction completeness. Smaller chunks improve targeting but increase boundary errors; larger chunks preserve context but waste tokens. Heading-aware chunks near 400 tokens, strict source deduplication, and a 2,600-token ceiling are a practical starting point, but real plugin-discovery queries—not synthetic similarity scores—must tune them.

What is your retrieval boundary when preserving complete operational instructions conflicts with keeping every request economically bounded?