How TokenCap Eliminates Context Query Latency with Background Pre-Warming

When integrating browser companions or IDE extensions with a local intelligence bridge, latency matters. If generating prompt context takes 4 or 5 seconds, developers stop using the integration.

To deliver sub-second responses, TokenCap implements background cache pre-warming in src/bridge/warmCache.js.

How Pre-Warming Operates

When you run tokencap serve, the HTTP server begins listening immediately. While the pairing handshake occurs, an asynchronous worker kicks off:

1. Graph Pre-Compilation: Traverses all files and loads AST symbol tables into memory.
2. Memory and ADR Indexing: Loads active session records and architecture decisions into RAM.
3. Git Diff Snapshotting: Caches uncommitted working tree diffs and affected symbols.

// src/bridge/warmCache.js logic
function preWarmBridgeContext(repoRoot) {
  setImmediate(() => {
    // Builds query-independent artifacts concurrently
    const graph = getOrBuildGraph(repoRoot);
    const memory = loadActiveMemory(repoRoot);
    const diff = computeWorkingDiff(repoRoot);
    bridgeCache.set(repoRoot, { graph, memory, diff, timestamp: Date.now() });
  });
}

Response Time Comparison

• Cold Generation (No Pre-Warm): 1,840ms
• With Pre-Warmed Cache: 180ms (10x faster first-click response)

When the Chrome Companion requests task-scoped context via POST /generate, TokenCap slices the pre-warmed graph and formats the prompt payload almost instantaneously.

Documentation available at tokencap.vansharora.app