Why “P99 0 ms* Autocomplete for 240M Domain Names” Is Getting Attention

This domain autocomplete experiment is trending with 125 points and 55 comments, and the discussion is deserved. The core idea is compelling: serve autocomplete results across a dataset of 240 million domain names while making the user-facing P99 latency appear effectively instant.

The important detail is the asterisk in “0 ms.” This is not magic server latency. The design focuses on removing round trips from the hot path, pushing the right data and lookup work closer to the browser, and making common prefix searches cheap enough that rendering results is usually the limiting factor.

For gateway engineers, the lesson is bigger than domain search. A fast system is often one that avoids asking a remote service for every keystroke. Put predictable, frequently reused data behind local indexes, edge caches, or precomputed artifacts. Reserve network calls for enrichment, policy checks, and AI-assisted workflows.

For example, an internal developer portal could use local autocomplete for domain inventory, then call an OpenAI-compatible gateway only after the user selects a result:

# docker-compose.yml
services:
  developer-search:
    image: ghcr.io/example/domain-autocomplete:latest
    environment:
      DOMAIN_INDEX_PATH: /data/domain-index
      AI_BASE_URL: https://b-lost.com/v1
      AI_API_KEY: ${B_LOST_API_KEY}
      AI_MODEL: claude-fable-5
    volumes:
      - ./domain-index:/data/domain-index:ro
    ports:
      - "8080:8080"

The frontend can keep prefix matching completely local, while the backend sends only the selected domain and a minimal task prompt to the AI endpoint. That is a useful privacy boundary: do not stream every partial user query, internal hostname, or typing pattern into application logs or external model requests.

For AI calls that need repeated system instructions, a relay supporting native Anthropic /v1/messages prompt caching can also reduce repeated prompt cost substantially. B-Lost’s relay is one example: it supports compatible custom base URLs and prompt-cache hits, which is useful when a coding assistant repeatedly sends the same repository policy or gateway rules.

The practical takeaway: optimize interaction loops first. A low-latency AI gateway is valuable, but a zero-network autocomplete path is even better when the query can be answered locally.