Building RAG Search and Q&A for a Game Site with Only Cloudflare

No servers and no separate vector database: Workers + Workers AI + Vectorize + KV. A single Worker handles ingestion, hybrid retrieval and streaming Q&A.

What it does

NESBox is an online multiplayer platform for retro games that I built. The library has over a hundred games, and their descriptions are a mix of Chinese, English and Japanese. Players often can't remember a game's name, only something like "that off-road motorbike one" or "a fighting game two people can play together".

So I added two AI features, both in the global search box (⌘K):

• Semantic search: describe a game in natural language to find it. The results are merged with the local matches by name and pinyin.
• AI Q&A: type a question starting with ?. If none of the local help entries match, an answer grounded in the game library streams in, in the user's current UI language.

The backend is one Cloudflare Worker with four endpoints:

Endpoint	Purpose
/sync	Full sync: fetch every game, rebuild the vector and BM25 indexes, stream progress line by line
/update	Update a single game
/search	Semantic search, returns game ids sorted by relevance
/completions	RAG Q&A, streamed as SSE

Architecture

                  ┌──────────────── Ingestion (/sync, /update) ────────────────┐
GraphQL game data ┤ parse description + labels + Wikipedia → chunk → bge-m3 ├─→ Vectorize (vectors + text)
                  │                                    └→ tokenize ─────────┼─→ KV (BM25 inverted index)
                  └────────────────────────────────────────────────────────┘

query → vector recall ┐
      → BM25 recall   ┴→ RRF fusion → (optional rerank) → top N chunks
                                                         ├─→ /search: dedupe by game, return ids
                                                         └─→ /completions: prompt → LLM → SSE
                                                               (retrieval and answers cached in KV)

What each service does:

Service	Role
Workers	Endpoints and all the logic
Workers AI	@cf/baai/bge-m3 for embeddings, @cf/baai/bge-reranker-base for optional reranking, @cf/nvidia/nemotron-3-120b-a12b for answers
Vectorize	Vectors, with each chunk's text stored in metadata
KV	Response cache, BM25 inverted index, chunk count per game

Declaring the bindings in wrangler.jsonc is all it takes. There are no API keys to manage:

{
  "vectorize": [{ "binding": "GAMES_SEARCH", "index_name": "nesbox-games-m3" }],
  "kv_namespaces": [{ "binding": "KV", "id": "..." }],
  "ai": { "binding": "AI" }
}

1. Ingestion

Building the text for each game

Game data comes from the site's own GraphQL API. Each game's text has two parts:

• Header: the game name plus its platform, genre and series labels in both Chinese and English, for example 越野机车 红白机 Nes 赛车竞速 Racing (Excitebike, Famicom, NES, Racing). The labels reuse the frontend's i18n strings, so Chinese and English queries both match.
• Body: the game description, in Markdown with images removed and links reduced to their text. If the description's frontmatter has a reference link, that page is fetched, converted to Markdown and appended. Wikipedia pages go through the MediaWiki action=parse API, which returns clean article HTML without the navigation and sidebars.

Chunking

Chunking splits text into sentences with Intl.Segmenter, the ICU segmenter built into the Workers runtime. A single instance handles mixed Chinese, Japanese and English text, so there are no punctuation regexes to maintain:

const sentenceSegmenter = new Intl.Segmenter('zh', { granularity: 'sentence' });

export function chunkText(text: string, maxChars = 800, overlap = 100) {
  const sentences = text
    .split(/\n+/)
    .flatMap((p) => Array.from(sentenceSegmenter.segment(p), (s) => s.segment.trim()))
    .filter(Boolean);
  // Merge sentences into chunks of at most maxChars.
  // When a new chunk starts, carry over whole sentences from the end of the
  // previous chunk, up to `overlap` characters, as the overlap.
  // ...
}

The rules:

• Each chunk is at most 800 characters, and neighboring chunks overlap by whole sentences, so no sentence gets cut in half.
• Every chunk starts with the header. A chunk retrieved on its own still says which game it belongs to and what kind of game it is.
• Each game gets at most 24 chunks, so a long Wikipedia article can't drive up inference costs.
• Vector ids are gameId#chunkIndex. The chunk count per game is stored in KV. If a game has fewer chunks after re-ingestion, the extra old chunks are removed with deleteByIds, so no stale content is left behind.

Embedding and writing

• Embeddings use @cf/baai/bge-m3: multilingual, 1024 dimensions, and it works well on mixed Chinese, Japanese and English content. Texts are sent 50 per request with truncate_inputs on, so overly long input is truncated instead of failing.
• Vectors are written to Vectorize in batches of 500. Each chunk's text lives in its metadata, so retrieval can use it directly without another database lookup.
• The BM25 inverted index is built in the same pass, tokenized with Intl.Segmenter at word granularity, and stored as a single KV value. A full sync rebuilds it, and a single update replaces only that game's chunks. At this data size, that is plenty.

/sync takes a while, so it returns a TransformStream and writes progress as it goes:

$ curl -N https://<worker>/sync
{"status":"start"}
{"status":"info","total":164}
{"status":"info","chunks":223}
{"status":"info","terms":...}
{"status":"done"}

2. Retrieval

Hybrid recall with RRF

Vector search and BM25 run in parallel, each recalling the top 20 chunks, and the results are merged with RRF (Reciprocal Rank Fusion):

const [vectorMatches, bm25Matches] = await Promise.all([
  embedding(env.AI, [query]).then(async ([values]) =>
    (await env.GAMES_SEARCH.query(values, { topK: 20, returnMetadata: 'all' })).matches),
  env.KV.get<Bm25Index>('bm25:index', 'json').then((index) => (index ? bm25Search(index, query, 20) : [])),
]);
const fused = rrf([vectorMatches.map((e) => e.id), bm25Matches.map((e) => e.id)]);

RRF looks only at each result's rank, not its score. That sidesteps the problem that cosine similarity and BM25 scores are on different scales and can't simply be added:

export function rrf(lists: string[][], k = 60) {
  const scores = new Map<string, number>();
  for (const list of lists) list.forEach((id, i) => scores.set(id, (scores.get(id) || 0) + 1 / (k + i + 1)));
  return [...scores].sort((x, y) => y[1] - x[1]).map(([id, score]) => ({ id, score }));
}

Chunks found only by BM25 have no metadata in the vector query results, so they are fetched once more with getByIds.

Optional reranking

The fused results can also go through @cf/baai/bge-reranker-base, a cross-encoder. But it is off by default, and only runs when a request includes rerank=1.

The reason: I compared approaches on a small labeled test set of 12 Chinese and English queries. On this mixed-language data, adding the reranker did not improve the ranking, and it costs one more model call. The code stays so I can compare again as the data grows. There is also a mode=vector|bm25 parameter to use only one retrieval path. All of these are URL parameters, and the cache key is the URL, so each combination is cached separately.

3. Endpoints

/search: semantic search

One game can match several chunks, so results are deduplicated by game before returning game ids:

const seen = new Set<string>();
const matches = (await retrieve(env, q, 20, { mode, useRerank }))
  .filter(({ gameId }) => !seen.has(gameId) && seen.add(gameId))
  .map(({ gameId, score }) => ({ id: gameId, score }));

Results are cached in KV for one hour, keyed by URL.

/completions: RAG Q&A

The five most relevant chunks go into the system prompt, which asks the model to answer from that context and reply in the given language:

const messages = [
  {
    role: 'system',
    content: `You are an application assistant.

Requirements:
1. Respond briefly to user input based on context.
2. Language: respond in (${l}).

Context:
${chunks.map((e) => e.text).join('\n\n---\n\n')}`,
  },
  { role: 'user', content: q },
];
const stream = await env.AI.run('@cf/nvidia/nemotron-3-120b-a12b', { messages, stream: true });

To cache a streamed answer, the stream is split with tee(). One copy goes straight to the user. The other is read to the end inside ctx.waitUntil and written to KV. The user never waits for the cache write, and the next identical question replays the cached SSE:

const [stream1, stream2] = stream.tee();
ctx.waitUntil(new Response(stream1).text().then((text) => env.KV.put(req.url, text, { expirationTtl: 3600 })));
return new Response(stream2, { headers: { 'Content-Type': 'text/event-stream; charset=utf-8' } });

4. Frontend

The frontend is built with Gem (Web Components), and the integration is small.

Semantic search supplements local search. Games are first matched locally by name and pinyin. If there are five or fewer local results, a debounced request goes to /search, and any games not already shown are appended. Every keystroke cancels the previous unfinished request with an AbortController.

Q&A is an <m-sse> element. It waits until the user has stopped typing for 2 seconds, then requests /completions and parses the SSE line by line with a small async generator, rendering Markdown with marked as text arrives:

export async function* eventStream(input: string, init?: RequestInit) {
  const res = await fetch(input, init);
  const reader = res.body!.getReader();
  const decoder = new TextDecoder();
  let buffer = '';
  while (true) {
    const { done, value } = await reader.read();
    if (done) return;
    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split('\n');
    buffer = lines.pop() || '';
    for (const line of lines) {
      if (!line.startsWith('data: ')) continue;
      const data = line.slice(6);
      if (data === '[DONE]') return;
      yield JSON.parse(data);
    }
  }
}

The Worker URL comes from an AI_BASE environment variable and defaults to production. When changing the Worker locally with wrangler dev, pointing AI_BASE at it lets me test through the real frontend before deploying.

Cost

Workers AI includes 10,000 free Neurons per day on both the Free and Paid plans. At Cloudflare's pricing:

Model	Price
bge-m3 (embeddings)	1,075 Neurons per million tokens
bge-reranker-base	283 Neurons per million tokens (not called by default)
LLM for answers	Far more than the other two; this is the main cost

A full sync is only a couple hundred chunks, so embedding cost is negligible. Retrieval results and answers are cached for an hour, so repeated questions don't use the quota again.

Takeaways

• Everything runs on Cloudflare: one Worker, no servers, no separate vector database.
• Built for multilingual content: multilingual embeddings, bilingual labels, and Intl.Segmenter for sentence splitting and tokenizing.
• Cache what can be cached: retrieval results, streamed answers and the BM25 index all live in KV.
• Enhancements behind switches: reranking and single-path retrieval are URL parameters. Defaults are decided by comparing on a test set, rather than turning on every "best practice" from the start.

What's next: citing sources in answers (sending a sources event at the start of the SSE stream so the frontend can link to the referenced games), and adding authentication to /sync and /update.

The code is in the packages/webapp/cf/ directory of mantou132/nesbox. Feedback welcome!