DEV Community: viswas saripalli

Why my RAG retriever couldn't find a number it was quoted verbatim

viswas saripalli — Sun, 19 Jul 2026 15:59:03 +0000

what is RAG?

Retrieval Augmented Generation. RAG gives an LLM the specific info relevant to a prompt by retrieving it from your own documents first, then answering from those passages.

Embedding

Representing words, phrases, or images as vectors in a high-dimensional space.

what is a vector in high-dimensional space?

The model gives 768 numbers per text. Each number = a coordinate. So every text = a point in 768-D space. Similar meaning → close, unrelated → far. Closeness ≈ meaning.

Chunking

Breaking the material into smaller pieces, so each vector represents one focused passage instead of a whole document averaged out.

Chunking strategies

recursive-character, semantic, page-level, LLM-based, size-based, late chunking
the wrong strategy can open a large recall gap

what is recall? — did retrieval fetch the right chunk to pass to the LLM?

what is recall@k? — was the right chunk inside the top-k cutoff?

Let's start

Took 4 con-call transcripts of a company (Company A) — chose transcripts because they're pure-text PDFs (no tables).
Used a node script to extract contents, with cleanup to remove the cover pages.
Chunked them into size 1000, overlap 200 → chunks span chars 0–1000, 800–1800 …

why overlap?

If a hard cut falls at char 1000, mid-sentence, that sentence gets split: "...revenue for the" ends chunk 0, "quarter was ₹340 crore" starts chunk 1. Now neither chunk contains the complete fact. When you embed them, chunk 0's vector is about a dangling fragment, chunk 1's about another — and a query like "what was quarterly revenue" may match neither well, because the whole idea was severed. Overlap fixes this by making adjacent chunks share their edges.

Now let's choose an embedding model

ollama nomic-embed-text or mxbai-embed-large?
nomic-embed-text — 274 MB, enough for 4 docs.

Let's query — recall@5

Query: how much did revenue grow in Q2

[0.717] CompanyA_Q3_FY26#11 (Q3 FY26) ...total standalone revenue for the...
[0.708] CompanyA_Q2_FY26#15 (Q2 FY26) ...hand over the call to our CFO...
[0.703] CompanyA_Q4_FY26#32 (Q4 FY26) ...royalty income added to revenue... EBITDA margin is stable...
[0.703] CompanyA_Q2_FY26#23 (Q2 FY26) ...targeting 10% to 15% revenue growth for FY '26...
[0.699] CompanyA_Q4_FY26#10 (Q4 FY26) ...pleasure to discuss our performance as we close...

what are 0.717, 0.708?

Cosine similarity scores — how close the query embedding vector is to each chunk's embedding vector.

1.0 = identical direction, 0.0 = unrelated, -1.0 = opposite direction (won't see this with real text)
0.717 means the chunk is closer to the query asked
Don't read the absolute value as "72% relevant" — cosine isn't a percentage, and what's "high" depends on the model. With nomic on similar prose, ~0.7 is ordinary. What's diagnostic is the gap between hits, and here it's flat — a genuine finding about why retrieval on near-identical earnings calls is hard.

Note: asked Q2, top hit is Q3. The quarter token got ignored.

what if I ask an irrelevant question?

Query: who will win fifa world cup

[0.563] CompanyA_Q2_FY26#2  ...brief overview of the past quarter and first half...
[0.562] CompanyA_Q1_FY26#2  ...open the floor for Q&A...
[0.558] CompanyA_Q2_FY26#43 ...how much can we expect to win? probable success ratio?...
[0.557] CompanyA_Q4_FY26#16 ...growing the business gradually... if we win this award...

Data retrieved but irrelevant. cosine ALWAYS returns k results — no "no match". Scores are lower (~0.56 vs ~0.71), so the intuitive fix: reject anything below 0.6 and say "I don't know". But that fix is wrong — see failure 1.

From now, the actual problem starts

failure 1 — entity-blindness (the threshold can't work)

Query: What was Company B's revenue last quarter?

[0.758] CompanyA_Q4_FY26#10 (Q4 FY26) ...pleasure to discuss our performance...

Company B is NOT in the corpus (different company). Yet it scored 0.758 — HIGHER than every real in-domain query (max 0.717). The embedding locked onto "revenue" and ignored "Company B".

→ So the 0.6 threshold idea is dead: any cutoff that admits real questions (≤0.717) also admits Company B (0.758). Pizza (0.55, fully unrelated) is rejected fine, but wrong-entity-right-domain sails through. Dense embeddings capture topic, not entity.

failure 2 — quarter-blindness (fixed by metadata)

Query: PAT for Q3 was 75 crores PAT margin 7.7

[0.758] CompanyA_Q4_FY26#11 (Q4 FY26) ...EBITDA for Q4 FY '26 stood at INR183 crores... margin of 14.1%...
[0.736] CompanyA_Q1_FY26#45 (Q1 FY26) ...order book of around Rs. 10,300 crores...
[0.709] CompanyA_Q2_FY26#24 (Q2 FY26) ...margins around 11.5%. Second half should be similar...

Quoted Q3 verbatim, top 3 are Q4/Q1/Q2 — the target quarter isn't even there. The quarter token is nearly ignored.

fix: metadata filtering — parse the quarter from the query and hard-filter chunks to that quarter before ranking.

after metadata filtering (Q3 only):
[0.631] CompanyA_Q3_FY26#34 ...early completion bonus...
[0.631] CompanyA_Q3_FY26#31 ...INR605 crores invested in HAM...
[0.607] CompanyA_Q3_FY26#33 ...sale of those projects...
[0.604] CompanyA_Q3_FY26#32 ...outstanding cash as of December...

All Q3 now. Cross-quarter confusion solved — the one clean win.

failure 3 — token-blindness (chunking can't fix it)

Even after isolating PAT into its own sentence-chunk (literally "PAT for Q3 was 75 crores"), the query "what was PAT" STILL doesn't retrieve it. Verified via grep the figures were never split. So it's not chunking — the dense model won't connect the token "PAT" to a chunk containing it. Needs hybrid dense+sparse.

The fixes each finding points to:

Entity-blindness → metadata (company field) or sparse; no cosine threshold works.
Quarter-blindness → metadata filtering (scope to Q3).
Token-blindness → hybrid dense+sparse (match the literal token dense smears away).

Cosine similarity is about retrieval, not answer accuracy

A chunk can score 0.72 and still not contain the number you need. recall@k is stricter (is the known-correct chunk in top-k) but still a retrieval metric — not answer correctness.

Moved towards sentence chunking

Questions used for the recall@k eval:

{ question: "What was total standalone revenue in Q3 FY26?", goldIds: ["CompanyA_Q3_FY26#38"] },
{ question: "What was standalone EBITDA margin in Q3 FY26?", goldIds: ["CompanyA_Q3_FY26#38", "CompanyA_Q3_FY26#39"] },
{ question: "What was PAT for Q3 FY26?", goldIds: ["CompanyA_Q3_FY26#39"] },
{ question: "What revenue growth guidance did management give for FY26?", goldIds: ["CompanyA_Q2_FY26#81", "CompanyA_Q2_FY26#82"] },

// out-of-domain: goldIds MUST stay empty (any hit = false positive)
{ question: "Who will win the FIFA World Cup?", goldIds: [] },       // ~0.56, rejected by 0.6
{ question: "What was Company B's revenue last quarter?", goldIds: [] }, // 0.758, HIGHER than in-domain
{ question: "Best pizza place in Mumbai?", goldIds: [] },            // ~0.55, no overlap

Strategy	recall@1	recall@5	PAT	Notes
Fixed 1000/200	0.75	0.75	miss	baseline
Sentence (3/1)	0.50	0.75	miss	theme fragmentation hurt guidance

Sentence chunking made recall@1 WORSE (0.75→0.50) and still didn't fix PAT — guidance fragmented across ~12 chunks. Strongly suggests the PAT failure is embedding-level, not chunk size. Next = hybrid.

Moving back to fixed chunking(recall@k got worse, not better — 0.75→0.50)

why did isolating PAT into a sentence chunk still miss?

Chunking and retrieval are different stages. Sentence-chunking only changed what's IN the chunk — but PAT didn't fail because of chunk contents, it failed because nomic places the query vector ("what was PAT") and the chunk vector ("PAT was ₹75 crores") FAR APART in embedding space. Chunk boundaries don't move those points closer. So a cleaner chunk can't fix a query↔chunk distance problem — the failure lives in the embedding, not the chunking. That's what makes it a clean negative result: it ruled out chunking as the cause.

So how does the search actually happen?

how to fix the query → chunk embedding distance problem?

The PAT chunk exists but the query "what was PAT" lands far from it in vector space. Two families of fix:

A. move the points closer

better embedding model (bge-m3 etc) — might place them closer, not guaranteed
query expansion — rewrite the query to look like the target text before embedding
HyDE — embed a fake answer instead of the question (answers sit closer to answers)

B. stop relying on distance

hybrid (dense + BM25 sparse) — match the literal token "PAT", ignores geometry
metadata extraction — pull figures into fields, query structure not prose
reranking — retrieve wide (top-30), re-score query+chunk together with a cross-encoder

For my case: HyDE or hybrid most likely to fix PAT. Both buildable, no training data.

where does PAT chunk rank in a top-30 dense retrieval? It ranks 26/30 (0.539) — in the net but near-bottom, below the 0.6 threshold. Too deep for reranking to be reliable → hybrid (BM25) or HyDE is the fix, since both lift it in the initial ranking rather than re-ordering a barely-retrieved net.

What is HyDE?

HyDE = Hypothetical Document Embeddings.

The problem: a question and an answer are shaped differently. "What was PAT for Q3?" is a short interrogative. "PAT for Q3 was ₹75 crores with a margin of 7.7%" is a declarative statement full of specifics. The embedding model reads these as different kinds of text and places them far apart — even though one answers the other.

HyDE's trick: don't search with the question. Instead:

Ask a cheap LLM to write a fake, made-up answer to the question — e.g. "Company A's PAT for Q3 FY26 was approximately 70 crores, with a margin of around 8%." The numbers don't have to be correct.
Embed that fake answer (not the question).
Search with the fake answer's vector.

The fake answer is shaped like an answer, so it embeds close to the real answer chunk. The hallucinated numbers are thrown away — the real chunk it retrieves is what you use. Tradeoff: one extra LLM call per query, versus hybrid/BM25 which is pure math and instant.

Implementing hybrid search

The problem: I search my documents by meaning. Ask "what was PAT?" and it should find the sentence with the PAT number. But it didn't — the computer thought a question and its answer looked too "different" to match, even though the answer was right there.

Why: meaning-search is fuzzy. Great at "these are about the same topic," bad at "these both contain the exact word PAT." The number sentence got lost.

The fix — two searchers instead of one:

Searcher A (what I had): matches by meaning. Good at topics, bad at exact words.
Searcher B (BM25): matches by exact words. Dumb but reliable — just checks "does this chunk literally contain 'PAT'?"

Run both, combine the answers. That combo is hybrid search.

The twist: combining them equally didn't work — the meaning-searcher was so confidently wrong about the PAT chunk that it dragged the right answer down, even though the word-searcher had it at #1. Fix: tell the combiner "trust the word-searcher more." That did it.

Setup	recall@1	recall@3	recall@5	PAT
meaning-only (dense)	0.75	0.75	0.75	❌ miss
hybrid (meaning + words)	0.75	1.00	1.00	✅ found

PAT went from "not found" to "found." recall@3 jumped 0.75 → 1.00 — all four questions now find their answer.

After adding the generation layer (llama3:8b)

npx tsx answer.ts "What was PAT for Q3 FY26?"

Retrieved 5 chunks:
[1] CompanyA_Q3_FY26#11
[2] CompanyA_Q3_FY26#16
[3] CompanyA_Q3_FY26#15
[4] CompanyA_Q3_FY26#0
[5] CompanyA_Q3_FY26#12

Q: What was PAT for Q3 FY26?
A: The answer can be found in [1]:
   "While our standalone PAT for Q3 FY 26 was INR75 crores with a PAT margin of 7.7%."
   Therefore, the answer is: INR75 crores

npx tsx answer.ts "What was PAT for Q3 FY26 of Company B?"

Retrieved 5 chunks:
[1] CompanyA_Q3_FY26#11
[2] CompanyA_Q3_FY26#16
[3] CompanyA_Q3_FY26#12
[4] CompanyA_Q3_FY26#34
[5] CompanyA_Q3_FY26#15

Q: What was PAT for Q3 FY26 of Company B?
A: Not found in the provided documents. The context only mentions PAT figures
   for Company A Limited, but not for "Company B".

"not found" is the LLM prompt, not a RAG filter

The refusal came from the grounding instruction, not from retrieval (retrieval still returned Company A chunks). The prompt:

You are a financial analyst assistant. Answer the question using ONLY the context below.
If the answer is not in the context, say "not found in the provided documents."
Do not use outside knowledge. Be concise and cite the figure.

two-quarter problem (known limitation)

"Q1 FY27 outlook given in Q4 FY26" → parseQuarterFY grabs the first quarter (Q1 FY27) → filters to an empty set (no FY27 chunks) → 0 candidates → "not found," even though the answer is in Q4 FY26. The metadata filter that fixed quarter-blindness breaks on questions referencing two quarters. Real fix needs query-intent parsing (which quarter is the filter vs the target) — out of scope for now.

AI Fluency Series · Project 1: Streaming Chat Or: how I spent a day proving a bug I was causing myself.

viswas saripalli — Sun, 12 Jul 2026 13:35:06 +0000

Disclaimer: a dumb little start to leveling up my AI fluency. Nothing fancy, no grand architecture, no hot takes — just a journal of me trying to actually understand things instead of pretending to.

Quick filter: if you're on React 19, have actually enabled the compiler (it's an opt-in plugin, not something the version hands you for free), and you write pure components — close the tab, you already know the ending. If you're on React 19 and assumed that meant the compiler was on: stay. That assumption is a trap, and I fell in it.

Why chat is Project 1

Streaming tokens into a growing list that re-renders on every one is the substrate under every AI feature I'm about to build — RAG answers stream, agents stream, all of it. If I can't make token-by-token rendering behave, nothing downstream will.

So the rule I gave myself: build it by hand before reaching for the library. No useChat until I've felt the problem it solves. I did it in Next.js — partly because that's where I'm headed, partly as an excuse to learn it — but none of this is Next-specific; any framework does the same thing. And I mocked the stream instead of calling a real model, because the subject is how the UI handles a stream, not the model. A real LLM would just be a slower way to make the same tokens.

The storm

The setup was deliberately dumb: append each token into one blob of state, map the whole array to the screen.

setChunks(prev => [...prev, chunk]);              // new array, every token
{chunks.map((c, i) => <div key={i}>{c}</div>)}    // whole list, every token

I wired it up, fired the request, watched the words stream in. Worked on the first try — which felt great for about ten seconds, until the first real problem surfaced: it re-rendered the entire message list on every single token. That's the O(N²) trap in the flesh. Token 200 re-processes 200 things, token 400 re-processes 400. It's not slow because it's badly written — it's slow because it recomputes the settled past on every update.

I also added an input box to test typing mid-stream and found a second bug I'd built for free: the input's state lived in the same component as the list, so every keystroke re-rendered all 400 messages. Two unrelated things welded together because they shared a component. Fix: move the input's state into its own component. Suddenly typing re-renders one input, not the whole conversation — the single most useful React habit, and it has nothing to do with streaming.

Then measuring got weird

I wanted numbers, not vibes, so I dropped a render counter and a console.log into the message component. Chunk 0 was rendering on every token — the full storm, right there in the console. The Profiler said the same thing in pictures: record one response and watch the whole .map re-run, every chunk lighting up in the flamegraph on every word.

Except the numbers wouldn't sit still. Dev said 872 renders; prod said 426 — React's Strict Mode double-invokes in dev, so half my "data" was a lie the dev server was telling me. (Lesson one: never trust a perf number from npm run dev.)

There was also a version goose-chase that ended in pure comedy:

TypeError: messages.some is not a function
🧠 "typo?" → 🧠🧠 "@ai-sdk/react is behind ai, upgrade React!" → 🧠🌌 "the whole SDK protocol changed, I must rewrite everything"
🤡 the actual fix: I forgot to type await

But the real head-scratcher: I knew the compiler was supposed to memoize my components — I'd enabled it, and DevTools showed the little ✨ badge. So why was chunk 0 still re-rendering every token?

The twist

Here's the thing that made the whole day worth it:

My measurement was causing the bug.

The React Compiler only memoizes components it can prove are pure — same inputs, same output, no side effects during render. And I'd shoved a console.log and a ref mutation straight into the render body to measure the renders. Those are side effects. So the compiler took one look, decided it couldn't safely optimize the component, and silently bailed — no warning, no error, it just quietly stopped memoizing.

Which means the O(N²) storm I was so carefully measuring was, in part, manufactured by the act of measuring it. The probe changed the experiment. I was shining a flashlight into a dark room to see how dark it was, then writing down "huh, pretty bright in here."

// impure → compiler refuses to memoize → every Chunk re-renders
const Chunk = ({ text }) => {
  const r = useRef(0); r.current++;      // ⛔ mutation in render
  console.log('render', r.current);      // ⛔ side effect in render
  return <div>{text}</div>;
};

// pure → compiler memoizes it; measure from an effect instead
const Chunk = ({ id, text }) => {
  useEffect(() => console.log('committed', id), [id, text]);
  return <div>{text}</div>;
};

Pull the side effects out, gate the logging into a useEffect, and chunk 0 went quiet. The ✨ badges meant what they said. The storm collapsed to O(N) — and I hadn't written a single React.memo. The compiler did it, the moment I stopped poisoning the well.

(Full honesty, since this is a journal and not a press release: the airtight proof is a clean Profiler read on a late commit with zero instrumentation. I eyeballed the sparkles more than I formally captured that. I'm ~95% there; the last 5% is a thirty-second measurement I owe myself.)

Then I used the library

Only after all that did I swap in useChat — and building it by hand first meant I could see exactly what it does. The fetch, the reader, the accumulation, the state updates: the entire machine I'd hand-cranked, gone, replaced by three destructured values.

But the surprise, which probably shouldn't have been one: useChat doesn't fix the render cost. It owns when you render — every token, via its own state updates — and has zero opinion about how expensive each render is. Drop a console.log back into a message component and the O(N²) is right back, useChat and all. The abstraction changed the plumbing. It did not change the physics.

What carried over

The React-specific trivia is fun, but the two keepers are bigger than React:

Re-processing the settled past on every update is the cost that bites. Streaming chat, an append-only log, a game loop — same shape, same fix: isolate what changed from what didn't. Memoization is just React's dialect for it.
Your instrument can change what you measure. I hit it three times in one day — dev double-renders, keystrokes polluting my counts, a console.log disabling the compiler — and I anchored on an exciting theory (version hell!) while the stack trace pointed at the boring truth the whole time. "Did my measurement perturb the thing?" is now a reflex, and it's a science habit, not a frontend one.

The 2026 punchline, and the reason for that snarky opener: you mostly don't do memoization anymore. Keep your components pure and let the compiler do it. The skill moved from "know where to sprinkle memo" to "write pure components — and know how to tell when the compiler quietly gave up on you."

Next up in the series: RAG. Where, I'm told, the actual AI begins.

— filed from a laptop that is now, finally, rendering only what changed.

How We Handled 100K Items in a React List

viswas saripalli — Sun, 29 Mar 2026 16:40:28 +0000

Imagine you're building a bulk configuration UI. Users connect to a data source — a Postgres database, a SaaS API, a data warehouse — and need to select which items to include: tables, fields, events, records. Dozens is easy. A few hundred is manageable. But what happens when the list hits 100,000?

That's the situation I ran into. This post is about the approach we landed on, why it works, and how to apply it anywhere you have a large server-side list with sparse user edits.

The naive approach and why it breaks

The instinctive approach is to fetch the list, copy it into local state, and update entries in place as the user interacts:

// Naive approach — don't do this at scale
const [items, setItems] = useState(() =>
  apiResponse.map(item => ({ ...item }))  // copy everything
);

function toggleItem(id: string) {
  setItems(prev => prev.map(item =>       // O(n) on every toggle
    item.id === id
      ? { ...item, isSelected: !item.isSelected }
      : item
  ));
}

This works fine at small scale. At 100K items, you have three compounding problems:

Startup cost. Copying 100K objects into state on mount takes meaningful time and memory. If some items start excluded, you often need an initialization loop to pre-populate a separate "excluded" set — another O(n) pass before the user sees anything.

Interaction cost. Every checkbox toggle triggers a state update that React has to reconcile. Even with virtualization, the state update itself touches the full array.

Downstream cost. Any feature that needs to know "what changed" — a review panel, a diff summary, a submission payload — has to scan the full list to find the delta. O(n) on every toggle.

The root problem is the implicit assumption that "UI state = copy of server data." Once you make that assumption, everything downstream inherits the O(n) cost.

A better approach: store only what changed

The better mental model: the server response is immutable ground truth. Your UI state only records the delta — what the user changed from that baseline.

Server response (immutable base)
  items[]: { id, name, isSelected, ... }
        │
        ▼
Delta state (user changes only)
  selectedIds   = Set<id>   ← user explicitly selected
  deselectedIds = Set<id>   ← user explicitly deselected
  allSelected   = boolean    ← "Select All" clicked
  noneSelected  = boolean    ← "Deselect All" clicked
        │
        ▼
Resolution layer (derived on demand)
  isSelected(id) → O(1)

The delta state starts empty. selectedIds and deselectedIds only grow when the user actually interacts. An item the user never touches has no entry in either set.

The resolution layer answers "is this item selected?" by checking the delta first, then falling back to the server base data:

function isSelected(id: string): boolean {
  // Bulk operations take priority
  if (delta.noneSelected) return delta.selectedIds.has(id);
  if (delta.allSelected)  return !delta.deselectedIds.has(id);

  // Individual overrides
  if (delta.deselectedIds.has(id)) return false;
  if (delta.selectedIds.has(id))   return true;

  // No user change — fall back to server data directly
  return itemsById.get(id)?.isSelected ?? false;
}

That last line is what makes everything else possible. Because the resolution layer can consult the original server data, the delta state never needs to be pre-populated. Start with empty sets, and checkboxes render correctly on first paint — no initialization loop.

No hydration: eliminating the startup cost

A common pattern when loading a list with some pre-excluded items is to hydrate the exclusion set on mount:

// Common pattern — unnecessary with this approach
useEffect(() => {
  const excluded = new Set(
    apiItems
      .filter(item => !item.isSelected)
      .map(item => item.id)
  );
  setExcludedIds(excluded);  // O(n) before user sees anything
}, []);

With delta state, this loop is unnecessary. The sets start empty. isSelected() falls through to item.isSelected from the server response for any item the user hasn't touched. The UI is correct from the very first render.

At 100K items, eliminating this loop measurably reduces time-to-interactive.

Toggling: O(1) state updates

Checkbox toggles now only update the delta sets — a Set add or delete:

function toggleItem(id: string) {
  setDelta(prev => {
    const next = { ...prev,
      selectedIds:   new Set(prev.selectedIds),
      deselectedIds: new Set(prev.deselectedIds),
    };

    if (next.deselectedIds.has(id)) {
      next.deselectedIds.delete(id);  // revert to server default
    } else if (next.selectedIds.has(id)) {
      next.selectedIds.delete(id);    // revert to server default
    } else if (isSelected(id)) {
      next.deselectedIds.add(id);     // was selected → deselect
    } else {
      next.selectedIds.add(id);       // was deselected → select
    }

    return next;
  });
}

The state object is tiny regardless of total item count. React reconciliation only touches components that consume the delta — not components that render individual items, because those only call isSelected(id) and get a stable result.

O(k) review and diff — for free

Here's where this approach pays compound dividends. Any feature that needs to know "what changed" just iterates the delta sets — not the full item list.

function useReviewChanges() {
  return useMemo(() => {
    const added = [], removed = [];

    if (delta.allSelected) {
      // Newly added = server had isSelected:false, not in deselectedIds
      for (const item of allItems) {
        if (!item.isSelected && !delta.deselectedIds.has(item.id))
          added.push(item);
      }
    } else {
      // Normal case — iterate only the delta sets
      for (const id of delta.selectedIds) {
        const item = itemsById.get(id);
        if (item && !item.isSelected) added.push(item);
      }
      for (const id of delta.deselectedIds) {
        const item = itemsById.get(id);
        if (item?.isSelected) removed.push(item);
      }
    }

    return { added, removed };
  }, [delta]);
}

Where k is the size of the user's change sets — typically single digits in a real session. A live "review changes" panel powered by this hook updates instantly on every toggle, regardless of total item count.

The same principle applies to the submission payload — only send what changed:

function generatePayload() {
  const changes = [];

  for (const id of delta.selectedIds) {
    if (!itemsById.get(id)?.isSelected)
      changes.push({ id, isSelected: true });
  }
  for (const id of delta.deselectedIds) {
    if (itemsById.get(id)?.isSelected)
      changes.push({ id, isSelected: false });
  }

  return changes;  // 3 entries, not 100K
}

The key insight: By designing state as "what changed from the server" rather than "copy of server data", every downstream operation — review panel, submission payload, undo history — inherits O(k) complexity automatically. You don't optimize each feature separately; the data structure does it for you.

Moving index-building off the main thread

Even with lean delta state, you still need to build a lookup index from the raw API response: a Map<id, item> for O(1) access, relationship maps for parent-child hierarchies, a flat list for rendering. At 100K items, doing this synchronously on the main thread causes a noticeable freeze.

Move it to a Web Worker. But be careful about what you send — postMessage uses structured clone, and serializing full item objects is expensive:

Items	Full objects (~500 bytes each)	Minimal projection (~80 bytes)
100K	~50 MB	~8 MB
500K	~250 MB	~40 MB
1M	~500 MB	~80 MB

Send the worker only the fields it actually needs for indexing — typically just IDs and relationship fields. The worker builds the structural index and sends back serialized maps of IDs. The main thread reconstructs full lookup structures from its own data reference, avoiding a large clone in the return direction.

// Only send what the worker needs for indexing
worker.postMessage({
  type: 'BUILD_INDEX',
  items: items.map(({ id, parentId, groupKey }) =>
    ({ id, parentId, groupKey })   // ~80 bytes vs ~500 bytes
  ),
});

// Worker returns ID maps only — main thread rebuilds full objects
worker.onmessage = ({ data }) => {
  if (data.type === 'INDEX_READY') {
    setIndex(deserializeIndex(data.result, items));
  }
};

The same worker can handle search: send a query string, receive back an array of matching IDs. The main thread resolves full items from its local index. Search never blocks the render cycle until results are ready.

One gotcha: be careful with useDeferredValue

If your list has cascade behavior — selecting a parent automatically selects its children — you may be tempted to wrap the resolution layer in useDeferredValue to avoid re-renders on rapid clicks. We tried this. It introduced a subtle correctness bug.

Cascade effects that work by falling through to isSelected() (rather than explicitly adding children to the delta sets) depend on the resolution layer being current. With a deferred (stale) resolution layer, the review panel called isSelected() on an old snapshot and reported incorrect counts for cascaded children until the deferred update caught up.

The fix was to remove useDeferredValue entirely. The resolution layer is just closures over small Sets — recomputing it is cheap. Profile before deferring. Stale derived state has correctness implications that are easy to miss.

Where else this applies

This approach is useful anywhere you have a large server-side collection with sparse user edits:

Permission managers — a list of users or roles where an admin toggles access. The server has the current state; the delta tracks only the changes before save.
Spreadsheet-style editors — a large grid where users edit individual cells. Store only the edited cells as a delta on the original data.
Bulk tag/label editors — applying or removing labels from a large item list. The delta records only touched items.
Feature flag consoles — enabling or disabling flags across a large service registry. Flags start at server defaults; the delta captures the diff before deployment.

In each case, the same invariant holds: the server data is ground truth, the delta captures user intent, and the resolution layer derives the current state on demand.

The takeaways

Server data is immutable ground truth. Never copy it into mutable local state. Store only the delta — what the user changed — as lightweight Sets or Maps.

Never hydrate if you can fall back. If your resolution layer can consult the original server data, the change sets don't need to be pre-populated. Start empty and let the fallback handle the initial render.

Design state around access patterns. Use Sets for O(1) membership checks. Use Maps for O(1) ID lookup. The right data structure makes every downstream operation trivially fast without explicit optimization.

Send workers only what they need. Structured clone is not free. Profile your postMessage payload — it's easy to accidentally serialize far more than necessary. A minimal projection costs seconds of engineering and pays off at every scale.

Be cautious with deferred values near cascade logic. useDeferredValue and useTransition create a window where derived state is stale. If other logic depends on that state being current, you'll get subtle bugs that are hard to reproduce.

This approach scales cleanly from hundreds to millions of items. The same core — immutable base, lightweight delta, on-demand resolution — works whether you're building a permission manager, a bulk config form, or a spreadsheet editor.