I was working on a couple of projects — one displaying FTP folders and files, another powering a RAG system — where I needed to show lists of data in cards. The lists had to handle large datasets without lagging, crashing, or freezing the browser. I came up with a four-phase plan, tested it in those projects, and then rebuilt it here as a clean reference implementation. This post walks through each phase, what problem it solves, and what trade-offs remain.
Phase 1: Initial List Rendering
Commit: 75c2be7
The simplest approach: fetch every item from the backend in one go and render every card as a DOM node.
The backend was an Express server with an in-memory array — no pagination, no limit. The frontend called GET /api/items, received the entire dataset as JSON, and rendered it with items.map(...) inside a <ul>. CRUD operations were straightforward: Create sent a POST and prepended the new item; Update sent a PATCH and spliced the item in place; Delete sent a DELETE and filtered the item out. After each mutation, the app called loadInitial() — a full refetch and re-render.
Memory: All the JS objects (the entire dataset) sat in the heap, and every card's DOM node was also created and stored in the heap. For 100 items this was fine. For 50,000 it was not.
What went wrong:
- Slow rendering after fetch. Even after the backend responded, the UI froze for seconds while the browser created 50,000 DOM nodes. The fetch was done, but the user saw nothing.
- JSON body limit. When the dataset grew large enough, the response exceeded the JSON body size limit configured in the backend middleware. The request failed outright — no data was rendered at all.
- Heap overflow. Even if the fetch succeeded, the browser's heap could fill up. The GC collector would thrash trying to free memory, and in extreme cases the tab would crash.
Demos:
Phase 2: Virtualization
Commit: 447c57d
The first problem — slow rendering — was caused by creating DOM nodes for every item. Virtualization fixes this by only rendering the cards visible in the viewport (plus a small overscan buffer).
I added react-window, which provides a <List> component. You give it a rowHeight (80px), a rowCount (the total number of items), and a custom Row renderer. The library handles the rest — it mounts only the ~10–15 rows that fit on screen and recycles them as you scroll.
Memory: The JS objects for all items still lived in the heap, but now only a handful of DOM nodes existed at a time. The virtualization library also maintained a height map in the heap, but this was negligible compared to the full DOM tree from Phase 1.
The win: Rendering 50,000 items became instant. The browser only ever had a dozen DOM nodes to worry about.
What remained: The backend still sent all 50,000 items in a single JSON response. If the dataset was large enough, the JSON body limit would still kill the request, and the heap could still overflow from the JS objects alone — no DOM needed.
Demo:
A note on library choice: before reaching for react-window or any virtualization library, check its bundle size, maintenance status, and vulnerabilities. You can also build your own — it's not much code for fixed-height rows — but I'd start with a proven library.
Phase 3: Cursor Pagination + Sliding Window
Commits: 8951667 (pagination) · 14441fc (CRUD fixes)
Phase 2 solved the DOM problem. Phase 3 solves the data problem: instead of fetching everything at once, fetch it in pages.
Cursor vs. Offset Pagination
I used cursor (keyset) pagination, not offset pagination. The cursor is a base64url-encoded string of "createdAt,id" — a compound keyset sorted by createdAt DESC, id ASC. It's a boundary value: "give me everything that sorts after this point."
The key difference: offsets shift when rows are inserted or deleted; keyset cursors don't. If you're on page 3 at offset 100 and someone deletes a row above you, offset 100 now points to a different row. A cursor pointing to a specific (createdAt, id) boundary stays valid no matter what happens elsewhere in the table.
On the backend (now backed by Postgres via TypeORM), findNextPage queries WHERE createdAt < cursor.createdAt OR (createdAt = cursor.createdAt AND id < cursor.id), ordered DESC. findPrevPage inverts the condition and ordering, then reverses the result to restore DESC order.
The Sliding Window
The frontend keeps at most MAX_PAGES = 3 pages (150 items) in memory. When a new page is fetched at the bottom, the top page is evicted. When a page is fetched at the top, the bottom page is evicted. A pageBoundariesRef tracks which slice of the items[] array belongs to which page, along with each page's cursors.
The Scroll Problem
Here's the tricky part: when you prepend items (scrolling up), the browser's scrollTop doesn't change, but the content above has grown — so the viewport jumps to a different item. The user loses their place.
The fix: capture scrollTop before the prepend. Then, in a useEffect that runs after React has committed the new items, restore the scroll position using pure arithmetic:
newScrollTop = oldScrollTop + prependedCount × ITEM_HEIGHT
No DOM queries needed — react-window hasn't rendered the anchored row yet at the new offset, so querySelector would fail. The formula is exact because both states (before capture and after restore) have no loading row, so the math is clean.
CRUD Without Full Reload
Commit 14441fc fixed the CRUD operations to work with the sliding window instead of calling loadInitial() after every mutation:
| Op | Handling |
|---|---|
| Create | If at the true top (!hasPrev): prepend to items[], shift page boundaries. Else: just set hasPrev = true — the new item exists above the window. |
| Update | Splice the updated item in place by id. Sort position is unchanged (createdAt and id are immutable), so all cursors stay valid. |
| Delete | Filter the item out locally. If the window becomes empty and hasNext, auto-trigger loadNext(). |
No full reloads, no scroll jumps.
A note on TanStack Query: I've seen teams adopt it to reduce CRUD boilerplate. It's genuinely useful when you need to invalidate a cache from a different component. But if your CRUD lives in a single component, plain
fetch(oraxios) is leaner — smaller bundle, fewer abstractions, and the code is just as readable.
Memory: Similar to Phase 2, but the dataset coming from the backend is tiny — only one page at a time. The heap holds at most 150 items. It's essentially a sliding window.
Demo:
What remained: An edge case where a single page contains one extremely large item. The JSON response for that page could still exceed the body limit, or the heap could spike. I hardcoded the page limit, but you could make it dynamic — monitor performance metrics and adjust the limit up or down in a useEffect to keep things smooth.
Phase 4: IndexedDB Cache Layer
Commit: 3b804ec
Phase 3 fixed the data problem but introduced a new one: every time a page was evicted and the user scrolled back, the app hit the backend again. Phase 4 adds a client-side cache to avoid those redundant fetches.
Three-Tier Storage
| Tier | Storage | Contents | Size |
|---|---|---|---|
| 1. Heap | React state items[]
|
Currently visible window | 3 pages (150 items) |
| 2. IndexedDB | Browser DB rendering-list-cache
|
Recently evicted pages — 3 above + 3 below | 6 pages (300 items) |
| 3. Backend | Postgres | Source of truth | All items |
When a page is evicted from the heap, it's written to IndexedDB. When the user scrolls back, the app checks IDB first. A cache hit means no backend call at all. A cache miss falls through to the backend, and the result gets cached on future eviction.
Cache Key Scheme
Pages are keyed by the cursor that would be used to fetch them:
-
Top-evicted page (evicted during
loadNext): stored asprev:${cursor}— scrolling back up callsloadPrev(cursor), so we look upprev:${cursor}in IDB. -
Bottom-evicted page (evicted during
loadPrev): stored asnext:${cursor}— scrolling back down callsloadNext(cursor), so we look upnext:${cursor}.
When a page is restored from IDB to the heap, it's deleted from IDB — a page exists in exactly one tier at a time.
CRUD with IDB Awareness
-
Create: If the topmost cached page in IDB is the true top (
hasPrev = false), the new item is prepended to that cached page so the user sees it on scroll-up without a backend call. Otherwise, no sync — the backend serves it on cache miss. - Update / Delete: Only touch the heap. Since a page lives in exactly one tier, if the item is in the heap, it's not in IDB — no IDB sync needed.
Demo:
What remained: The same edge case as Phase 3 — a single extremely large item in a page could still exceed limits. IDB doesn't solve that; it solves the number of backend calls, not the size of a single response.
Summary
| Phase | What it solves | What remains |
|---|---|---|
| 1. Initial render | Baseline — fetch all, render all | Slow render, JSON limit, heap overflow |
| 2. Virtualization | DOM node count — only visible rows rendered | JSON limit, heap overflow (all data still fetched) |
| 3. Cursor pagination | Data volume — only one page fetched at a time; CRUD-safe cursors | Single huge item per page could still hit limits |
| 4. IndexedDB cache | Backend call count — evicted pages cached client-side | Same single-huge-item edge case |
Each phase addresses a specific bottleneck without regressing on the previous one. The result is a list that can handle tens of thousands of items smoothly — visible rows are virtualized, data is paginated with CRUD-safe cursors, the heap is bounded by a sliding window, and recently viewed pages are cached in IndexedDB to avoid redundant fetches.
Top comments (0)