Fast search, filter & sort for large client-side collections in JavaScript

Hey everyone!

I'd like to share an npm package I built for working with large collections of 100K+ items on the client side.

The problem

Sometimes the server sends back a huge array 100K+ items. And you need search, filter, and sort on the client side.

The issue is that standard JS methods like .filter() or .sort() go through the whole array every time. On small data it's fine, but on large arrays the UI starts to feel slow.

What this package does

It has three parts, and you can use each one separately or all together:

• TextSearchEngine — text search. Builds an index once, then searches through it instead of scanning the full array every time.
• FilterEngine — filtering by one or more fields. Indexed fields are looked up directly without going through the whole array.
• SortEngine — sorting. For a single field it uses a pre-built index, for multiple fields it falls back to a regular sort.

The key idea: the index is built once when you pass in the data. After that, every search, filter, or sort call works against that index — not the raw array.

No dependencies. If you only need search — import only search. Filter and sort code won't end up in your bundle.

Usage

All three together with MergeEngines

import { MergeEngines } from "@devisfuture/mega-collection";
import { TextSearchEngine } from "@devisfuture/mega-collection/search";
import { FilterEngine } from "@devisfuture/mega-collection/filter";
import { SortEngine } from "@devisfuture/mega-collection/sort";

interface User {
  id: number;
  name: string;
  city: string;
  age: number;
}

const engine = new MergeEngines({
  imports: [TextSearchEngine, FilterEngine, SortEngine],
  data: users,
  search: { fields: ["name", "city"], minQueryLength: 2 },
  filter: { fields: ["city", "age"], filterByPreviousResult: true },
  sort: { fields: ["age", "name"] },
});

engine.search("john");
engine.filter([{ field: "city", values: ["Miami", "New York"] }]);
engine.sort([{ field: "age", direction: "asc" }]);

Data is passed once — no need to pass it again on every call.

---

Use only what you need

Search:

import { TextSearchEngine } from "@devisfuture/mega-collection/search";

const engine = new TextSearchEngine({
  data: users,
  fields: ["name", "city"],
  minQueryLength: 2,
});

engine.search("john"); // search across all fields
engine.search("name", "john"); // search in a specific field

Filter:

import { FilterEngine } from "@devisfuture/mega-collection/filter";

const engine = new FilterEngine({
  data: users,
  fields: ["city", "age"],
  filterByPreviousResult: true,
});

// second call filters inside the result of the first one
engine.filter([{ field: "city", values: ["New York"] }]);
engine.filter([{ field: "age", values: [22] }]);

Sort:

import { SortEngine } from "@devisfuture/mega-collection/sort";

const engine = new SortEngine({
  data: users,
  fields: ["age", "name", "city"],
});

engine.sort([
  { field: "age", direction: "asc" },
  { field: "name", direction: "desc" },
]);

When does it make sense to use this?

When the data is large enough that you actually feel the lag. For a few hundred items, plain JS is perfectly fine. This starts to matter somewhere around tens of thousands of items and up.

Links

• GitHub: mega-collection
• React demo: demo

I'd love your feedback!
Are there any alternatives I haven’t come across, and are they easier to use?