How Meilisearch works

Here is the complete end-to-end journey of searching for "Evengers" (notice the typo!) in a movie database.

This flow ties together everything we've discussed—Rust application logic, LMDB offsets, Linux mmap, Page Faults, and CPU memory reads.

---

The End-to-End Execution Flow

1. 1. API Ingestion & Typo Tolerance: User Space (Rust App). The HTTP request GET /indexes/movies/search?q=Evengers hits the Meilisearch Rust web server.

• Tokenization: Meilisearch cleans and extracts the token "evengers".
• FST Lookup: Meilisearch queries a memory structure called a Finite State Transducer (FST) to handle typos. It calculates a Levenshtein distance of 1 and maps the typo "evengers" $\rightarrow$ real indexed term "avengers".

1. 2. Calculating the Virtual Address: Meilisearch + LMDB Logic. To find which movies contain "avengers", Meilisearch needs to read the Inverted Index entry for that word.

LMDB knows "avengers" sits at Offset `0x8400` inside the database file. Meilisearch performs pointer math:

$$\text{Virtual Address} = \text{Base Address } (0\text{x}10000000) + \text{Offset } (0\text{x}8400) = 0\text{x}10008400$$

Meilisearch asks the CPU: "Read memory at 0x10008400."

1. 3. Kernel Memory Check & Page Fault: Linux Kernel + Hardware. The CPU intercepting 0x10008400 checks the Page Table in physical RAM:

• Cold Search (Present Bit = 0): The CPU raises a Page Fault. The Linux OS pauses the thread, fetches the 4 KB page containing the "avengers" index node off the SSD, loads it into Physical RAM, and flips Present Bit = 1.
• Warm Search (Present Bit = 1): The data is already in Physical RAM. The CPU reads it instantly in nanoseconds.

1. 4. Bitset Intersection (Roaring Bitmaps): CPU Execution. At address 0x10008400, Meilisearch reads a Roaring Bitmap—a compressed set of document IDs containing the word "avengers" (e.g., [Doc ID 12, Doc ID 45, Doc ID 890]).

If you also searched for another word (e.g., "marvel"), Meilisearch would fetch both bitmaps and execute a CPU bitwise AND instruction to instantly find only the movies containing both terms.

1. 5. Ranking & Document Retrieval: LMDB + Memory Read. Meilisearch runs its ranking rules (Typo score, Word matching, Proximity) on the matching IDs. Doc ID 45 (The Avengers, 2012) wins top spot.

Now it needs the actual movie details (Title, Plot, Poster URL):

• Calculates: Virtual Address = Base Address + Offset_for_Doc_45
• Reads the raw JSON bytes directly from memory (via RAM or a final Page Fault from SSD).

1. 6. HTTP Response: Network Output. Meilisearch serializes the JSON payload into an HTTP 200 OK response and hands it off to the network socket back to the client—all completed in under 15 milliseconds.

---

The Big Picture Takeaway

Notice how Meilisearch never wrote a single line of code to handle reading from the SSD, managing buffer caches, or allocating disk blocks.

It simply performed pointer math (Base + Offset), asked for a Virtual Address, and let the Linux Kernel handle fetching bytes off the SSD whenever a Page Fault occurred.