DEV Community: Sergey Nikolaev

14 faster embeddings: how we rebuilt the ONNX path in Manticore

Sergey Nikolaev — Thu, 25 Jun 2026 11:50:04 +0000

When we shipped Auto Embeddings — the feature that turns any text column into a vector automatically, with no separate model service to run — the most common piece of feedback was about speed. The previous path went through SentenceTransformers on top of Candle, Hugging Face's pure-Rust ML inference runtime, and it left a lot of CPU on the floor: most workloads sat in the low-double-digits of docs/sec no matter how we fed them, and concurrent calls serialised on a single model session.

So we spent a few weeks rebuilding how Manticore runs ONNX models. The new ONNX Runtime backend shipped in Manticore Search 27.1.5. ONNX (Open Neural Network Exchange) is the portable model format that most of the popular open-source embedding models — MiniLM, BGE, E5, and friends — already publish. The result is a backend that's ~14× faster on average than the previous SentenceTransformers/Candle path on the same hardware (average cheap 16 cores / 32 threads server), same model, same weights, averaged over the full threads × batch workload grid — and that advantage holds whether you run 1 client thread or 32. The old path stayed in the 5–11 docs/sec range across the entire grid; the new one lives in the 70–230 docs/sec band.

This post is the engineering log: what we tried, what surprised us, what we threw away, and what the final design looks like.

TL;DR

~14× faster on average than the previous SentenceTransformers/Candle path, averaged across the full threads × batch workload grid (1 / 2 / 4 / 8 / 16 / 32 threads × batch sizes 1…128) on the same box (16 cores / 32 threads), same model, same weights.
Released in Manticore Search 27.1.5, the new ONNX path is now the default fast path for any HuggingFace model that ships an .onnx file.
On all-MiniLM-L12-v2, the old Candle path sat at 5–11 docs/sec across every configuration we tried. The new ONNX path lands in the 70–230 docs/sec range — the same ~14× margin holds whether you run 1 client thread or 32.
Single-insert latency on our test box: ~14 ms with a single client, ~56 ms under 8-way concurrent load — both well below the 200+ ms Candle was hitting.
Want maximum bulk ingest throughput? Use a high batch size (32–128) on a single client thread. The new backend parallelises inside the call, so client-side fan-out just piles coordination overhead on top — peak on our box was 233 docs/sec at 1 thread + batch=64.
The two changes that mattered most: turning intra_op_spinning off, and giving up on batching documents inside the worker.
No user-facing API changes. A table that already points at an ONNX-capable MODEL_NAME picks up the new path automatically. Switching an existing table to a different model isn't a one-liner — Manticore doesn't allow altering MODEL_NAME on a FLOAT_VECTOR field in place — but you don't have to recreate the whole table either: you can add a new column with the new model alongside, rebuild its embeddings, and drop the old one.

Why this matters

With auto-embeddings, the database itself runs the model on every INSERT. That means embedding speed is INSERT speed — your ingest throughput is whatever the embedding step can sustain.

The old SentenceTransformers/Candle path left performance on the table. Concurrency hit lock contention, batched calls plateaued because of padding overhead, and between calls the runtime parked threads in ways that prevented the next call from picking up where the previous one left off. The headline symptom was simple: top would show the box well under full load no matter what you threw at it. The whole sweep — single-row INSERTs, 128-row bulk INSERTs, one client thread, thirty-two client threads — sat at 5–11 docs/sec, because nothing about how you fed it could buy you more CPU.

The new ONNX path raises the floor by an order of magnitude and gives users meaningful performance tuning options. A single-thread, single-row INSERT now lands 72 docs/sec — already ~7× the old Candle ceiling. Add concurrency or batch size and it climbs into the 130–230 docs/sec range, with the top of the grid at 233 docs/sec on a single client thread at --batch-size=64. Averaged across the whole threads × batch matrix, the new path is ~14× the old one.

Why ONNX, and not Candle

Manticore's embeddings library has supported a few backends for a while. The Candle path is great for correctness and easy to ship. But for production inference of small encoder models like the MiniLM and BGE family, ONNX Runtime is hard to beat:

ONNX Runtime (or ORT — Microsoft's official, hand-tuned C++ inference engine for ONNX models) does graph fusion, constant folding, kernel autotuning.
Most of the popular embedding models on HuggingFace already publish a pre-fused model.onnx in their onnx/ directory. The on-disk file is already in the shape ORT wants.

On the same all-MiniLM-L12-v2 weights, on CPU, the ONNX path is a noticeable step up over the Candle path. Same quality, much less per-document work.

The ORT session is created with a small set of opinions:

let session = ort::session::Session::builder()?
    .with_optimization_level(GraphOptimizationLevel::Level3)?
    .with_intra_threads(0)?            // let ORT pick (= all cores)
    .with_intra_op_spinning(false)?    // do NOT busy-wait between calls
    .with_flush_to_zero()?             // kill denormals on attention softmax
    .with_approximate_gelu()?          // ~10% faster activation, no quality loss
    .commit_from_file(&onnx_path)?;

Most of these are uncontroversial, "of course you turn that on" knobs. One is not: intra_op_spinning(false). We'll come back to it — it's the single biggest win in the whole branch, and it's not really an ORT setting so much as a load-shape decision.

The concurrency model — the part most readers will find new

If you give a Rust developer "make ONNX go fast" with no other constraints, they reach for one of two patterns. We tried both. They are both wrong for this workload.

Pattern 1: a single shared Session behind a Mutex (a Mutex is a lock that lets only one thread touch the session at a time). Easy to reason about, easy to get right. Throughput collapses under concurrency because every caller serialises on the lock. Fine for a CLI tool, awful for a database serving many concurrent INSERTs.

Pattern 2: a session pool, one Session per CPU. No more lock contention, but cold-start time multiplies, RAM use multiplies, and small inputs pay a dispatch cost just to land on a session. We had a working version of this in a development branch and it never quite delivered.

The thing that unlocked the design is something most Rust ONNX wrappers get wrong: on Linux and macOS, ORT's C Run() API is thread-safe. You can share one Session across many concurrent callers without any locking. The C++ side already serialises what needs serialising; the Rust API just hides it behind borrow-checker rules that do not match what the underlying library actually allows.

So we wrap the session in a small platform-aware type:

#[cfg(not(target_os = "windows"))]
struct SessionWrapper {
    inner: std::cell::UnsafeCell<ort::session::Session>,
}

#[cfg(not(target_os = "windows"))]
unsafe impl Sync for SessionWrapper {}
#[cfg(not(target_os = "windows"))]
unsafe impl Send for SessionWrapper {}

impl SessionWrapper {
    fn with_session<R>(&self, f: impl FnOnce(&mut Session) -> R) -> R {
        f(unsafe { &mut *self.inner.get() })
    }
}

Yes, this is unsafe. We're taking the borrow checker out of the loop because the underlying library is documented to be safe under the access pattern we're using. It's a deliberate unsafe with a one-line justification, not a foot-gun.

On Windows, ORT's threading model has known issues, so we serialise Run() with a Mutex. Importantly, the lock is held for the entire closure, not just the call to run() — that's what fixed the race we saw on Windows where one thread's SessionOutputs were still being read while another thread had already started a new run(). Closure-scoped locking, not call-scoped.

Adaptive parallelism — the wrong turns we took

This is the part of the work that took the longest, because every textbook says "to make ONNX fast, batch your inputs". So our first attempts followed the textbook.

We tokenized chunks of 8, 16, 32 documents at a time, padded them to max_len, and ran a single forward pass per worker thread. The throughput numbers came back lower than processing the same texts one-by-one through the same session. We ran it again. Same result. We spent a while trying to disprove it before accepting it. The reverted commit 980b24b "Revert: perf(model): batch inference in worker threads" is the moment we stopped fighting and rebuilt around what the profiler kept telling us.

Two things were behind the surprise.

The padding tax. A batch of mixed-length texts pads every row up to the longest row. The model then does work proportional to batch_size * max_len * hidden_dim, regardless of how much real content is in the batch. Real text inputs are highly variable in length: a typical batch of 8 random sentences might have one 60-token outlier and seven 8-token rows. The model spends most of its cycles multiplying padding tokens against attention weights. With one-doc batches, the model only does work proportional to that doc's actual token count. Per-document, "no batching" is cheaper than "batching" once the variance in input length is realistic.

Spinning. ORT's intra-op thread pool defaults to spinning between dispatches — threads burn CPU in a tight loop waiting for the next chunk of work. With one big batch per session call this is invisible: the thread is always busy with real work. With many concurrent small calls, it becomes a disaster: every worker's intra-op pool is pinned at 100% CPU between calls, and there's no CPU left for anything else. We saw exactly this pattern in top: every core at 100%, throughput lower than spinning-off. This sounds wrong until you remember the rest of the system needs CPU time too — the tokenizer, the HNSW build, the rest of searchd. Flipping with_intra_op_spinning(false) on was a one-line change that immediately raised throughput and dropped CPU usage at the same time.

So the final shape is the opposite of the textbook recipe:

One shared session, no pool.
One document per inference call, no batching inside the worker.
Many concurrent callers, scaled to CPU count.
No spinning between calls — yield the CPU like a polite citizen.

fn predict_pipelined(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>, _> {
    let bs = batch_size();

    // Small input — single tokenize + infer, no thread overhead.
    // This is the path a 1-doc INSERT takes.
    if texts.len() <= bs {
        return Self::tokenize_and_infer(&self.session, &self.tokenizer, texts, ...);
    }

    // Large input — split across workers, each running 1-doc-at-a-time
    // through the SHARED session. This deliberately mimics the
    // many-concurrent-callers pattern that ORT is happiest with.
    let num_workers = (texts.len() / bs).min(available_cpus()).max(1);
    let docs_per_worker = texts.len().div_ceil(num_workers);

    std::thread::scope(|s| {
        for worker_texts in texts.chunks(docs_per_worker) {
            s.spawn(move || {
                for text in worker_texts {
                    Self::tokenize_and_infer(&session, &tokenizer,
                                             std::slice::from_ref(text), ...)?;
                }
                Ok(())
            });
        }
    });
    // ...
}

The two-branch design is on purpose. A 1-row INSERT comes in with texts.len() == 1, which is <= bs, so it takes the fast path with zero thread spawning, zero channel sends, zero coordination overhead. A bulk REPLACE INTO with thousands of rows takes the parallel branch and gets the throughput benefit. The cheap case stays cheap, the expensive case stays parallel.

We also enable parallel tokenization once at startup (TOKENIZERS_PARALLELISM=true) and pre-truncate inputs by character count before BPE, so a 100KB blob of text doesn't pin a CPU on the tokenizer for a second before the model even sees it.

Numbers

All runs on our standard benchmark box, using all-MiniLM-L12-v2-onnx, 1000 documents per run. Generated with manticore-load:

manticore-load --quiet --drop --batch-size=1 --threads=8 --total=1000 \
  --init="CREATE TABLE t (
    f text,
    v FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
      MODEL_NAME='onnx-models/all-MiniLM-L12-v2-onnx' FROM=''
  )" \
  --load="INSERT INTO t(f) VALUES('<text/10/100>')"

Same command with --batch-size=2, 8, 32, 128, all at 8 threads:

`--batch-size`	docs/sec	avg call latency (ms)	per-doc latency (ms)
1	143	55.9	55.9
2	113	141.6	70.8
8	91	703.3	87.9
32	146	1753.4	54.8
128	147	6966.0	54.4

Compared against Candle at the same 8 threads — which sat flat at 10 docs/sec across every batch size — that's between 9× and 15× more documents per second depending on the batch you pick. The "avg call latency" column is the time for one full INSERT statement to return, not per document; divide by the batch size and the per-doc cost lands in the 55–90 ms band.

If you swap the table to 1 client thread — the configuration that turns out to be optimal for bulk loading — the numbers climb further: 72 / 76 / 93 / 175 / 233 / 222 docs/sec at batches 1 / 2 / 8 / 32 / 64 / 128. The peak in the entire grid is 233 docs/sec at 1 thread × batch=64, with per-document latency of ~4.3 ms.

How to feed it for maximum throughput

If you're loading a lot of data in bulk and want maximum docs/sec, the recipe is straightforward: send large INSERT ... VALUES (..), (..), ... statements (batch 32–128) from a single client thread, not many small inserts from many threads. The new backend already parallelises inside the call (see the predict_pipelined code above), so client-side fan-out just piles coordination overhead on top of what ORT is already doing — that's why 1 thread × batch=64 (233 docs/sec) beats 8 threads × batch=128 (147 docs/sec) by a clear margin.

If your workload is naturally one-row-at-a-time — web requests, queue consumers, MCP servers — just use INSERT INTO. The single-thread / single-row floor of 72 docs/sec is already ~7× the old Candle path, low enough latency that this isn't a tier you need to optimise around any more.

Before vs after, across the whole grid

To make the before/after concrete, we also swept the full threads × batch grid against the old Candle/trans path on the same box, same weights:

Each X-axis tick is backend threads/batch-size. The left half (trans …) is the old Candle path — docs/sec sits at 5–11 across the entire grid no matter how many threads or how large the batch, while CPU is already pinned. The right half (onnx …) is the new path — docs/sec is an order of magnitude higher across the whole sweep. Within the new path: at small batches, adding client threads helps (1T/batch=1 = 72 → 8T/batch=1 = 143); at large batches, a single client thread wins (1T/batch=64 = 233 is the global peak).

Same sweep, but plotting efficiency (docs/sec per % CPU) alongside docs/sec. On the Candle (trans) side, both lines hug the floor — the box is spending CPU without producing documents. On the ONNX (onnx) side, efficiency is highest at 1–2 threads with mid-sized batches, where each percent of CPU buys the most embeddings, and it stays well above the old path even as we crank threads up to 32.

What's next

A few things are queued behind this work:

GPU path. The current ONNX setup is CPU-only. The _use_gpu parameter is plumbed through but not yet wired to the ORT CUDA execution provider.
Windows perf parity. We currently serialise on Windows because of an ORT threading bug. Once that bug is resolved upstream, Windows should get the same shared-session behaviour Linux/macOS already have.
More architectures down the ONNX path. Right now ONNX is the path for BERT-family encoders. T5, causal-LM and quantized GGUF models still go through Candle for now.

Try it

If your existing table is already pointed at an ONNX-capable model, the new path takes over once you upgrade to Manticore Search 27.1.5 or newer — no schema changes, no re-ingest. You should just see your INSERTs go faster.

If you're not on an ONNX model yet — or you want to move to a smaller / faster one to take maximum advantage of the new backend — note that you can't swap the model on an existing field. Manticore doesn't support altering MODEL_NAME on an existing FLOAT_VECTOR field, so migrating in place isn't an option. You have two practical paths to choose between, depending on what's easier in your setup:

Option A — dump, edit, reload. Even if you no longer have the original source data, you can mysqldump the existing table to a SQL file, edit the CREATE TABLE in that dump to point MODEL_NAME at the ONNX-optimised model you want, and replay the dump into a fresh table. Manticore will re-embed every row through the new path on the way in.

Option B — add a new column alongside, rebuild, drop the old one. If you'd rather stay in SQL and avoid the dump round-trip, add a new FLOAT_VECTOR column on the same table that points at the ONNX model, then trigger a one-shot re-embed of that column from the source text:

ALTER TABLE t ADD COLUMN v_new FLOAT_VECTOR KNN_TYPE='hnsw'
  HNSW_SIMILARITY='l2'
  MODEL_NAME='Xenova/all-MiniLM-L6-v2'
  FROM='text_field';

ALTER TABLE t REBUILD EMBEDDINGS v_new;
-- once you've cut over reads to v_new, drop the old column
ALTER TABLE t DROP COLUMN v_old;

See the Rebuilding embeddings section of the docs for the exact syntax and constraints.

On brand-new tables, none of this matters — just pick an ONNX-optimised MODEL_NAME from the start.

A good place to shop for ONNX-ready embedding models is the Xenova collection on Hugging Face — these are pre-converted to ONNX and ready to drop into MODEL_NAME='...'. Filter the list by the feature-extraction task to narrow it down to embedding-style models. Some sensible starting points:

Xenova/all-MiniLM-L6-v2 — small and fast, 384-dim, great default.
Xenova/all-MiniLM-L12-v2 — the model we benchmarked in this post, 384-dim, a step up in quality.
Xenova/bge-small-en-v1.5 — strong English retrieval, 384-dim.
Xenova/multilingual-e5-small — multilingual coverage, 384-dim.

If you aren't using auto-embeddings yet at all, the original announcement walks through the SQL from scratch.

📚 KNN search documentation
💬 Slack community — we'd love to see how the new path holds up on your data.

Український лематизатор тепер вбудовано в Manticore Search

Sergey Nikolaev — Tue, 23 Jun 2026 11:14:40 +0000

Коротко

починаючи з релізу 27.1.5 український лематизатор більше не потребує окремого Python-стека.
Раніше потрібно було встановлювати окремий пакет, Python 3.9, pymorphy2 і українські словники.
Гарна новина - тепер словник уже входить до Manticore.

Достатньо лише ввімкнути явно морфологію:

morphology='lemmatize_uk_all'

Окремо додавати українські символи до charset_table також вже не потрібно: стандартний non_cont містить мапінги для є, і, ї, ґ.
А от апостроф для української мови важливий, але тут є один важливий нюанс. Якщо просто додати його в charset_table це може зачепити данні на англійській мові, де апостроф також використовується.

Саме тому для українських текстів ми рекомендуємо використовувати окрему таблицю з власним charset_table та апострофом, а не змішувати українську з англійською чи іншими мовами в одній таблиці.

Це все що потрібно урахувати для повноцінної підтримки Української в ManticoreSearch. Ніяких словників, пакетів чи скриптів. Тепер все працює прямо "з коробки"

Що таке лематизатор

У повнотекстовому пошуку часто потрібно знайти слово не лише в тій формі, яку ввів користувач. У документі може бути мрії, а користувач шукає мрія. Або в тексті є інтернет-магазину, а в запиті приходить інтернет-магазин. Людина легко бачить, що це форми того самого слова. Для пошукового рушія без морфології це різні токени.

Для цього в пошукових рушіях використовують стемінг і лематизацію.

Стемер зазвичай працює за правилами: відкидає або замінює закінчення. Це швидко, але результат буває грубим і не завжди схожим на справжнє слово.

Лематизатор спирається на словник і морфологію, щоб отримати нормальну форму слова. Для української мови це особливо помітно через відмінки, рід і число.

Що змінилося

Якщо ви вже пробували українську лематизацію в Manticore, то проблема могла бути не в самому пошуку, а у встановленні:

окремий manticore-lemmatizer-uk;
Python 3.9 з --enable-shared;
pymorphy2 і pymorphy2-dicts-uk;
додаткові системні залежності.

Тепер український словник постачається як звичайний мовний файл uk.pak, а Manticore завантажує його напряму. Вам залишається налаштувати таблицю: вказати потрібну morphology і працювати далі.

Мінімальна конфігурація

Створимо таблицю для українських текстів:

CREATE TABLE uk_docs(title text)
  morphology='lemmatize_uk_all'
  charset_table='non_cont,U+0027';

Тут важливо ввімкнути морфологію:

morphology='lemmatize_uk_all' вмикає український лематизатор та індексує усі знайдені нормальні форми.

Для української мови додаємо лише апостроф (U+0027), щоб слова на кшталт обов'язковим індексувалися як один токен.

Для однієї нормальної форми підійде lemmatize_uk. Щоб індексувати усі можливі форми, виберіть lemmatize_uk_all.

Перевіримо на прикладі

Додамо кілька документів:

INSERT INTO uk_docs VALUES
  (1, 'мрії про червону сукню'),
  (2, 'каталог інтернет-магазину'),
  (3, 'команд-учасниць запросили на зустріч');

Запит мрія знаходить документ, де слово записано як мрії:

SELECT id, title FROM uk_docs WHERE MATCH('мрія') ORDER BY id ASC;

+------+---------------------------+
| id   | title                     |
+------+---------------------------+
|    1 | мрії про червону сукню    |
+------+---------------------------+

Запит червоний знаходить червону:

SELECT id, title FROM uk_docs WHERE MATCH('червоний') ORDER BY id ASC;

+------+---------------------------+
| id   | title                     |
+------+---------------------------+
|    1 | мрії про червону сукню    |
+------+---------------------------+

А інтернет-магазин знаходить інтернет-магазину:

SELECT id, title FROM uk_docs WHERE MATCH('інтернет-магазин') ORDER BY id ASC;

+------+---------------------------+
| id   | title                     |
+------+---------------------------+
|    2 | каталог інтернет-магазину |
+------+---------------------------+

Що відбувається з токенами

Якщо хочете побачити не лише результат пошуку, а й саму нормалізацію, використовуйте CALL KEYWORDS:

CALL KEYWORDS(
  'мрії червона інтернет-магазину команд-учасниць',
  'uk_docs'
);

+------+--------------------+--------------+
| qpos | tokenized          | normalized   |
+------+--------------------+--------------+
| 1    | мрії               | мрія         |
| 2    | червона            | червоний     |
| 3    | інтернет           | інтернет     |
| 4    | магазину           | магазин      |
| 5    | команд             | команда      |
| 6    | учасниць           | учасниця     |
+------+--------------------+--------------+

Тут добре видно різницю з простим обрізанням закінчень: на виході маємо нормальні форми слів, за якими вже можна шукати. мрії перетворюється на мрія, червона на червоний, магазину на магазин.

Що варто пам'ятати

Користуватися українським лематизатором стало простіше, але для кожної таблиці його все одно треба ввімкнути явно через morphology.

Стандартний charset_table=non_cont уже покриває українські символи є, і, ї, ґ. Якщо ви задаєте таблицю саме для українських текстів, достатньо додати до нього апостроф: charset_table='non_cont,U+0027'.

Якщо ви використовуєте офіційні пакети або образи Manticore Search актуальних версій, український uk.pak уже має бути на місці. Якщо у вас власна збірка або нестандартне розташування файлів, перевірте, що lemmatizer_base вказує на каталог, де лежить uk.pak.

Докладніше про налаштування морфології можна прочитати в документації: morphology.

Faster KNN search in Manticore: 2-pass HNSW, batched distances, and AVX-512

Sergey Nikolaev — Tue, 23 Jun 2026 10:43:59 +0000

TL;DR: Three changes to the HNSW search engine improve KNN throughput by up to 29% at high k, with over 20% gains under concurrent load. No API changes, no index rebuild, no configuration. Just faster searches.

Faster KNN search in Manticore

Manticore's KNN search is built on top of hnswlib, an open-source HNSW implementation. Historically, most of our KNN work focused on custom distance functions, such as those used for binary quantization, rather than on hnswlib's core search loop. We also added features like prefiltering with ACORN-1 and early termination, but the main search loop stayed the same: hnswlib still visited neighbors, computed distances, and maintained its set of candidates the same way.

These changes go further, modifying hnswlib's core search loop itself - restructuring how it traverses neighbors, how it calls distance functions, and how it interacts with the CPU's memory hierarchy. Combined with new AVX-512 distance implementations in the columnar library, these changes target three sources of overhead: inefficient memory access patterns, redundant data loads, and indirect function call overhead.

Compile-time distance function specialization

Previously, the distance function was a runtime function pointer stored in the HNSW index and called for every candidate. For large search budgets, that can mean a large number of indirect calls per query. Indirect calls prevent the compiler from inlining the distance function into the search loop, and they create branch prediction overhead.

The new code resolves the distance function at compile time using C++ templates. When the search begins, a single switch statement selects the right template specialization based on the distance metric and quantization settings. From that point on, the entire inner loop - neighbor traversal, distance computation, candidate set updates - runs as one monolithic function with the distance calculation fully inlined. The compiler can now optimize register allocation, instruction scheduling, and loop unrolling across the distance computation boundary.

2-pass neighbor processing

The HNSW algorithm explores the graph by visiting nodes and computing distances to their neighbors. In the original implementation, each neighbor was processed in a single pass: check if visited, fetch its vector data, compute distance, update the set of candidates. This meant that memory prefetch hints had little time to take effect before the data was needed.

The new implementation splits this into two passes. Pass 1 iterates all neighbors of the current node, skips already-visited ones, and collects the unvisited neighbors into a small batch array. As each neighbor is added to the batch, a prefetch hint is issued for its vector data. Pass 2 iterates the batch and computes distances. By the time Pass 2 reaches each vector, the prefetch from Pass 1 has had time to bring the data into cache.

Pass 2 walks a compact sequential array of candidate IDs, not the graph structure itself. The underlying vector loads are still scattered, but the data has been prefetched ahead of time.

For unfiltered queries (no WHERE clause on the KNN search), the new code also takes a fast path that eliminates the per-candidate filter check entirely.

Batched distance computation

The 2-pass structure helps in two ways: it gives prefetching more time to work, and it makes batching easy. Once Pass 2 has a compact list of candidates, it can score them two at a time instead of one by one.

When scoring two candidates, the query vector is loaded once per SIMD iteration and reused for both distance computations, eliminating redundant loads.

This reduces repeated query-side loads and lets the scoring loop process candidates in pairs, with a fallback for an odd remainder. Batch-2 functions are provided for inner product, L2, and their binary-quantized variants.

AVX-512 support

The new AVX-512 distance code processes 16 floats per iteration instead of 8 with AVX2. For inner product and L2 distance, the core loop uses fused multiply-add (_mm512_fmadd_ps), which combines multiplication and accumulation in a single instruction. For binary-quantized vectors, the AVX-512 VPOPCNTDQ extension speeds up bit-counting operations used in distance calculation.

Manticore now ships three library variants: a baseline build, an AVX2 build, and an AVX-512 build. At startup, Manticore detects the CPU's capabilities and loads the appropriate library automatically. No configuration is needed.

Benchmark results

The following benchmarks were run on the dbpedia-openai-1M-1536-angular dataset (1M vectors, 1536 dimensions, cosine distance) on an AMD Ryzen 7 9700X (Zen 5, 8 physical cores / 16 logical cores). All data uses 1-bit binary quantization with oversampling and rescoring disabled. For multithreaded runs, throughput is reported as average per-thread queries per second: each worker runs its own batch of queries, its QPS is measured independently, and the final number is the average across workers. Each result is the average of 6 independent runs. Early termination was also disabled to isolate the effect of these optimizations on raw HNSW traversal.

Zen 5 was chosen because it supports AVX-512 with native 512-bit datapaths, avoiding the split-512 execution behavior and heavy AVX-512 downclocking associated with some older Intel processors. This helps isolate the algorithmic effects of these changes from CPU-specific AVX-512 throttling behavior.

Algorithmic improvements alone

The first chart isolates the effect of the algorithmic changes (2-pass processing, batched distances, compile-time dispatch) by comparing the new AVX2 build against the previous AVX2 build. Both builds use the same SIMD instruction set, so the difference is purely from the new code structure.

On a single thread, the gain grows steadily from +3% at k=10 to +24% at k=1000 as distance computation comes to dominate the search workload. With more threads competing for memory bandwidth, the per-thread gain shrinks: +9-10% at 4 or 8 threads, and only +2-5% at 16 threads.

The 16-thread case is SMT (each physical core runs two threads). Distance computation is memory-bound, so when two threads share a core's L1/L2 caches, the prefetching and batching wins are partially absorbed by shared-resource contention. The algorithmic improvements still help, but the headroom shrinks.

SIMD width benefit (AVX-512 vs AVX2)

The second chart isolates the effect of AVX-512 by comparing the AVX-512 build against the new AVX2 build (both share the same algorithmic improvements).

AVX-512 is slightly slower than AVX2 at k=10 (around -2%) regardless of thread count. This is specific to AVX-512: the algorithmic improvements alone don't show this regression, so it's not a uniform per-query overhead. From k=30 upward, AVX-512 pulls ahead at every thread count.

The interesting pattern is that AVX-512's benefit grows with thread count. Although this benchmark disables oversampling, the default Manticore KNN query uses LIMIT 20, and with the default oversampling=3.0 (which multiplies the effective HNSW search budget for rescoring after quantized search) that becomes k=60 internally. At k=60, AVX-512 vs AVX2 (new) is +1.2% on a single thread, +2.6% at 4 threads, +3.4% at 8 threads, and +6.5% at 16 threads.

Combined improvement (AVX-512 vs the old code)

The third chart shows the cumulative effect: AVX-512 with all the new code, compared against the previous AVX2 build. This is what a user upgrading from the previous Manticore version to the new one would see if their CPU supports AVX-512.

The single-thread curve climbs from +0.5% at k=10 to +29% at k=1000. The multi-thread curves all reach +22-24% at k=1000. The improvement is broadly distributed across thread counts - the algorithmic and SIMD gains compose differently at different concurrency levels, but the combined result is consistently large at moderate-to-high k.

Why the gain grows with k

All three charts show the same shape: small improvement at low k, large at high k. The reason is that low-k queries spend a larger share of their time on graph traversal (visiting nodes, checking visited bits, popping the candidate set) - work that scales with the graph structure, not k. As k grows, the effective search budget grows proportionally, and the queries spend more time on distance computation. The optimizations target distance computation and the loops around it, so their benefit scales with the share of work that distance computation represents.

What this means for you

These improvements require no action. They are available in the recent Manticore Search 27.1.5 release; there are no API changes, no new configuration options, and no need to rebuild indexes.

The gains stack with the KNN early termination: early termination reduces the number of distance computations per query, and these optimizations make each computation faster.

The biggest improvements show up with:

High-dimensional vectors (more arithmetic per distance computation, more SIMD benefit)
Large k values (more total distance computations, more opportunity for batching and cache optimization)
Queries with oversampling (oversampling multiplies the effective k, pushing queries into the range where gains are largest)

Manticore Search 27.1.5: Authentication, sharded tables, conversational search and faster vector search

Sergey Nikolaev — Mon, 22 Jun 2026 02:19:45 +0000

Manticore Search 27.1.5 has been released. This release brings built-in authentication and authorization, sharded tables, conversational search, faster HNSW builds, better faceting and aggregations, and a long list of fixes across KNN, replication, protocol compatibility and other areas.

This post is a catch-up for everything shipped from 25.0.1 through 27.1.5.

Upgrade Notes

Please review these before upgrading:

27.0.0 adds built-in auth/authz, and enabling it changes access assumptions. Auth is not enabled by default, but once you enable it, anonymous access no longer works. Roll it out in stages: upgrade remote agents and replication peers first, then upgrade the masters that query or manage them, and enable auth only after the whole topology is on the new version. Distributed remote-agent and replication-related operations also need matching stored auth data across the participating daemons. A successful JOIN CLUSTER replaces the joining node's local auth data with the donor cluster's auth data. (Issue #2833, PR #3648)
26.0.0 changed replication storage layout. Incoming replicated tables now live under the normal data_dir/<table> layout instead of the cluster path. If you run replication clusters with a custom path, you may need to move or re-synchronize replicated tables after upgrade. Downgrade is only safe before the new layout is adopted. (Issue #4431, PR #4598)
If you manage MCL separately from the daemon, upgrade it together with Manticore. This release line moves through several MCL updates, from vector-performance work to multithreaded HNSW builds and later stability fixes. Mixing an older library with a newer daemon is not recommended. (25.2.0, 25.15.0, 26.0.3, 26.3.2, 27.1.0)

Highlights

Built-in authentication and authorization

Manticore now supports users, passwords, bearer tokens, and fine-grained permissions across MySQL, HTTP/HTTPS, distributed remote agents, and replication-related operations. This makes access control a first-class part of the product instead of something that always has to be handled outside the database.

Sharded tables

Manticore can now create and manage sharded tables, distribute inserts across shards, and handle more of the surrounding lifecycle in one place. That makes larger write-heavy deployments easier to operate and reduces the amount of sharding-specific logic that has to live outside the engine.

Conversational search

This release adds conversational search to Manticore Search. It is exposed through CREATE CHAT MODEL and CALL CHAT, so you can ask questions over an existing vectorized table instead of building a separate retrieval layer around the same data.

Under the hood, Manticore Search runs KNN on a FLOAT_VECTOR field, builds LLM context from that field's from='...' source columns, keeps conversation history by conversation_uuid, and returns both the answer and the supporting sources. If you already keep embeddings in Manticore, this makes document Q&A and support-style assistants much easier to wire up.

Faster vector builds and KNN improvements

Vector search kept improving throughout this cycle.

Manticore improved KNN performance, added local ONNX embeddings support, sped up ONNX inference, and then made HNSW build and rebuild work much faster with multithreaded index construction.

A few important steps in that work:

25.1.0 improved KNN distance calculation and AVX-512 loading.
25.2.0 added local ONNX embeddings support in MCL and improved vector-search performance further.
25.14.0 and 25.15.0 added multithreaded HNSW builds together with the required library support.

The biggest practical improvement here is a much faster auto-embedding and shorter build and rebuild time for large vector tables. Initial KNN builds, chunk merges, and ALTER TABLE ... REBUILD KNN are all affected.

Better faceting and aggregations

Faceting and aggregations also became more useful.

facet_filter_mode makes it easier to build e-commerce-style filters that preserve selected, available, and unavailable buckets under active filtering.

On the analytics side:

date_histogram() gained time_zone and offset
Opensearch dashboards support
Manticore added statistical aggregations such as percentiles, percentile_ranks, and mad

Other Notable Improvements

This release line also includes several smaller but useful additions:

searchd --check validates configuration before startup without side effects.
EXIT CLUSTER lets a node leave a replication cluster online without restarting.
dict=keywords_32k makes it possible to index very long machine-generated tokens such as hashes and message IDs without silent truncation.
The built-in Ukrainian lemmatizer expands native morphology support for Ukrainian text search.
Systemd Type=notify improves startup and shutdown supervision.
searchd process under systemd management now logs to systemd journal
JOIN queries now support explicit left-table column prefixes.
OpenSearch Dashboards support.
manticore-load gained multi-query support.

Bug Fixes

This release line also includes 65 changelog-listed fixes. The latest follow-up releases added a few more worth calling out:

27.1.5 fixed a crash when fetching columnar float_vector attributes.
27.1.4 fixed ALTER TABLE ... RECONFIGURE and SHOW CREATE TABLE for one-way upgrades from dict='keywords' to dict=keywords_32k.
27.1.3 updated Buddy to 4.0.1 and tightened Queue-plugin mutation permission handling under auth.
KNN-by-doc_id queries now preserve offset and max_matches correctly.
KNN rescoring order was fixed, so explicit ORDER BY tie-breakers work again.
Hybrid fused queries with GROUP BY on columnar tables stopped crashing.
Replication and node-rejoin crash paths were cleaned up further.
Binary MySQL protocol behavior was fixed in 25.12.1, which matters for integrations that expect real client compatibility.
Fluent Bit bulk-ingest interoperability was fixed, preventing successful responses from being replayed as duplicate inserts.
27.1.2 fixed sql_attr_multi handling for plain indexes built from multiple source blocks.

For the complete list, see the changelog.

Need help or want to connect?

Join our Slack
Visit the Forum
Report issues or suggest features on GitHub
Email us at contact@manticoresearch.com

The Evolution of 'More Like This

Sergey Nikolaev — Tue, 02 Jun 2026 04:01:20 +0000

In many search scenarios, the user does not start from an empty query box, but from an existing result.

A user opens an article and wants to find related material. A buyer views a product card and looks for close alternatives. A support engineer investigates an incident and wants to see earlier cases with the same symptoms. In all these situations, the user already has a relevant document to start from.

This scenario is traditionally called More Like This (MLT): a function for finding documents similar to the selected one. In this article, MLT means search that starts from a known document, not from a newly typed query.

The classic MLT approach, or similar-document search, was based on comparing textual matches. Modern implementations increasingly use embeddings: numerical representations of documents. A search index stores embeddings as vectors, and the search system can find documents with close vector representations.

Short glossary

To avoid repeating definitions throughout the article, here are the main terms:

Term	Meaning in this article
More Like This (MLT)	search for documents similar to an already selected document
embedding	a numerical representation of text, a product, an image, or another object
embedding vector	a numerical representation of an object, such as text or a product, stored in the index to find similar objects by vector proximity
KNN, nearest-neighbor search	search for nearest neighbors, meaning objects with close vectors
ANN, approximate nearest neighbors	approximate nearest-neighbor search; it speeds up KNN on large datasets without scanning every vector
RAG, Retrieval-Augmented Generation	an approach where the search system retrieves context for a generative model
hybrid search	combining full-text search and vector search in one scenario
reranking	an additional sorting step for already retrieved candidates using a more precise model or rule

What classic More Like This did

Classic MLT was lexical. It answered a simple question: which documents use similar important words?

The process usually looked like this:

The search system took the source document.
It analyzed its text.
It selected informative terms.
It built a query from those terms.
It searched for documents with a similar set of words.
It returned a list of similar documents.

Internally, this used familiar full-text search mechanisms: TF-IDF or BM25, term frequency, stopwords, field boosts, and document-frequency limits. That is why older MLT implementations exposed parameters such as min_term_freq, min_doc_freq, max_doc_freq, and max_query_terms.

This was not just an interface element, but a full search mechanism. MLT was used for related articles and products, duplicate detection, support-ticket matching, legal search, patent research, and internal knowledge bases.

Where the lexical approach is still strong

Lexical MLT works well when specific words, identifiers, and stable formulations matter.

Examples:

error codes;
product SKUs;
part numbers;
function names;
stack traces;
legal wording;
nearly identical product or ticket descriptions.

The reason is that exact matching is critical here. If two incident reports contain the same error code or the same stack trace, full-text search sees a direct match. For example, when searching tickets with the code ERR_404, lexical MLT quickly finds every mention of that code, while vector search may return tickets that describe similar but not identical problems.

Lexical MLT had another advantage: it was cheap to run. The inverted index is already in the search engine. The analyzers are already configured. Ranking already works. There is no need to deploy separate search infrastructure just to support a “find similar” feature.

The limitation is also clear. If two documents describe the same thing in different words, lexical MLT may fail to connect them. Synonyms work unevenly. Paraphrases are harder. Cross-lingual similarity is usually unavailable. For example, memory leak and unbounded heap growth may describe the same problem, but a standard analyzer sees different tokens.

Lexical MLT efficiently finds documents with matching or similar wording. Semantic search helps when the meaning matches, not the words.

What embeddings change

Using embeddings — numerical representations of documents — changes the comparison principle: instead of words, the system compares vector representations.

A document no longer has to be represented only as a set of weighted terms. It can be stored as a dense vector. Nearby vectors usually correspond to documents that are similar in meaning, even if they are written in different words.

The lexical approach looks for matches by words and terms, while embedding search looks at the proximity of document vector representations. The first approach is optimal for exact matches such as error codes and SKUs. The second finds semantically close documents, even when they are phrased differently.

This expands the scope of this kind of search. You can compare not only articles, but also products, images, code fragments, user events, or context fragments in a RAG system. In RAG, the search system first retrieves relevant context, and then the generative model uses that context to produce an answer.

Lexical search does not disappear. Exact error codes, SKUs, names, statute references, and near duplicates are still better handled lexically. That is why production systems often use hybrid search: full-text search provides exact matches, vector search adds results by meaning, filters constrain the search space, and reranking refines the final order.

As shown in our comparison of lexical and vector search, the former wins on precise strict matches, while the latter improves coverage of semantic relationships.

MLT as lookup by a vector from the index

If a vector representation has already been computed for a document and stored in the index, modern MLT can be described without a separate API example:

Take the source document.
Retrieve its precomputed vector representation from the index.
Find the nearest vectors.
Return the documents those vectors belong to.

This is still More Like This: the user starts from one document and gets related results. Only the comparison method changes. Instead of extracting terms, the search system uses the vector representation of the source document.

In Manticore Search, this operation can be performed directly at the search-engine level: the query specifies the ID of the source document, and Manticore takes its embedding vector from the index and runs KNN search. The application does not need to fetch the vector separately, serialize hundreds or thousands of numbers, and send them back in a second request.

A minimal SQL example looks like this:

SELECT id, title, knn_dist()
FROM products
WHERE knn(embedding, 10, 123)
LIMIT 10;

Here, embedding is the field with the precomputed embedding vector, 123 is the ID of the source document, and 10 is the number of nearest documents to return. The knn_dist() function returns the distance between vectors: a smaller value means greater semantic proximity to the source document. The same operation can be performed through the HTTP JSON API; the search logic does not change. The application passes the document ID, and Manticore performs lookup using that document’s vector from the index.

For large datasets, KNN is usually implemented through an ANN index. This speeds up search through approximate computation and avoids scanning every vector. For the user, the important part is not the internal structure of the index, but the result: quickly finding documents that are close to the source in meaning.

Why search is better handled in the engine

You can implement this scenario in the application: first fetch the document, then extract its vector, then send a separate KNN query, and then combine the result with filters.

That approach makes the system architecture more complex. The application has to:

pass the vector between services;
prevent accidental logging;
check the embedding model version;
keep data synchronized with the main index;
apply the same filters used in normal search.

When the search system performs the lookup itself, the path is shorter:

The application passes the ID of the source document.
The search system finds the precomputed vector representation in the index.
The search system runs nearest-neighbor search (KNN) or its approximate variant (ANN).
The search system returns the found documents with the same access filters and metadata.

Benefits of this approach:

fewer inter-service requests from the application;
large vectors do not have to be sent through external APIs;
filters stay close to search;
the result is easier to reproduce and debug;
the application does not need an additional layer for similarity calculation — everything runs inside the search engine.

This will not fix poor embeddings or remove the need to tune ranking. But it reduces the number of interacting components in the search chain, which makes the system easier to maintain.

Practical examples and the evolution of MLT

Search from an existing object is especially useful when the user has already found a relevant starting point.

Scenario	Source object	What to find
Support	ticket with an error	past tickets with similar symptoms and related fixes
Catalog	product card	close alternatives, similar models, or products from the same category
RAG	relevant fragment already found by the first search	context expansion: neighboring sections of the same document, related documentation fragments, or similar discussions
Developer tools	stack trace, diff, or bug description	related code changes, discussions, and past incidents

In these examples, there is no need to type a new query manually. The system uses the source object as a reference point and finds documents similar to it lexically, semantically, or by both criteria.

In the context of RAG, this is not about the primary search by the user’s query, but about subsequent context selection: the system has already found a relevant fragment and uses it as the reference object to collect surrounding context. This is useful when one fragment is too narrow: nearby content may include a term definition, a configuration example, a related discussion, or a neighboring section of the same guide.

In systems with personalization or AI agents, it is important to clearly define which data is used for search: the system may consider the user’s search-query history, the context of previous interactions, or saved working notes. This makes it clear which data participates in retrieval and why the result is considered similar.

The evolution of MLT can be described like this:

Period	What changed
2000s	MLT mostly relied on lexical analysis, TF-IDF, BM25, and term overlap.
2010s	Word2Vec and GloVe appeared and became widely used, making it possible to build semantic embeddings of words and texts.
Early 2020s	FAISS and similar ANN libraries made it possible to run vector search efficiently even on very large datasets.
Mid-2020s	RAG, recommendations, and search from an existing object made lookup by stored vectors a common product scenario.

The evolution of MLT is a shift from lexical comparison to matching document vector representations. But the practical request stayed the same: find documents relevant to the source result.

What to keep in mind

Semantic MLT does not replace all search engineering.

Production systems still need:

exact search for identifiers, error codes, and other strict matches;
embedding model metadata and versioning;
ACL filters: rules for document access by roles or users;
tenant filters: data isolation between customers or workspaces;
hybrid search when both meaning and exact matches matter;
reranking when result order is critical;
search-quality monitoring: precision and recall metrics, false-positive frequency, and missed relevant documents caused by ANN-index approximation errors.

Lexical MLT can miss documents that use different words. Vector search sometimes returns overly broad results, or false positives, and can miss relevant documents because of the approximate nature of ANN indexes. That is why the quality of this kind of search should be evaluated on real queries and real data.

Conclusion

More Like This has moved from purely lexical search to hybrid solutions that combine lexical, vector, and filtering mechanisms.

The core concept remains the same: the user selects a source document, and the system finds materials relevant to it, taking both lexical and semantic similarity into account.

KNN early termination in Manticore Search

Sergey Nikolaev — Mon, 01 Jun 2026 09:51:55 +0000

Modern search engines do more than match keywords. When you search for "cozy mystery set in Paris" and get results for "atmospheric detective novel in France" that's vector search at work: documents and queries are converted into lists of numbers, called embeddings, and the search engine finds the documents whose numbers are closest to the query's.

Manticore Search supports this natively. Under the hood, it uses a data structure called HNSW: a graph that connects nearby vectors, so it can find nearest neighbors quickly without scanning every document. That makes vector search fast enough to run on millions of documents in milliseconds.

But HNSW has an inefficiency. Early in the traversal, almost every distance computation finds a better candidate than the ones already in the result set.

As the search goes on, those improvements become rarer, but the algorithm keeps traversing the graph until it exhausts its exploration budget. By that point, the result set has often already converged, and the remaining work does little or nothing to improve it. Early termination fixes this by detecting that point and stopping early.

The effect becomes more noticeable as k grows, where k is the number of nearest neighbors the query asks Manticore to return. Returning more neighbors requires more graph exploration, and much of that extra work happens after the result set has already stabilized. That also makes early termination more valuable, because it has more unnecessary work to cut.

This gets more pronounced with vector quantization. Quantization compresses stored vectors to save memory, which slightly lowers search precision. To recover it, Manticore uses oversampling: it fetches 3x more candidates than requested, then rescores them using the original full-precision vectors. With the default 3x oversampling, HNSW explores many more candidates per query. Large k values often come from this kind of candidate expansion: an application may ask the vector index for hundreds or thousands of candidates, then rescore, rerank, or filter them down to a much smaller final result set to improve recall and precision. That raises latency, and early termination helps win some of it back.

The waste is measurable. Benchmarks on a 1M-vector dataset show that with k=60, which is the default result limit with default 3x oversampling, early termination reduces distance computations to about 65% of the full search. At k=1000, computations drop to 30%. At k=10000, just 20%. The search converges long before the exploration budget runs out, and the savings grow with k.

Early termination lets Manticore detect this convergence and stop. The algorithm was designed with a specific precision target: lose no more than 2-4% of result set precision compared to a full HNSW search.

How it works

The algorithm tracks a simple signal: discovery rate - the fraction of distance computations that actually improve the result set.

Each time a new node's distance is computed, one of two things happens: either it's good enough to enter the heap - the priority queue that holds the current best candidate neighbors - or it's worse than everything already there and gets discarded. Entering the heap counts as a "discovery." Early in the search, discoveries are frequent - the heap is filling up and most candidates are useful. As the search progresses and the heap saturates with good results, discoveries become rare. Most new distance computations just confirm that the algorithm has already found the best candidates.

Manticore monitors this transition. After each round of neighbor expansion, it computes the discovery rate:

discovery_rate = new_candidates_collected / distances_computed

If this rate stays below a threshold for several rounds in a row, the search stops.

The idea is simple: if the algorithm keeps computing distances but nothing improves the result, the search has converged.

The threshold: quantile-based adaptation

That raises the obvious next question: what threshold should count as "low"? A fixed threshold wouldn't work well - different datasets and different regions of the same dataset have wildly different discovery rate distributions. What counts as "low" depends on context.

Manticore uses a quantile-based adaptive threshold. Instead of comparing the discovery rate against a fixed number, it continuously estimates a low percentile of recent rounds (20th percentile, or 14th percentile for L2 distance) and uses that as the baseline. This keeps the method lightweight while letting it adapt to different datasets and different regions of the graph.

In other words, the threshold adapts to the local search pattern. If the algorithm enters a sparse region of the graph, the threshold drops and avoids stopping too early. If it enters a richer region, the threshold rises.

Patience: how many bad rounds before stopping

The threshold alone is not enough, though. A single round with a low discovery rate isn't enough to declare convergence. It could just be a temporary dip before the search finds a better path. Manticore uses a "patience counter" that requires multiple consecutive bad rounds before terminating.

The patience value scales inversely with ef, the HNSW exploration factor that controls how many candidates the search keeps exploring. For example, patience ranges from 9 at low ef values down to 6 at very high ef. Larger ef values mean more total rounds, so even with lower patience the algorithm has seen more evidence before deciding to stop. The counter resets to zero whenever a round has a healthy discovery rate, so a single good round restarts the patience window. This prevents the algorithm from stopping during a temporary plateau that leads to a productive region of the graph.

Warm-up phase

The algorithm ignores the termination signal while the heap is still filling up, meaning fewer than ef candidates have been collected. During this phase, discovery rates are artificially high because almost everything enters the heap, so the signal is not useful. Early termination only starts once the heap is full and new candidates must replace existing ones.

Benchmark results

The quantile thresholds were tuned to keep precision loss within 2–4%. They were tuned separately for L2 and cosine/IP distance metrics, and validated across both quantized and non-quantized data.

The following benchmarks were run on the dbpedia-entities dataset (1M vectors, 768 dimensions) on a machine with 8 physical cores / 16 logical cores.

"Precision" here means the fraction of true k-nearest neighbors that appear in the result set (with fixed k, this is the same as recall@k).
"Precision ratio" is the precision of HNSW with early termination ("ET") divided by precision without it (1.0 means no precision loss).
"Visit ratio" is the fraction of distance computations performed compared to full HNSW search (lower is better).

Oversampling and rescoring were disabled to isolate the effect of early termination on raw HNSW traversal.

The green line on the chart (precision) stays almost flat across all k values, with precision ratio remaining above 0.97 throughout the benchmark. Meanwhile the orange line (visit ratio) drops steeply. At k=100, it cuts distance computations nearly in half. At k=1000, it saves 70%. At k=10000, 80%.

At k <= 10, early termination is disabled because the search is already cheap and the savings are too small to justify any precision loss. The savings grow with k, because larger result sets lead to more rounds of neighbor expansion and more chances to detect convergence early.

Performance under concurrent load

The benchmarks above show that early termination cuts distance computations a lot while preserving precision. But what does that mean for actual query latency, especially under concurrent load? The chart below shows latency ratios (ET / no ET) at 1, 8, and 16 concurrent threads on the same dbpedia dataset:

At k=1000, early termination reduces distance computations by 71% (ratio 0.29). The latency improvement depends on how many threads are running at the same time:

1 thread: 24% faster (ratio 0.76)
8 threads: 45% faster (ratio 0.55)
16 threads: 48% faster (ratio 0.52)

The distance computation savings stay the same regardless of thread count, but the latency benefit nearly doubles from 1 to 16 threads.

The main reason is lower pressure on the CPU memory system. Each distance computation pulls vector data and graph links into cache. When several threads run HNSW traversal at the same time, they compete for shared cache and memory bandwidth. Doing fewer distance computations per query reduces memory traffic, keeps each thread’s working set smaller, and lowers cache churn between queries. As a result, each thread finishes faster and interferes less with the others.

Single-thread benchmarks understate the benefit of early termination. Under production-like concurrent load, the percentage latency reduction is roughly twice as large.

When early termination kicks in (and when it doesn't)

Early termination is enabled by default and works on both quantized and non-quantized vector data. It is automatically disabled when k <= 10.

The benefit grows with the effective exploration budget, which is max(ef, k). Since hnswlib uses this internally as the number of candidates it keeps in play, larger k means more candidates, more rounds, and more chances to detect convergence.

Quantized vectors are typically used with rescoring and oversampling (both enabled by default) to recover precision lost from quantization. Oversampling (default 3x) multiplies the effective k passed to HNSW. For example, a query with k=100 uses 300 candidates internally when oversampling is 3x. That larger search budget gives early termination more room to detect convergence and stop early. Since the performance benefit of early termination grows with k, oversampling pushes queries into the range where the savings are largest.

Syntax

Early termination is on by default. To disable it:

SQL:

-- default: early termination enabled
SELECT id, knn_dist()
FROM products
WHERE knn(embedding, (0.12, 0.45, 0.78, 0.33));

-- explicitly disable early termination
SELECT id, knn_dist()
FROM products
WHERE knn(embedding, (0.12, 0.45, 0.78, 0.33), { early_termination=0 });

-- combine with other KNN options
SELECT id, knn_dist()
FROM products
WHERE knn(embedding, (0.12, 0.45, 0.78, 0.33), { ef=200, early_termination=0 });

JSON:

POST /search
{
    "table": "products",
    "knn": {
        "field": "embedding",
        "query": [0.12, 0.45, 0.78, 0.33],
        "early_termination": false
    }
}

When to disable it

There are a few scenarios where you might want to turn early termination off:

Maximum precision is critical. Early termination trades a small amount of recall for speed. If your application requires the absolute best recall that HNSW can provide at a given ef, disable it.
Small k values (<= 30). The algorithm auto-disables for k <= 10, but even for k between 11 and 30, the performance benefit is modest. If you notice any recall difference in this range, disabling early termination costs little in latency.
Benchmarking HNSW recall. If you are measuring HNSW recall, you probably want deterministic behavior without adaptive shortcuts. Disable early termination to get a clean baseline.

How it relates to other KNN optimizations

Early termination is one of several optimizations that Manticore applies to KNN search. It works independently of and stacks with the others:

Prefiltering reduces wasted work by skipping filtered-out documents during HNSW traversal. Early termination reduces wasted work by stopping the traversal once the result set has converged. They solve different problems and work well together.
Oversampling retrieves more candidates than k to improve recall after rescoring. Early termination can reduce the cost of that expanded search by stopping once enough good candidates have been found.
Rescoring recalculates distances using full-precision vectors after the initial search with quantized vectors. Early termination operates during the initial quantized search phase, reducing the number of candidates evaluated before rescoring kicks in.
Automatic brute-force fallback skips HNSW entirely when a linear scan is cheaper. Early termination only applies when HNSW is actually used.

How to Make xt850 Match xt 850

Sergey Nikolaev — Fri, 08 May 2026 05:30:14 +0000

TL;DR

Since version 23.0.0, Manticore can make searches like xt850 match xt 850 using bigram_delimiter together with digit-aware bigram_index modes.

This solves a common tokenization mismatch in product search, where users remove spaces from model names but the source data stores them as separate tokens.

Assumptions and verification

This article assumes:

RT tables created with SQL examples exactly as shown
default tokenization unless the example explicitly changes a setting
ASCII digits in model names, because second_numeric and second_has_digit are digit-aware modes built around 0-9

All SQL examples and expected outputs in this article were verified against a real Manticore 23.0.0 instance before publishing, using fresh tables created from scratch for each scenario.

The broader search problem

Imagine a catalog containing:

xt 850 action camera
iphone 5se battery case
canon eos 80d body
thinkpad x1 carbon

Now imagine users searching for:

xt850
iphone5se
eos80d
thinkpadx1

From the user's point of view, these should obviously match. From the engine's point of view, they often do not, because the indexed text is tokenized as separate terms.

Search systems usually attack that mismatch in one of four ways:

index prefixes or infixes
add custom normalization rules
duplicate content into alternate normalized fields
index adjacent token pairs and optionally store glued variants too

Manticore's newer bigram functionality is a structured way to do the fourth option without awkward field duplication.

Baseline: why `xt850` fails by default

Here is the problem in its simplest form:

DROP TABLE IF EXISTS bi_default_demo;

CREATE TABLE bi_default_demo(title text);

INSERT INTO bi_default_demo VALUES
  (1,'xt 850 action camera');

SELECT id, title FROM bi_default_demo WHERE MATCH('xt850');

Expected result:

Empty set

Why does this fail?

Because the document is indexed as two separate tokens, xt and 850, while the query is a single token, xt850.

By default, Manticore does not assume that:

xt850 should be split into xt + 850
or xt + 850 should also be searchable as xt850

So this is not really a typo-tolerance problem or a phrase problem. It is a tokenization mismatch: the index sees two tokens, while the query provides one.

That is the gap the newer bigram settings are designed to close. They let Manticore index selected adjacent token pairs in a form that can also match glued queries.

Why bigrams help here

bigram_index can help with both phrase acceleration and model-name matching, and in this article we focus on the xt 850 vs xt850 problem.

The key idea is simple:

detect adjacent token pairs that look like model names
store those pairs in a glued form too
let queries such as xt850, iphone5se, or thinkpadx1 hit the spaced text

That is where bigram_delimiter matters.

A note about bigram_delimiter

bigram_index decides which adjacent pairs are eligible.

bigram_delimiter decides how eligible bigrams are stored:

true: internal delimited token only
none: glued token only, such as galaxy24
both: both forms

The practical difference is easiest to understand from the query side:

with true, Manticore keeps the internal bigram form used for phrase optimization, but it does not keep the glued user-facing form, so a query like xt850 will not match xt 850
with none, Manticore keeps only the glued form, so xt850 can match xt 850, but you are leaning entirely on the glued representation for those pairs
with both, Manticore keeps both the internal bigram representation and the glued form, so xt850 can match xt 850 without giving up ordinary phrase behavior

For this use case, both is usually the safer default because it covers the user-visible problem directly while keeping behavior less surprising for normal phrase queries and mixed workloads.

Mode 1: `second_numeric`

bigram_index = second_numeric
bigram_delimiter = both

This mode is aimed at model names where the second token is purely numeric.

That is common in product catalogs:

xt 850
galaxy 24
playstation 5
pixel 8

The idea is simple: users often search these as glued terms such as xt850, galaxy24, or playstation5, even though the source text stores them with a space.

second_numeric stores the pair only when the second token is ASCII digits only.

Use it when:

you have product generations and numbered models
users often remove spaces in search
the second token is usually just digits

Example

DROP TABLE IF EXISTS bi_second_numeric_demo;

CREATE TABLE bi_second_numeric_demo(title text)
  bigram_index='second_numeric'
  bigram_delimiter='both';

INSERT INTO bi_second_numeric_demo VALUES
  (1,'xt 850 action camera'),
  (2,'galaxy 24 ultra'),
  (3,'playstation 5 slim'),
  (4,'iphone 5se case'),
  (5,'canon eos 80d body'),
  (6,'thinkpad x1 carbon');

Then test the queries one by one:

SELECT id, title FROM bi_second_numeric_demo WHERE MATCH('xt850');

+------+----------------------+
| id   | title                |
+------+----------------------+
|    1 | xt 850 action camera |
+------+----------------------+

SELECT id, title FROM bi_second_numeric_demo WHERE MATCH('galaxy24');

+------+-----------------+
| id   | title           |
+------+-----------------+
|    2 | galaxy 24 ultra |
+------+-----------------+

SELECT id, title FROM bi_second_numeric_demo WHERE MATCH('playstation5');

+------+--------------------+
| id   | title              |
+------+--------------------+
|    3 | playstation 5 slim |
+------+--------------------+

SELECT id, title FROM bi_second_numeric_demo WHERE MATCH('iphone5se');

Empty set

SELECT id, title FROM bi_second_numeric_demo WHERE MATCH('eos80d');

Empty set

SELECT id, title FROM bi_second_numeric_demo WHERE MATCH('thinkpadx1');

Empty set

That boundary is the whole point of the mode:

24 and 5 qualify
5se, 80d, and x1 do not

Mode 2: `second_has_digit`

bigram_index = second_has_digit
bigram_delimiter = both

This mode is the more flexible sibling of second_numeric.

It stores the pair when the second token contains at least one ASCII digit. That makes it a much better fit for real product catalogs, where model identifiers are often mixed alphanumeric strings:

xt 850
iphone 5se
eos 80d
thinkpad x1

Use it when:

your model names mix letters and digits
users frequently remove spaces in their searches
you want catalog-friendly matching without indexing every pair in the table

Example

DROP TABLE IF EXISTS bi_second_has_digit_demo;

CREATE TABLE bi_second_has_digit_demo(title text)
  bigram_index='second_has_digit'
  bigram_delimiter='both';

INSERT INTO bi_second_has_digit_demo VALUES
  (1,'xt 850 action camera'),
  (2,'galaxy 24 ultra'),
  (3,'playstation 5 slim'),
  (4,'iphone 5se case'),
  (5,'canon eos 80d body'),
  (6,'thinkpad x1 carbon'),
  (7,'kindle paperwhite signature');

Then test the queries one by one:

SELECT id, title FROM bi_second_has_digit_demo WHERE MATCH('xt850');

+------+----------------------+
| id   | title                |
+------+----------------------+
|    1 | xt 850 action camera |
+------+----------------------+

SELECT id, title FROM bi_second_has_digit_demo WHERE MATCH('galaxy24');

+------+-----------------+
| id   | title           |
+------+-----------------+
|    2 | galaxy 24 ultra |
+------+-----------------+

SELECT id, title FROM bi_second_has_digit_demo WHERE MATCH('iphone5se');

+------+---------------------+
| id   | title               |
+------+---------------------+
|    4 | iphone 5se case     |
+------+---------------------+

SELECT id, title FROM bi_second_has_digit_demo WHERE MATCH('eos80d');

+------+---------------------+
| id   | title               |
+------+---------------------+
|    5 | canon eos 80d body  |
+------+---------------------+

SELECT id, title FROM bi_second_has_digit_demo WHERE MATCH('thinkpadx1');

+------+---------------------+
| id   | title               |
+------+---------------------+
|    6 | thinkpad x1 carbon  |
+------+---------------------+

SELECT id, title FROM bi_second_has_digit_demo WHERE MATCH('kindlesignature');

Empty set

This is often the better fit for mixed model identifiers, because real catalog data frequently includes forms like 5se, 80d, or x1 rather than only clean numeric suffixes like 24.

How to choose between the two

If your search problem is specifically "How do I make xt850 find xt 850?", the practical rule is:

use second_numeric when the second token is digits-only
use second_has_digit when the second token may be mixed, like 5se, 80d, or x1

There is one practical caveat: this is compatible with other common text-processing settings in the straightforward case. xt 850 still matches xt850 with morphology='stem_en' enabled and with a wordforms rule enabled.

But that does not mean those settings rewrite the glued query for you. In tests, iphones 5 matched iphones5, but not iphone5, even with stemming or a wordforms rule mapping iphones to iphone. So the short version is: basic xt 850 vs xt850 matching stays compatible with morphology and wordforms, but if you rely on them, test the exact query shape you care about.

Final takeaway

The xt850 problem is not really about one product name. It is about a broader mismatch between how users type model names and how search engines tokenize them.

Since version 23.0.0, Manticore gives you a built-in way to handle that mismatch with bigram_delimiter plus the digit-aware bigram_index modes, which is much cleaner than duplicating fields or inventing custom preprocessing pipelines.

If your main problem is phrase-search performance rather than glued model-name matching, see How to Speed Up Phrase Search with bigram_index.

How to Speed Up Phrase Search with bigram_index

Sergey Nikolaev — Thu, 07 May 2026 08:50:15 +0000

TL;DR

bigram_index can be used for several purposes, and in this article we focus specifically on phrase-search performance: on the 1M-document benchmark below, bigram_index='all' improved QPS by about 2.9x and cut average phrase-query latency by about 3.2x.

If your main problem is matching xt850 against xt 850 rather than speeding up phrase search, see How to Make xt850 Match xt 850.

Phrase search can be expensive. Even when a query is short, the engine still has to verify ordering and adjacency, and that work gets more noticeable when:

the individual words are common
the dataset is large
phrase queries are frequent in your workload

That is exactly what bigram_index is for.

What bigram indexing actually does

Normally, a phrase like "noise cancelling headphones" is handled as separate tokens that also need to appear in the right order and next to each other. Bigram indexing lets Manticore pre-store adjacent token pairs such as:

noise cancelling
cancelling headphones

That gives the engine a faster way to narrow down candidate documents during phrase matching.

This article focuses specifically on phrase acceleration.

Important caveat: bigrams work at tokenization level

This is the part that is easy to miss when you only look at the happy-path speedup story.

bigram_index works at the tokenization level only. It does not account for later transformations such as morphology, wordforms, or stopwords, and that can materially change phrase-matching expectations.

The practical conclusion is simple: bigrams can be excellent for phrase speed, but if your index relies heavily on morphology, wordforms, or stopwords, test the actual phrase behavior you care about before rolling the setting out broadly.

Mode 1: Default behavior

This is the baseline. No explicit bigram indexing is enabled, so no bigram posting lists are stored.

Use it when:

phrase search is rare
documents are short
you want the leanest indexing path

Example

DROP TABLE IF EXISTS bi_none_demo;

CREATE TABLE bi_none_demo(title text);

INSERT INTO bi_none_demo VALUES
  (1,'wireless noise cancelling headphones'),
  (2,'noise cancelling microphone'),
  (3,'wireless gaming headset');

SELECT id, title FROM bi_none_demo WHERE MATCH('"noise cancelling"');

This is the baseline behavior. The query matches the expected rows, but Manticore has no precomputed bigram posting lists to help resolve the phrase more efficiently.

Mode 2: `all`

bigram_index = all

This is the most aggressive phrase-acceleration mode. Every adjacent token pair gets indexed as a bigram.

Use it when:

exact phrase search is a core feature
phrase queries often include common words and produce many candidates
you want the strongest phrase acceleration
you do not want to tune a frequent-word list

Example

DROP TABLE IF EXISTS bi_all_demo;

CREATE TABLE bi_all_demo(title text)
  bigram_index='all';

INSERT INTO bi_all_demo VALUES
  (1,'lord of the rings trilogy'),
  (2,'house of the dragon season 2'),
  (3,'made for iphone charger');

SELECT id, title FROM bi_all_demo WHERE MATCH('"house of the dragon"');
SELECT id, title FROM bi_all_demo WHERE MATCH('"made for iphone"');

The important point here is not different matches, but different indexing strategy: all stores every adjacent pair, so phrase queries have the maximum amount of bigram help available at search time.

The reason to choose all is when phrase search becomes more expensive because many documents match the individual words, and Manticore then has to do more positional verification to confirm the exact phrase. all helps by narrowing candidates earlier.

Mode 3: `first_freq`

bigram_index = first_freq
bigram_freq_words = for, of, the, with

This mode stores a pair only when the first token is in your frequent-word list.

Use it when:

phrase search matters
you want a lighter alternative to all
many phrases in your data contain words that are genuinely frequent in your own corpus

With the list above:

for iphone is eligible
of the is eligible
the dragon is eligible
made for is not eligible
lord of is not eligible

For production use, do not pick bigram_freq_words from memory. Derive it from your own data. A practical way is to dump dictionary stats with indextool using --dumpdict ... --stats, review the most frequent tokens, and then build a small bigram_freq_words list from those results.

Example

DROP TABLE IF EXISTS bi_first_freq_demo;

CREATE TABLE bi_first_freq_demo(title text)
  bigram_index='first_freq'
  bigram_freq_words='for,of,the,with';

INSERT INTO bi_first_freq_demo VALUES
  (1,'made for iphone charger'),
  (2,'lord of the rings trilogy'),
  (3,'house of the dragon season 2');

SELECT id, title FROM bi_first_freq_demo WHERE MATCH('"made for iphone"');
SELECT id, title FROM bi_first_freq_demo WHERE MATCH('"lord of the"');

The queries still return the expected rows. What changes is which pairs get indexed:

"made for iphone" benefits from for iphone
"lord of the" benefits from of the

This makes first_freq a lighter alternative to all when many useful phrases involve common bridge words.

Mode 4: `both_freq`

bigram_index = both_freq
bigram_freq_words = for, of, the, with

This is the narrowest frequency-based mode. A pair is stored only when both tokens are in the frequent-word list.

Use it when:

you want the most conservative bigram footprint
you mainly care about pairs built from words that are highly frequent in your corpus
you are tuning a large corpus and do not want to index every adjacent pair

With the same list:

of the is eligible
for iphone is not eligible
the dragon is not eligible

Example

DROP TABLE IF EXISTS bi_both_freq_demo;

CREATE TABLE bi_both_freq_demo(title text)
  bigram_index='both_freq'
  bigram_freq_words='for,of,the,with';

INSERT INTO bi_both_freq_demo VALUES
  (1,'lord of the rings trilogy'),
  (2,'house of the dragon season 2'),
  (3,'made for iphone charger');

SELECT id, title FROM bi_both_freq_demo WHERE MATCH('"lord of the"');
SELECT id, title FROM bi_both_freq_demo WHERE MATCH('"made for iphone"');

The queries still match, but the internal selectivity differs:

"lord of the" includes of the, which both_freq is willing to store
"made for iphone" includes for iphone, which first_freq would cover but both_freq would not

Which performance mode should you choose?

The benchmark in this article shows that all can deliver a strong speedup, but it is still just one benchmark on one workload.

Manticore's own documentation says that for most use cases, both_freq is the best mode. That is a sensible default because it aims for a more balanced trade-off between phrase acceleration and indexing cost.

Use the modes like this:

choose both_freq as the default starting point for general phrase-search workloads
choose all when phrase search is especially important and you want the strongest acceleration, accepting higher indexing cost
choose first_freq when many useful phrases in your data involve common bridge words and you want something broader than both_freq
choose the default behavior when phrase acceleration is not important

Benchmark: does bigram indexing really speed up phrase search?

Yes. In a simple local benchmark, the difference was easy to measure.

I used manticore-load to build two 1M-document tables against the same Manticore instance:

one with no explicit bigram_index setting
one with bigram_index='all'

The documents were random 60-80 word texts, and the benchmark repeatedly ran random 2-word phrase queries.

For clarity, both indexing and search were run with --threads=1. Multi-threaded numbers would of course be higher, but single-thread runs make it easier to see what the feature changes on one CPU core.

SELECT COUNT(*) FROM bench_bigram_* WHERE MATCH('"<text/2/2>"')

Benchmark setup

Data load without bigrams:

manticore-load \
  --drop \
  --wait \
  --threads=1 \
  --batch-size=1000 \
  --total=1000000 \
  --init="CREATE TABLE bench_bigram_none_rand(title text)" \
  --load="INSERT INTO bench_bigram_none_rand(id,title) VALUES(<increment>,'<text/60/80>')"

Data load with all bigrams:

manticore-load \
  --drop \
  --wait \
  --threads=1 \
  --batch-size=1000 \
  --total=1000000 \
  --init="CREATE TABLE bench_bigram_all_rand(title text) bigram_index='all'" \
  --load="INSERT INTO bench_bigram_all_rand(id,title) VALUES(<increment>,'<text/60/80>')"

Search benchmark without bigrams:

manticore-load \
  --threads=1 \
  --total=5000 \
  --load="SELECT COUNT(*) FROM bench_bigram_none_rand WHERE MATCH('\\\"<text/2/2>\\\"')"

Search benchmark with all bigrams:

manticore-load \
  --threads=1 \
  --total=5000 \
  --load="SELECT COUNT(*) FROM bench_bigram_all_rand WHERE MATCH('\\\"<text/2/2>\\\"')"

What I observed

On this local run:

Table	QPS	Avg latency
`bench_bigram_none_rand`	`755`	`1.3 ms`
`bench_bigram_all_rand`	`2175`	`0.4 ms`

That is roughly a 2.9x improvement in QPS and about a 3.2x improvement in average latency on the same 1M-document workload.

Indexing was slower with bigram_index='all', which is expected:

without bigrams: about 45k docs/sec
with all: about 17k docs/sec

That trade-off is exactly why multiple modes exist.

Final takeaway

If your main problem is phrase-search performance, treat bigram_index first and foremost as an acceleration feature.

For most real workloads, start with both_freq and measure. Move to all if you need a stronger effect and can afford the extra indexing cost. Consider first_freq when your phrase workload is heavily shaped by common bridge words.

Build a Searchable Catalog with Filters, Facets, and Semantic Search

Sergey Nikolaev — Wed, 06 May 2026 06:17:07 +0000

A search box is easy. A searchable catalog that keeps being useful after the first query is the harder part.

That is the problem this demo takes on. It uses a small board-game catalog, but the shape of the problem is familiar: users type something half-remembered, misspell it, narrow by constraints, keep browsing, open a result, then want "more like this" without starting over. If your product has that flow, most of the work is not the UI polish. It is getting the search behavior right without turning the stack into a science project.

In this article, we build a searchable catalog with autocomplete, typo tolerance, filters, facets, deep pagination, semantic search, and similar-item recommendations.

You can try the hosted version first:

https://catalog.manticoresearch.com

The app itself is implemented in PHP, but that is not really the story here. The interesting part is how little ceremony you need to get from a basic query box to something that already feels like a working catalog: search, filters, facets, and similar-item discovery all show up quickly.

Run it locally

To run the same demo locally, you only need PHP 8.1+, Composer, and Docker (or any other way to run Manticore).

In this setup, Manticore is the search engine behind the catalog: it handles indexing, filtering, faceting, and semantic retrieval. The repo already includes a Docker setup for it, so the quickest way to get the demo running is to clone the repo and start Manticore from the project root:

git clone https://github.com/manticoresoftware/php-catalog-demo
cd php-catalog-demo
docker compose up -d

docker compose ps should show the container as running.

Inside the cloned repo, create the app environment file:

cp app/.env.example app/.env

For a local run, the important part is just how the app reaches Manticore:

MANTICORE_HOST=127.0.0.1
MANTICORE_PORT=9308

Install dependencies:

cd app
composer install

The demo reads those settings and creates a Manticore client:

$settings = require $root . '/config/settings.php';

$client = new Client([
    'host' => $settings['manticore']['host'],
    'port' => $settings['manticore']['port'],
    'transport' => 'Http',
]);

Then load the demo dataset:

php bin/bootstrap-demo.php

That command recreates the demo table and imports the starter catalog, so you begin from a known state instead of debugging old data.

Start the app:

php -S localhost:8081 -t public

Open http://localhost:8081/ and you have a working catalog to search.

Not glamorous. Still worth it. A lot of search demos lose people before the first query because setup sprawls. This one does not need much.

What makes the app feel usable

The part I care about most is not that the demo returns results. Plenty of demos do that. It is that the search flow holds together as users get more specific.

Start with autocomplete

People usually begin with fragments. Sometimes they remember the exact game title. Often they do not.

So the first layer is autocomplete:

$payload = [
    'body' => [
        'query' => $term,
        'table' => $this->tableName,
        'options' => ['limit' => $limit, 'force_bigrams' => 1],
    ],
];
$suggestions = $this->client->autocomplete($payload);

Using force_bigrams here helps tighten typo-tolerant matching for short or slightly wrong input, which is exactly where autocomplete can otherwise get mushy.

This is a small feature, but it changes the feel of the app immediately. Users stop guessing what your catalog calls things.

Make the first results page forgiving

Once the query is submitted, the first page needs to be useful even when the spelling is off by a bit.

$search = (new Search($this->client))
    ->setTable($this->tableName)
    ->limit($limit);

if ($query !== '') {
    $search->search($query);
    if ($fuzzy) {
        $search->option('fuzzy', 1)->option('force_bigrams', 1);
    }
} else {
    $search->search('*');
}

Fuzzy mode is doing plain practical work here: recovering close matches when users do not type the title exactly right.

If you want the lower-level details, see Spell correction and fuzzy search.

Let users narrow without rewriting

This is where many search interfaces get annoying. The query is close enough, but the result set is still too broad, so now the user has to reformulate it from scratch.

Better to let them narrow in place.

Range filters handle constraints like price, player count, play time, and release year. Facets expose the shape of the current result set so users can click into categories or tags instead of thinking up a more precise sentence.

$attributeFilters = [
    'price_min' => $priceMin,
    'price_max' => $priceMax,
    'play_time_min' => $playTimeMin,
    'play_time_max' => $playTimeMax,
    'player_count_min' => $playerCountMin,
    'player_count_max' => $playerCountMax,
    'release_year_min' => $yearMin,
    'release_year_max' => $yearMax,
];

if ($categoryIds !== []) {
    $search->filter('category_id', 'in', $categoryIds);
}
if ($tagIds !== []) {
    $search->filter('tag_id', 'in', $tagIds);
}
$this->applyNumericFilters($search, $attributeFilters);
$search->facet('category_id')->facet('tag_id');

That combination matters more than it may look on paper. In practice, this is where the catalog starts feeling easy to use: a broad query can shrink fast once you click into a category or tag, without losing the original search intent.

Keep deep pagination stable

If people browse further, offset pagination starts showing its age. Data changes between requests, offsets get larger, and eventually "show more" becomes less trustworthy than it should be.

This demo uses scroll tokens instead:

// Page 1 starts a fresh scroll session; next pages continue with returned token.
$effectiveScrollToken = $page > 1 ? $scrollToken : null;
$search->option('scroll', $effectiveScrollToken ?? true);

$resultSet = new ResultSet($this->client->search(['body' => $body], true));
$nextScroll = $resultSet->getScroll();
$hasMore = $nextScroll !== null && (string) $nextScroll !== '';

That gives the app a much better foundation for deep pagination: each request continues from a returned token rather than recomputing larger and larger offsets.

Operationally, this is one of those choices users never notice when it works and definitely notice when it does not. More on the mechanism here: Scroll-Based Pagination.

Add semantic retrieval where keywords fail

Keyword search gets you far. It does not solve everything.

Sometimes users describe something in roughly the right language, but not in the same words your catalog uses. That is where hybrid search earns its keep.

Use hybrid search on the results page

In this demo, one request includes both a lexical query block and a semantic knn block, then combines them with reciprocal rank fusion via options.fusion_method = rrf:

$body = [
    'query' => ['bool' => ['must' => [['query_string' => ['query' => $query]]]]],
    'knn' => [
        'field' => 'description_vector',
        'query' => $query,
    ],
    'options' => ['fusion_method' => 'rrf'],
    'limit' => $limit,
];

The vector field uses auto-embeddings, so the app does not have to generate query vectors on its own:

'description_vector' => [
    'type' => 'float_vector',
    'options' => [
        'MODEL_NAME' => 'sentence-transformers/all-MiniLM-L6-v2',
        'FROM' => 'description',
    ],
],

Because the knn block names the vector field directly ('field' => 'description_vector'), Manticore can embed the query text automatically for KNN search.

That keeps the application logic simpler than many teams expect when they first hear "semantic search." It also lets the results page stay in one flow instead of bolting a separate semantic experience onto the side.

Use similar-item discovery on the detail page

The same vector field does a different job on the item page: "show me similar games" without forcing the user to invent another query. This part uses KNN directly against the current item.

$search = new Search($this->client);
$search->setTable($this->tableName)
    ->knn('description_vector', $source->getId(), self::SIMILAR_KNN_LIMIT)
    ->notFilter('id', 'in', [$source->getId()])
    ->limit(self::SIMILAR_RESULT_LIMIT);

$resultSet = $search->get();
$hits = $this->formatResultSet($resultSet)['hits'];
return array_slice($hits, 0, self::SIMILAR_RESULT_LIMIT);

That is where search stops being a utility and starts helping discovery. On a real detail page, this is the part that makes it easy to keep exploring instead of bouncing back to the search box.

For reference: Auto Embeddings and KNN Search.

Keeping writes and search results in sync

A demo app is easy to trust when the data never changes. Real apps do not get that luxury.

Here, the table stays in sync through the same application flow users and admins already touch: bootstrap for a clean baseline, batched imports from the admin UI, and update/delete actions for individual items.

Prepared imports use the client's batch write methods:

$table = $this->client->table($this->indexConfig['name']);

if ($appendAsNewIds) {
    $table->addDocuments($batch);
} else {
    $table->replaceDocuments($batch);
}

For individual item changes, the app uses the table API directly:

if ($id > 0) {
    $this->table->replaceDocument($document, $id);
} else {
    $this->table->addDocument($document);
}

$this->table->deleteDocument($id);

And if you want to reset the experiment, the admin UI can drop imported records and return to the baseline dataset:

$baseMaxId = $this->resolveBaseMaxId();

$this->table->deleteDocuments([
    'range' => [
        'id' => ['gt' => $baseMaxId],
    ],
]);

No extra background machinery in the demo, no detached sync story to explain away. Just writes going where they need to go.

Why this matters

What this demo shows is not just that Manticore can return results. It shows you can assemble a searchable catalog that feels complete: users can start loosely, narrow quickly with filters and facets, recover from imperfect queries, open an item, and keep discovering from there without the whole stack getting complicated.

That is already enough to make search feel like part of the product, not a bolt-on feature.

Why monitoring your search engine matters: Manticore ➡ Prometheus ➡ Grafana

Sergey Nikolaev — Thu, 09 Apr 2026 04:13:56 +0000

One of our users reached out recently with a familiar problem: search had suddenly become noticeably slower, even though nothing looked obviously broken.

The service was up, no errors in the logs, CPU usage looked normal — yet users were starting to complain that results felt sluggish.

This is how search problems usually show up in production. Not with a dramatic outage, but as a slow, creeping degradation. A little more traffic here, some extra indexing there, and before you know it, performance has slipped.

By the time users notice, the real issue has often been building for hours. Without good visibility you’re left guessing: Is the system overloaded? Is one table eating up resources? Or is something else quietly going wrong?

That’s why monitoring matters. It turns the vague “search feels slow” complaint into something you can actually diagnose and fix.

Introducing the Manticore Grafana dashboard

This is exactly what our new Manticore Grafana dashboard is built for.

Instead of raw metrics, it gives you a clean, practical view of what really matters when running search in production. At a glance you can see:

Is the node healthy?
How heavy is the current load?
Are queries slowing down?
Which tables are using the most resources?

It’s designed to help you move quickly from a user symptom to the actual root cause.

How the stack works

The setup is straightforward: Manticore → Prometheus → Grafana.

Manticore exposes rich internal metrics, Prometheus collects and stores them as time-series data, and Grafana visualizes everything with our pre-built dashboard — including 21 production-ready alerts.

You can launch the entire stack with a single Docker command:

docker run -e MANTICORE_TARGETS=localhost:9308 -p 3000:3000 manticoresearch/dashboard

(Just change the MANTICORE_TARGETS environment variable if your Manticore instance is running somewhere else.)

If you prefer to set things up manually, grab these files:

Minimal Prometheus scrape config:

scrape_configs:
  - job_name: "manticore"
    static_configs:
      - targets: ["localhost:9308"]

Exploring the dashboard

The dashboard is laid out so you can follow a natural troubleshooting flow.

1. Health summary (start here)

Open the dashboard and look at the top row first. It gives you an instant picture of the node’s overall health.

Key panels to watch:

Health / Up — Is Prometheus even able to scrape metrics?
Health / Crash indicator — Any recent crashes?
Workers Utilization % + Load / Queue pressure — These two together are gold. High utilization plus rising queue pressure is one of the clearest early signs the node is approaching saturation.

The System Score panel also gives you a quick overall health rating at a glance.

2. Query load and latency

Next, check what kind of workload the system is handling.

QPS Total shows overall traffic levels.
Search Latency (p95/p99) is one of the most important panels — averages can hide problems, but percentiles show what your users are really experiencing.
Slowest Thread helps spot expensive or stuck queries.
Work Queue Length and Worker Saturation together tell you whether the node is keeping up or starting to fall behind.

3. Memory and resources

This section is one of the most useful because memory pressure is a very common (and often hidden) cause of slowdowns in search engines. Instead of showing one vague number, the dashboard breaks it down so you can see exactly where the growth is happening.

Searchd RSS and Buddy RSS show the total resident memory — how much physical RAM the main search daemon (searchd) and the Buddy helper process are actually using right now.
The Anon RSS panels go one level deeper. “Anonymous” memory is the private, dynamic RAM allocated by Manticore itself (think heap, query caches, loaded data structures, temporary buffers — everything not backed by a file on disk). Unlike file-mapped memory (which the OS can page out or reclaim), anon memory is what usually puts real pressure on your system.

Why show both RSS and Anon RSS? Total RSS gives you the big picture, but Anon RSS tells you the story behind it. If total RSS is climbing but Anon RSS is stable, the growth might be harmless (e.g. more cached files). If Anon RSS is also rising fast, that’s usually a sign that Manticore’s own data structures or query activity are consuming more and more memory — exactly the kind of thing that leads to slower queries or even swapping.

At the bottom you’ll also see several quick counters:

Resources / FDs (searchd) — current number of open file descriptors used by the search daemon. Manticore opens a lot of files for indexes (especially large real-time tables with many disk chunks). If this number gets too high you can hit the OS limit and start seeing “Too many open files” errors. You can raise the soft limit with the max_open_files setting (see the Manticore docs on server settings).
Active workers, table counts, and non-served tables — all quick signals that something might need attention.

4. Table-level insights

Now zoom in on the data itself.

Document counts per table
Top 10 tables by RAM and disk usage
Tables / Health panel — this one is particularly valuable because it combines docs, RAM, disk, and state flags (locked/optimizing) in a single view.

5. Cluster state and history

For distributed setups you get node status and sync state. The history section is excellent for answering the most important question during any incident: what changed right before things slowed down?

Conclusion

Remember the user who reached out because search had suddenly become noticeably slower?

Once he enabled this dashboard, the problem became obvious almost immediately: workers were getting busier, queues were growing, and memory pressure was building — all before any obvious errors or crashes appeared. With clear visibility into what was actually happening inside the engine, he quickly pinpointed the root cause, made the right adjustments, and got performance back to the fast, reliable level his users expected.

The real value of monitoring isn’t just seeing pretty graphs. It’s catching those creeping issues early — before they cost you money or customers.

This dashboard removes that blind spot. It gives you the visibility you need to keep your search fast and reliable.

Monitor Manticore Search in Grafana with One Command

Sergey Nikolaev — Wed, 08 Apr 2026 02:27:24 +0000

The most annoying kind of incident is when database doesn’t go down completely - it just gets slower.

Users start noticing it right away. Complaints come in. Everything is technically still running, but clearly something is off.

And that is usually the hardest part: not noticing the problem, but figuring out what is actually happening.

When everything looks fine, but search is still slow

Let’s take a pretty normal scenario.

Search starts slowing down. It is not crashing. It is not returning obvious errors. The service is up. From the outside, nothing looks broken in a dramatic way.

But users can feel it.

So you open your monitoring:

CPU looks fine.
Average latency does not look too bad.
No obvious alerts.

At first glance, nothing really explains the slowdown.

So you keep digging...

You check the queue. Nothing jumps out immediately.
You look at worker usage. They are busy, but not in a way that tells you much on its own.
You check the logs. Still nothing obvious.

And after a while you get to that frustrating point where you realize you have already checked the usual things, and you still do not know where the problem is.

Each metric, by itself, looks more or less okay. But together, the system is clearly degrading.

So now you are no longer following a clear line of investigation. You are just checking everything you can think of and hoping the pattern shows up.

Meanwhile, time is passing.

What was actually going on

A couple of hours later, the picture finally starts to make sense.

It turns out:

the request queue has been slowly growing;
workers have been sitting near 100% utilization;
one heavy query keeps blocking execution from time to time;
p99 latency is much worse than the average suggests;
and one of the nodes restarted recently.

So the signals were there all along.

The problem was that they were scattered across different places, and it took too long to connect them into one clear story.

The solution: see the whole picture right away

Instead of spending hours piecing all of that together by hand, it is much better to have one place where the important signals are already visible.

That is why we put together a ready-to-use dashboard for Manticore Search that starts with a single Docker command. It comes with Grafana, Prometheus, a preconfigured data source, and built-in alerts.

docker run -p 3000:3000 manticoresearch/dashboard

Environment variables

The container supports two environment variables:

MANTICORE_TARGETS - comma-separated list of Manticore Search instances (default: localhost:9308)
GF_AUTH_ENABLED - set to true to enable Grafana login (by default, anonymous admin access is enabled)

Example:

docker run -p 3000:3000 \
  -e MANTICORE_TARGETS=your-host:9308 \
  manticoresearch/dashboard

If you monitor multiple nodes, pass them as a comma-separated list:

docker run -p 3000:3000 \
  -e MANTICORE_TARGETS=node1:9308,node2:9308,node3:9308 \
  manticoresearch/dashboard

If Manticore is running on a remote server

By default, the dashboard expects Manticore at localhost:9308. If your instance is running on a remote machine, the simplest option is SSH port forwarding:

ssh -L 9308:localhost:9308 user@your-server

After that, local connections to localhost:9308 will be forwarded to the remote server, so the dashboard can connect without additional changes.

A minute later, you have a usable overview of your system.

Not just a pile of graphs, but a dashboard that helps you quickly answer the questions you actually care about when something feels wrong.

You can see queue growth, worker saturation, latency, process state, and query behavior in one place, instead of bouncing between tools and trying to stitch the story together in your head.

What the dashboard shows

The value here is not that there are a lot of panels. The value is that the panels answer the right questions quickly.

The first place to look is the overall system view:

This gives you the basic picture right away:

is the service up;
has it restarted recently;
is there queue pressure;
are workers already under load.

If this row looks healthy, maybe the issue is narrow and local. If it does not, you know right away that the system is under real pressure.

Then you move to load and query behavior:

This is where you can quickly see:

whether work is starting to pile up;
whether workers are saturated;
whether latency is getting worse, especially p95 and p99;
whether one slow thread is causing a disproportionate amount of trouble.

And if you need more context, you can drill down into the rest of the dashboard:

cluster state:

tables and data:

At that point, you are no longer looking at disconnected metrics. You are looking at the system as a whole.

Why this matters

In the kind of situation that used to cost you a couple of hours just to understand, now you can usually spot the direction in a few minutes.

You can see that the queue is growing.
You can see that workers are pinned.
You can see that p99 is climbing.
You can see that one node restarted.
You can see that one query is probably doing most of the damage.

That does not mean the dashboard magically fixes the issue for you.

What it does do is remove the slowest part of the whole process: figuring out where to look.

And in practice, that is often the difference between spending two hours trying to understand the incident and spending five minutes getting to the real problem.

Parallel chunk merging in Manticore Search

Sergey Nikolaev — Tue, 07 Apr 2026 03:52:38 +0000

Starting from Manticore Search 24.4.0, RT table compaction has a more capable execution model. Instead of merging chunk pairs one-by-one in a serial flow, optimization now supports two important improvements:

disk chunk merges can run in parallel
each merge job can merge more than two chunks at once
parallel_chunk_merges: how many RT disk chunk merge jobs may run at the same time
merge_chunks_per_job: how many RT disk chunks a single job can merge in one pass

The compaction docs were also updated to describe optimization as an N-way merge handled by a background worker pool rather than a single serial merge thread.

Why this matters

For RT workloads, the interesting number is often not just how fast you can insert documents, but how long it takes until compaction catches up and the table returns to its target chunk count.

That is especially noticeable when:

you ingest data at a sustained rate
optimize_cutoff is low enough that merges kick in early
you wait for compaction to finish before considering the load fully complete

This matters most in two common cases:

you are doing an initial bulk upload into a real-time table and want the table not just searchable, but already compacted to its steady state before putting more pressure on it
you regularly ingest large batches and want each batch to finish cleanly before the next one arrives

The table is searchable before compaction finishes, but "fully searchable" and "fully optimized" are not the same thing. A higher chunk count can still matter if you care about keeping the table close to its target shape, limiting background merge work before the next ingest wave, or reducing the window where storage is busy with post-load compaction.

To show the difference, we loaded 10 million documents into an RT table. Each document contains:

id bigint
name text with generated text between 10 and 100 words
type int

The table was created with:

CREATE TABLE test(id bigint, name text, type int) optimize_cutoff='16'

So the target was to compact the table back down to roughly 16 disk chunks.

For the benchmark we used manticore-load, our load generation and benchmarking tool. It is useful for reproducing scenarios like this, stress-testing ingestion, and comparing configuration changes without building custom scripts every time.

The data was loaded with:

manticore-load \
  --cache-gen-workers=5 \
  --drop \
  --batch-size=1000 \
  --threads=5 \
  --total=10000000 \
  --init="CREATE TABLE test(id bigint, name text, type int) optimize_cutoff='16'" \
  --load="INSERT INTO test(id,name,type) VALUES(<increment>,'<text/10/100>',<int/1/100>)" \
  --wait

Before: one merge job, two chunks at a time

With the old behavior forced explicitly:

mysql -P9306 -h0 -e "set global parallel_chunk_merges=1; set global merge_chunks_per_job=2"

the run looked like this:

merging started at 14 seconds, when about 1.8M documents had been inserted
all 10M documents were loaded after 1 minute 18 seconds
at that point the data was already fully searchable
compaction kept running in the background until 3 minutes 23 seconds

At 01:18, the table still had more than 50 chunks. Near the end of loading the status looked like:

17:14:50  01:17     98%         133      128.4K   21%     5          53        1         4.22GB      9.9M
17:14:51  01:18     100%        131      310.9K   15%     1          53        1         4.27GB      10.0M
...
17:16:55  03:22     100%        0        49.4K    4%      1          17        1         4.27GB      10.0M
...
Total time:       03:23

This is the classic pattern of a healthy ingest pipeline followed by a long merge tail.

After: parallel merges plus larger merge jobs

With the new settings:

mysql -P9306 -h0 -e "set global parallel_chunk_merges=3; set global merge_chunks_per_job=5"

the same workload finished much faster:

merging again started at about 14 seconds
all 10M documents were again loaded after about 1 minute 18 seconds
full compaction finished after only 1 minute 31 seconds

The end of the run looked like this:

17:19:22  01:17     99%         127      127.9K   28%     6          26        1         4.22GB      9.9M
17:19:23  01:18     100%        132      1883.8K  17%     1          23        1         4.25GB      10.0M
...
17:19:36  01:31     100%        0        110.2K   3%      1          17        1         4.25GB      10.0M
...
Total time:       01:31

What changed in practice

The ingest phase itself stayed roughly the same:

old settings: 1:18 to load all data
new settings: 1:18 to load all data

The big gain came from post-ingest compaction:

old settings: about 2:05 of additional merge time after loading finished
new settings: about 0:13 of additional merge time after loading finished

That is roughly:

55% lower total time overall, from 3:23 down to 1:31
about 90% less merge tail after the last document was inserted

Chunk pressure during ingest was much lower too. Near the end of loading:

old settings: 53 chunks
new settings: 23 chunks

So the improvement is not just that compaction finishes sooner. It also keeps the chunk count under control much more aggressively while data is still being inserted.

What about the new defaults?

On this server, with the new default settings and no explicit tuning at all, the same workload finished in:

Total time:       01:57

That already cuts the old 03:23 result substantially, while still leaving room for additional tuning with:

parallel_chunk_merges
merge_chunks_per_job

In other words, the new defaults already improve the out-of-the-box experience, and systems with enough I/O headroom can push compaction even further by increasing both settings carefully.

Broader benchmark results: row-wise and columnar storage

The 10M-document example above shows the mechanics clearly, but the larger picture is even more interesting. In a wider test matrix we measured the combined load + optimize time for both row-wise and columnar storage across multiple values of:

parallel_chunk_merges
merge_chunks_per_job

The headline result is that, in some cases, tuning these settings can reduce total load + optimize time by:

up to 4.6x for row-wise storage
up to 6.8x for columnar storage

Here is the best-vs-worst picture from that test set:

Storage	Best settings	Best time	Slowest settings	Slowest time	Improvement
Row-wise	`parallel_chunk_merges=4`, `merge_chunks_per_job=5`	14:35	`parallel_chunk_merges=1`, `merge_chunks_per_job=2`	67:15	4.61x
Columnar	`parallel_chunk_merges=4`, `merge_chunks_per_job=5`	15:10	`parallel_chunk_merges=1`, `merge_chunks_per_job=2`	99:14	6.80x

There is also a useful tuning pattern in the full results:

the best runs for both storage modes clustered around parallel_chunk_merges=4..5
the best runs also clustered around merge_chunks_per_job=4..5
the slowest results were consistently at parallel_chunk_merges=1 with merge_chunks_per_job=2

In other words, the old serial two-chunk pattern is not just a little slower. On large workloads it can become dramatically slower, especially with columnar storage.

How to think about the two settings

The new docs describe two separate levers:

parallel_chunk_merges increases how many merge jobs can run at once
merge_chunks_per_job increases how many chunks each job can consume

Lower merge_chunks_per_job values make it easier to schedule more jobs in parallel because each job consumes fewer chunks from the available pool. If a table has many chunks waiting to be compacted, smaller jobs leave more independent chunks available for other workers, so the scheduler can keep several merges active at once. Higher values reduce the total number of merge steps, but each job becomes heavier and grabs a larger portion of the available chunks, which can leave less room for concurrent jobs.

The right balance depends on your storage and workload, but the benchmark above shows that combining both approaches can dramatically reduce the time spent waiting for RT chunk compaction to finish.

Takeaway

If your RT workloads spend too long waiting for chunk compaction after bulk inserts, the new parallel merge model changes that equation significantly.

On this 10M-document test with optimize_cutoff=16:

Mode	Searchable at	Fully optimized at
Old settings: `parallel_chunk_merges=1`, `merge_chunks_per_job=2`	1:18	3:23
New defaults	1:18	1:57
Tuned new settings: `parallel_chunk_merges=3`, `merge_chunks_per_job=5`	1:18	1:31

the time until all data became searchable stayed the same
the time until chunk compaction completed dropped from 3:23 to 1:31
even the new defaults reduced the total time to 1:57

This is exactly the kind of improvement that matters for operational RT indexing. The data is searchable as soon as it is loaded, and that point stayed about the same in both runs. The difference is what happens after that: how long the server keeps spending time compacting chunks in the background before the table returns to its target shape. If your workflow depends on the table becoming compact again before the next heavy ingest, before a maintenance window closes, or before you hand the system over to a search workload that should run with fewer chunks and less background merge pressure, the improvement is substantial.

DEV Community: Sergey Nikolaev

14 faster embeddings: how we rebuilt the ONNX path in Manticore

TL;DR

Why this matters

Why ONNX, and not Candle

The concurrency model — the part most readers will find new

Adaptive parallelism — the wrong turns we took

Numbers

How to feed it for maximum throughput

Before vs after, across the whole grid

What's next

Try it

Український лематизатор тепер вбудовано в Manticore Search

Коротко

Що таке лематизатор

Що змінилося

Мінімальна конфігурація

Перевіримо на прикладі

Що відбувається з токенами

Що варто пам'ятати

Faster KNN search in Manticore: 2-pass HNSW, batched distances, and AVX-512

Faster KNN search in Manticore

Compile-time distance function specialization

2-pass neighbor processing

Batched distance computation

AVX-512 support

Benchmark results

Algorithmic improvements alone

SIMD width benefit (AVX-512 vs AVX2)

Combined improvement (AVX-512 vs the old code)

Why the gain grows with k

What this means for you

Further reading

Manticore Search 27.1.5: Authentication, sharded tables, conversational search and faster vector search

Upgrade Notes

Highlights

Built-in authentication and authorization

Sharded tables

Conversational search

Faster vector builds and KNN improvements

Better faceting and aggregations

Other Notable Improvements

Bug Fixes

Need help or want to connect?

The Evolution of 'More Like This

Short glossary

What classic More Like This did

Where the lexical approach is still strong

What embeddings change

MLT as lookup by a vector from the index

Why search is better handled in the engine

Practical examples and the evolution of MLT

What to keep in mind

Conclusion

KNN early termination in Manticore Search

How it works

The threshold: quantile-based adaptation

Patience: how many bad rounds before stopping

Warm-up phase

Benchmark results

Performance under concurrent load

When early termination kicks in (and when it doesn't)

Syntax

When to disable it

How it relates to other KNN optimizations

How to Make xt850 Match xt 850

TL;DR

Assumptions and verification

The broader search problem

Baseline: why xt850 fails by default

Why bigrams help here

A note about bigram_delimiter

Mode 1: second_numeric

Example

Mode 2: second_has_digit

Example

How to choose between the two

Final takeaway

How to Speed Up Phrase Search with bigram_index

TL;DR

Baseline: why `xt850` fails by default

Mode 1: `second_numeric`

Mode 2: `second_has_digit`

Mode 2: `all`

Mode 3: `first_freq`

Mode 4: `both_freq`