DEV Community: Sergey Nikolaev

Closing the KNN rescoring gap between columnar and row-wise storage

Sergey Nikolaev — Mon, 05 Oct 2026 09:00:00 +0000

Originally published on the Manticore Search website on October 5, 2026

Closing the KNN rescoring gap between columnar and row-wise storage

Why default KNN rescoring makes columnar vector access slower than row-wise storage, and how Manticore closes most of the gap while preserving out-of-core search performance.

Manticore supports both row-wise and columnar attribute storage. Row-wise storage works well when the working set fits in memory, but columnar storage becomes especially useful when large datasets exceed RAM: queries that need only a few attributes can read and cache mostly the data they actually use.

There was one important performance gap between the two layouts. During KNN search, Manticore rescored HNSW candidates using their original full-precision vectors. With the previous columnar access path, that step was much slower than with row-wise storage. In our DBpedia benchmark, row-wise storage delivered 2.80x–3.53x the KNN throughput.

The problem wasn't the columnar layout itself. Columnar vectors were read through a reusable buffer, which prevented Manticore from keeping stable pointers to multiple vectors and processing them in batches.

We changed columnar access to use memory mapping. This gives vectors stable addresses while still allowing the operating system to load and evict file pages on demand. The result is 2.57x–3.13x higher KNN throughput, reaching 85–92% of row-wise performance, without sacrificing columnar storage's ability to work efficiently when the data is larger than available memory.

How slow can rescoring from columnar storage be?

We measured KNN throughput on a 16-core AMD Ryzen 9 5950X using the DBpedia dataset:

975,000 vectors
1,536 coordinates per vector
1-bit quantization
5,000 different queries per run
A working set that fits in available memory
The default oversampling and rescoring behavior

The initial comparison used row-wise and columnar vector storage:

Across the three measurements, the row-wise path delivered 2.80x - 3.53x the throughput.

Graph traversal contributes equally in both results. The throughput gap comes from the rescoring pass.

Why the defaults make this important

Rescoring and oversampling are part of Manticore's default KNN behavior:

oversampling=3.0 multiplies the requested k before HNSW search, retrieving more approximate candidates than the final query needs.
rescore=1 fetches the original 32-bit vectors for those candidates, recalculates their distances, sorts them again, and returns the final top k.

The default query path is therefore:

requested k -> retrieve up to 3 x k candidates -> rescoring -> return k results

That changes how the benchmark values should be interpreted:

Requested k	Target HNSW candidate pool	Final results after rescoring
20	60	20
100	300	100
500	1,500	500

For example, k=500 makes HNSW search with an effective k of 1,500. Those candidates become eligible for exact rescoring, and the best 500 are returned. Filtering, disk chunks, and candidate availability can affect the exact number of physical reads, while the target candidate pool remains three times the requested result count.

Oversampling and rescoring improve ranking quality, particularly with quantized vectors. Disabling them changes the normal quality/performance tradeoff. Optimizing the default path benefits typical KNN queries directly.

The graph search is the same; the vector access is different

Row-wise and columnar KNN tables use the same HNSW graph search. In both scenarios, the graph navigates the approximate index and produces candidate document IDs. Vector storage becomes relevant after that stage, when rescoring needs the original full-precision value for every candidate.

The row-wise accessor can provide a stable address for each resident vector. Manticore can retain several vector pointers, prefetch their data, and calculate multiple distances together.

Columnar access works differently. It fetches a vector through a reusable read buffer. A later read can overwrite that buffer, so the rescoring code processes one vector before fetching the next instead of retaining a group of pointers for batch processing.

This distinction grows more expensive with high-dimensional vectors and as the user increases k. The benchmark illustrates the effect at effective candidate pools of 60, 300, and 1,500, but users can request other k values and the same mechanism applies.

Since HNSW traversal is the same, this rescoring behavior accounts for the observed storage-mode gap.

Why columnar storage still matters

Columnar storage was originally intended for cases where there is not enough memory to load all the data for a queried attribute. It arranges all values of one attribute next to one another. A page read for price, for example, contains mostly more price values rather than prices interleaved with categories, timestamps, and other attributes.

Row-wise storage keeps the attributes for one document together. That is useful when a query needs the complete row, while a query that reads one attribute may also bring unrelated values into memory. For scans, filters, and aggregations over individual attributes, this lowers the useful-data density of each loaded page.

The contiguous columnar layout normally behaves better under memory pressure because it reads and caches more of the requested attribute and less unrelated data. That advantage becomes especially important when the table is much larger than RAM.

The slowdown had a simpler cause: random vector reads during rescoring had to pass through one reusable buffer.

A new access path for columnar vectors

Memory mapping provides stable access to the columnar files.

Mapping reserves virtual address space for a file. Physical pages enter RAM on demand as the process accesses them, and the operating system can reclaim those pages under memory pressure. A mapping can therefore be larger than the available physical memory.

For rescoring, pointer stability is the key change. A vector can be addressed directly in the mapped region instead of copied through a reusable read buffer. Manticore can then:

Sort candidates by disk chunk and row ID to improve locality.
Collect stable vector pointers in batches of up to 256 candidates.
Prefetch vector data before it is needed.
Calculate distances for the batch, processing pairs together where supported and handling any remainder individually.

Memory mapping enables the batched rescoring path that row-wise storage already benefited from.

Fit-in-memory result: most of the gap disappears

We repeated the DBpedia test with memory-mapped columnar access and compared all three storage paths:

At k=20, columnar throughput rose from 178 to 509 QPS, or 2.86 times the file result. At k=100, it rose from 97 to 304 QPS, a 3.13-times improvement. At k=500, it rose from 61 to 157 QPS, a 2.57-times improvement.

Columnar file access reached 28-36% of row-wise throughput. Memory-mapped columnar access reached 85-92%. Its remaining gap to row-wise storage was 15.4% at k=20, 11.1% at k=100, and 8.2% at k=500.

The narrowing gap is consistent with batching becoming more valuable as k and the resulting rescoring work increase. At the benchmark's k=500 point, memory-mapped columnar storage finished within about 8% of row-wise performance instead of delivering roughly one-third of its throughput.

That establishes the result for a resident working set. The next question is how the new path behaves in the memory-constrained conditions columnar storage was designed for.

Out-of-core result: the taxi benchmark

For generic search, we used a much larger taxi dataset on the same Ryzen 9 5950X:

1.74 billion documents
32 disk chunks
372 GB total table size
88 GB of queried .spc columnar storage files
32 GB Docker memory limit

The queried columnar footprint was 2.75 times the container's entire memory limit. The search server also needed memory for its own data structures, leaving less than 32 GB for filesystem-backed pages. This is an out-of-core workload, so queries run under page eviction and physical I/O.

We ran the same generic-search suite against two versions of the data:

taxi: the complete 1.74-billion-document table, which exceeds available memory.
taxi1: one disk chunk, whose working set fits in memory.

The 17 queries included full-text search, unfiltered aggregates, equality and range filters, indexed lookups, and high- and low-cardinality GROUP BY operations. These are the same taxi queries used in our public comparisons on db-benchmarks.com.

Each access path was tested in three complete runs. The figures below use the arithmetic mean of server-reported time across those runs, with minimum and maximum values shown by the whiskers. Each value represents the total for the entire 17-query suite.

Cold and hot measurements are analyzed separately:

Cold is the first measured execution in the benchmark's cache-drop phase.
Hot is the mean of 10 repeated executions per query on taxi and 50 on taxi1.

Cold runs

Dataset	Columnar access	Average	Min-max	Change vs file
Whole taxi table, out of core	`file`	25.086 s	24.489-25.581 s	-
Whole taxi table, out of core	`mmap`	25.155 s	24.561-25.952 s	+0.28%
Single taxi chunk, resident	`file`	845.667 ms	793-905 ms	-
Single taxi chunk, resident	`mmap`	832.667 ms	815-848 ms	-1.54%

Lower is better.

On the complete taxi table, mmap was 0.28% slower in cold runs, a negligible difference.

On the resident single chunk, mmap was 1.54% faster: 832.667 milliseconds versus 845.667 milliseconds, which is a small improvement.

Hot runs

Dataset	Columnar access	Average	Min-max	Change vs file
Whole taxi table, out of core	`file`	22.196 s	22.033-22.521 s	-
Whole taxi table, out of core	`mmap`	22.223 s	22.074-22.500 s	+0.12%
Single taxi chunk, resident	`file`	758.667 ms	754.120-765.920 ms	-
Single taxi chunk, resident	`mmap`	751.087 ms	742.660-759.540 ms	-1.00%

Lower is better.

On the out-of-core table, mmap was 0.12% slower in hot runs, a negligible difference.

On the resident chunk, mmap was 1.00% faster: 751.087 milliseconds versus 758.667 milliseconds. Grouping and range queries included improvements, while some simple aggregates moved slightly in the other direction. The mixed per-query changes and small aggregate difference again support near-parity.

Together, the cold and hot runs show that the mapped path preserves non-KNN search performance. It was neutral when the queried columnar files exceeded available memory and slightly faster when one chunk fit in memory.

Why non-KNN search remains close

The large KNN improvement comes from changing the rescoring strategy: stable vector pointers enable better locality, prefetching, and batched distance calculations. The non-KNN taxi queries do not use this batched KNN rescoring path, so they don't benefit from it.

On Linux, normal reads and file-backed mmap both go through the kernel page cache. The file mode reads cached data into an application buffer. mmap exposes the file-backed pages through the process address space and brings missing pages in through page faults. Both paths remain subject to the same physical-memory limit, page reclaim, and backing storage.

This shared foundation helps explain the non-KNN parity. mmap changes the pointer and access model enough to unlock batched KNN rescoring, while the operating system continues to manage the underlying file pages under both access modes.

Conclusion

Rescoring can be a substantial part of default KNN execution. Three-times oversampling means a query asking for 500 results uses an effective HNSW k of 1,500 before the rescoring pass. As users raise or lower k, the rescoring workload changes with it. When candidate vectors were read one at a time through columnar file access, KNN throughput was 64-72% lower than with row-wise storage, even though HNSW traversal itself was unchanged.

Stable mapped addresses allow Manticore to batch that work. On DBpedia, columnar throughput rose by 2.57-3.13 times and reached 85-92% of row-wise performance.

The memory-constrained tests show that the approach also preserves columnar storage's original strength. With 88 GB of queried columnar data under a 32 GB container limit, non-KNN search time changed by only +0.28% cold and +0.12% hot. The resident single-chunk results were slightly favorable at -1.54% cold and -1.00% hot. The overall result is a large improvement to default KNN rescoring with non-KNN search parity under both memory conditions. As a result, mmap is now the default access mode for columnar storage.

Columnar access configuration

The access_columnar_attrs option controls how columnar files are accessed. Its default value is mmap, so no per-table setting is required. The mode can also be selected explicitly when creating a table:

CREATE TABLE products (
    title TEXT,
    embedding FLOAT_VECTOR
        KNN_TYPE='hnsw'
        KNN_DIMS='1536'
) ENGINE='columnar'
  access_columnar_attrs='mmap';

The previous file mode remains available. To use it as the server-wide default, add it to the searchd section of the configuration file:

searchd {
    access_columnar_attrs = file
}

The option changes how columnar files are accessed. Their stored format and the KNN query syntax stay the same.

How search works in Kiva's SaaS, where every customer has their own database

Sergey Nikolaev — Fri, 02 Oct 2026 12:00:01 +0000

Originally published on the Manticore Search website on October 1, 2026

How search works in Kiva's SaaS, where every customer has their own database

How Kiva uses Manticore Search in a SaaS platform where every customer has their own database and schema: combining a periodically rebuilt table with an RT table, processing changes in real time, and running search without dedicated servers.

In a SaaS platform that customers configure with low-code/no-code tools, it is difficult to know in advance exactly what data users will want to search. One customer adds their own fields and forms, another configures custom business processes, and a third uses the data they find for filtering, bulk editing, or marketing campaigns.

Kiva Teknoloji faced exactly this challenge. The Turkish company has been developing cloud business applications since 2009. Its main product, KivaCRM, is built on the company's own Kiva Cloud Platform and combines CRM with tools for business process automation, reporting, and analytics. Customers can configure forms, lists, processes, reports, and dashboards themselves, so the same platform is used by companies across more than 40 industries.

That flexibility directly affects search. Every Kiva customer has their own database, and the table structure changes as they configure the application around their own processes. This means search cannot be configured once for a fixed set of fields and then left unchanged.

Manticore Search works here as a separate search layer alongside MySQL. It handles full-text search, while MySQL remains the primary database. This approach has allowed Kiva to reduce the load on its main database servers, add more flexible search capabilities, and do so with almost no increase in infrastructure costs.

Search is part of the platform, not just a search box

Kiva indexes all kinds of data that customers create in their applications. Search is available in different parts of the product, and the records it finds are used for more than just viewing.

According to Arda Beyazoglu, Lead Engineer at Kiva Teknoloji, search is used for analytics, filtering, bulk record editing, email and SMS campaigns, and other tasks.

"We use Manticore for full-text search and index all kinds of data generated by our customers."

This is not a typical catalog search with a predefined schema. Every Kiva customer has a separate database, and its structure depends on how the application is configured. As customers customize it, new fields and entities appear.

So the search layer has to adapt to each customer's data rather than to a single predefined model.

Why MySQL full-text search was not enough

Before moving to Manticore, Kiva used MySQL's built-in full-text search.

According to Arda, it was slower than Manticore and consumed resources on the main database server. That matters in a SaaS environment: the application itself needs the same CPU and memory, so search starts competing with its primary workload.

Kiva therefore moved full-text search into a separate layer instead of making MySQL handle both the main application workload and search.

MySQL remained the primary data store, while Manticore took over search.

Why Kiva moved to Manticore

Before Manticore, the team used Sphinx for some time. Kiva later moved to Manticore because of version compatibility issues and the slower pace of development of the previous solution. The team was still able to keep its existing search architecture.

A separate search table for every customer

The architecture now looks like this:

customer MySQL → search cache → periodically rebuilt table + RT table → Manticore distributed table → search → record IDs → MySQL

For every table containing customer data, Kiva generates a search cache. It then creates a separate Manticore distributed table for each customer.

It combines two local tables:

one is fully rebuilt once a week;
real-time changes go into an RT table.

This lets users search up-to-date data without Kiva having to constantly rebuild the entire search table.

When a user searches in the application, Manticore finds the matching records. The application then performs point lookups in the source MySQL tables to retrieve the rest of the data.

This avoids duplicating the full contents of the source tables in Manticore. Search only needs to return the IDs of matching records, while the application retrieves the rest of the data from MySQL.

Each system therefore has a clear role:

MySQL stores the application's source data;
Manticore handles search;
the application uses IDs to connect search results back to the source records.

No dedicated search servers required

The amount of data varies significantly between Kiva customers: the platform is used by both small businesses and large enterprises.

Arda gives the following approximate figures:

Metric	Value
Size of most tables	up to a few GB
A few large tables	30 GB and above
Full rebuild of smaller tables	a few minutes at most
Full rebuild of larger tables	about 30–60 minutes
Dedicated server for Manticore	not used by Kiva

The last point is especially notable.

Kiva does not provision dedicated servers for Manticore. The search engine runs either on an application server or on a server hosting a database replica.

"We never run Manticore on a separate server. It uses very few resources when idle and is very CPU-efficient, so at our scale the additional infrastructure cost is almost zero."

For Kiva, this means that adding a separate search layer did not turn into another cluster that has to be paid for and maintained.

Less load on MySQL, more resources for the application

The main benefit for Kiva is not just faster full-text search.

Moving the search workload out of MySQL freed up resources on the database servers. Those resources can be used for more important application data and operations, while the same infrastructure can now serve more customers.

Manticore also gave Kiva capabilities it did not have before, including more advanced search methods and support for more languages.

Next step: hybrid search

Kiva is also considering Manticore for new applications with AI features.

According to Arda, the team is considering hybrid search, which combines full-text and vector search. For Kiva, this is a natural extension of the existing architecture: Manticore already serves as the search layer for customer data, so new search methods can be added there without moving the source data out of MySQL.

For now, this is only a plan, not something already running in production. But it shows how the search layer can evolve from full-text search toward search that considers both the words in a query and their meaning.

Searching data with a changing structure

At Kiva, Manticore fits into the existing infrastructure and does not require a separate search cluster.

Every customer has their own database and a schema that changes as the application is configured. Manticore combines a periodically rebuilt table with changes from the RT table and returns the IDs of matching records. MySQL remains the primary data store.

As a result, Kiva has moved the full-text search workload off its primary database, made more efficient use of its existing servers, and turned search into a shared platform capability — even when it is impossible to know in advance what fields and entities the next customer will create.

How wine.co.za made an archive of about 145,000 records easier to search with Manticore Search

Sergey Nikolaev — Tue, 29 Sep 2026 13:00:04 +0000

Originally published on the Manticore Search website on September 29, 2026.

How wine.co.za made an archive of about 145,000 records easier to search with Manticore Search

How wine.co.za uses Manticore Search alongside MariaDB to improve search across its wine industry archive, provide better suggestions, and keep maintenance to a minimum.

For wine.co.za, search is about more than finding a bottle of wine.

The site's archive covers much of the South African wine industry: wines, wineries, contacts, international agents, news articles, and classified adverts. Visitors might be looking for a particular producer, an old article, or a wine they can describe but don't know by name.

wine.co.za used to handle these searches directly in MariaDB, using full-text indexes and wildcard queries. It worked, but visitors had to be fairly precise about what they typed.

Manticore Search made those searches more flexible and made search suggestions more useful. For Kevin Kidson, IT Director at WineNet, the other big benefit is that it needs very little attention:

“It is just so damn solid — build and forget — it has been running rock solid for years.”

An archive of the South African wine industry

Manticore powers search across all parts of wine.co.za. When Kevin shared details of the system, it indexed the following:

Dataset	Records
Wineries	907
Wines	45,833
Contacts	4,644
International agents	3,447
News articles	41,602
Classified adverts	48,482
Total in these collections	144,915

There are a few smaller databases as well. Kevin describes wine.co.za as an archive for the South African wine industry, where every record matters.

The aim was straightforward: make all this information easier to find with a search engine that was easy to set up and could work alongside the existing application.

Before Manticore: visitors had to know what to type

Kevin describes the old search as a combination of full-text indexes and wildcard queries using patterns such as xx* and xx%.

“The user had to know what they were looking for, or at least how to start spelling it.”

That was limiting for someone who knew what kind of wine they wanted but didn't have a particular name in mind. They might want to search for:

dry chenin stellenbosch

Those three words describe a wine style, a grape variety, and a region. Kevin says the ability to combine them in one search was the most important improvement Manticore brought to the site.

“It has allowed me to combine search words, so dry chenin from Stellenbosch etc — I could never do that before.”

Visitors can now describe what they want rather than having to start with a specific wine or winery name.

MariaDB remains the source of truth

The integration is simple. wine.co.za still stores and updates its data in MariaDB. Once a day, it copies the searchable data into Manticore. The website sends search queries through the .NET library.

MariaDB → daily indexing → Manticore Search → .NET application → wine.co.za search

MariaDB continues to handle the primary data, while Manticore handles search. The two work alongside each other rather than one replacing the other.

Ease of setup mattered because Kevin had little time to spare:

“It was very easy and logical to set up.”

He remembers the setup being spread over about two days because he was fitting it around other work. He estimates the total time spent installing Manticore and learning how to use it at about 24 hours.

Better suggestions as visitors type

Manticore also powers suggestions in wine.co.za's search boxes. These help visitors find what they're looking for before they've finished typing.

Kevin says Manticore is fast enough to generate suggestions as people type, and its more forgiving search makes the dropdowns easier to build. Visitors aren't tied to a particular word order: they can type dry chenin stellenbosch or stellenbosch chenin dry.

That helps people who know the exact name of a wine or producer, as well as those who only know a few details about what they want. Both can use the same search box.

In his survey response, Kevin highlighted both improvements:

“MUCH better searching across the site — and MUCH better suggestions in drop-downs etc.”

Years of reliable search

wine.co.za hasn't shared detailed performance benchmarks. Kevin says the company is small and doesn't measure much, so there are no before-and-after latency or resource-use figures to report.

For Kevin, the more important point is that Manticore has been running reliably for years without needing much of his time. He would like to explore more of what it can do, but he's already happy with how it works.

For a small team, that's a practical benefit. Time that might otherwise go into maintaining search can go into other work.

“It's doing a brilliant job & I don't have to worry about it.”

Better search without a big engineering project

wine.co.za's setup shows how a separate search engine can fit into an existing application without replacing the main database. MariaDB still stores the data. Manticore receives a copy each day, and the .NET application uses it for search.

The improvement is visible in ordinary searches. Someone looking for a dry Chenin from Stellenbosch can put those words together, try a different order, and get suggestions as they type. They don't have to know a specific wine name before they begin.

For Kevin, the value is in that combination: visitors have an easier time finding what they need, while search requires very little day-to-day attention from him.

How Bookbot powers 3.5 million saved searches and ~10% of revenue with Manticore Search

Sergey Nikolaev — Thu, 17 Sep 2026 09:00:00 +0000

Originally published on https://mnt.cr/go/wx99Cf on September 15, 2026

How Bookbot powers 3.5 million saved searches and ~10% of revenue with Manticore Search

How Bookbot uses Manticore Search to keep 1.29 million second-hand books searchable, match new arrivals against 3.5 million saved searches, and power a channel responsible for ~10% of revenue.

For a second-hand bookstore, search has to do more than find a title. It has to find a copy that actually exists.

Bookbot, known as Knihobot in the Czech Republic, takes in used books, photographs them, catalogs them, sells them, and ships them to their next readers. Two copies of the same title may have different editions, prices, conditions, and photographs. When one copy sells, the search result may need to change immediately-even if the title itself remains available.

Bookbot uses Manticore Search to keep this constantly changing inventory searchable. Its main book index contains around 5.3 million documents, including approximately 1.29 million physical copies currently in stock. Manticore also stores around 3.5 million saved-search queries for Bookbot's Watchdog feature.

According to Bookbot's internal figures, Watchdogs account for roughly 10% of revenue.

Bookbot co-founder and CTO David Gazdoš shared how the company uses Manticore not only to power customer search, but also to keep search results aligned with physical inventory, choose the right copy to show, match newly available books with waiting readers, search a 203-million-record bibliographic dataset, and support internal operations.

When every copy is different

A conventional online store can often represent a product with one record and a stock counter.

Bookbot's inventory is different. A title can have several editions, and each edition can have many individual physical copies. Those copies can differ in price and condition, and each has its own photograph and availability.

That means a Bookbot search result cannot simply say, "we have this title." It needs to represent an offer the customer can actually buy.

Imagine that the copy currently shown for a title costs €6. That copy sells, but another copy of the same title is still available for €9.

The title can remain in the results, but the offer needs to change: new price, new photograph, and potentially a different edition. If the customer filtered for books below €8, the title should disappear from that result set altogether.

This is where Manticore's real-time tables are important.

MySQL remains the source of truth for Bookbot's business data. Kafka carries events generated by orders, reservations, price changes, lost or discarded books, temporary withdrawals, and catalogue corrections. As the application processes those events, it can replace, update, or remove individual records in Manticore.

Instead of treating search as a periodically rebuilt copy of the catalogue, Bookbot can keep the searchable offer aligned with what is happening to the physical books.

“Manticore lets us update the searchable offer as the physical inventory changes. When one copy sells, we can replace or remove that record without rebuilding the whole index.”

For the customer, the benefit is simple: the price, edition, condition, photograph, and availability shown by search continue to describe something they can actually buy.

One title in the results, many physical copies underneath

Keeping the inventory fresh solves only half of the problem.

Customers normally want to browse titles, not dozens of nearly identical copies of the same book. But Bookbot still needs to preserve the differences between those copies underneath.

Manticore's grouping lets Bookbot bridge those two levels.

The application first filters the physical copies that satisfy the customer's request. It then groups the remaining records by title and uses WITHIN GROUP ORDER BY to choose which copy should represent that title in the result set.

The order matters.

Suppose a title has copies from different publishers at €6, €9, and €14. A customer asks for Publisher B, a price below €10, and no recorded damage. The result needs to be a physical copy that satisfies all three conditions at once.

Bookbot cannot take the publisher from one copy, the price from another, and the condition or photograph from a third. That would describe an offer that does not exist.

Manticore lets the filters operate on the underlying copy-level records before the matches are grouped. The representative copy can then be selected according to the requested ordering. Sorting by price may pick one copy; sorting by publication year may pick a different edition.

“Manticore gives us the copy-level precision we need while still letting us present one clean result per title. The filters apply to the real physical copies first, so the result always represents an offer we can actually sell.”

When that representative copy sells, another suitable copy can take its place.

The same principle applies to facets. Bookbot counts distinct titles rather than raw physical copies, so a title with hundreds of copies does not dominate a category simply because more of it happens to be in stock.

For browsing requests that can safely be answered at title level, Bookbot maintains a smaller precomputed index with about 3.59 million title records. When a query requires copy-level correctness-such as full-text search or filters that must all match the same physical item-the application uses the full book index instead.

This gives Bookbot both sides of the tradeoff: a faster path where title-level data is sufficient, and copy-level precision where it is not.

Turning unavailable books into future demand

The most interesting use of Manticore at Bookbot begins when there is nothing to sell.

Second-hand inventory is unpredictable. A reader may know exactly which book or edition they want while Bookbot has no suitable copy available.

Instead of making that reader come back and repeat the search every few days, Bookbot lets them create a Watchdog. It can track a title, author, genre, edition, price, or a more detailed combination of criteria and notify the reader when a matching book appears.

This reverses the normal direction of search.

With ordinary search:

one query → matching books

With a Watchdog:

one newly available book → matching saved queries

Manticore's percolate tables are built for this "search in reverse" pattern. Instead of storing documents and running a query against them, Bookbot stores the queries and submits a book to find which saved searches it matches.

At the time of Bookbot's production snapshot, Manticore held around 3.5 million Watchdog rules.

When a pricing event makes a book available, the application sends its searchable representation for percolate matching. Manticore returns the matching saved searches. Bookbot then applies its own business logic: whether a Watchdog is still active, whether the customer is eligible for a notification, and whether the match should be sent immediately or included in a digest.

Manticore handles the matching; Bookbot controls the customer experience around it.

This turns search into something more valuable than a way to browse current inventory. It becomes a way to preserve demand when supply is missing and reconnect that demand with a future copy.

According to Bookbot, Watchdogs account for around 10% of revenue.

“Manticore lets us match each newly available book against around 3.5 million saved searches. Watchdogs now account for roughly 10% of our revenue, so this is not just a search feature-it has become an important business channel for us.”

For Bookbot, this is a direct business result of being able to search in both directions: readers can search the catalogue, and newly available books can search for readers who already want them.

The same pattern can apply far beyond books-to back-in-stock alerts, classified listings, jobs, property, tickets, price alerts, or any product where new supply needs to be matched with previously expressed intent.

Search that helps readers even when they do not type perfectly

Book search also has a relevance problem: people do not always remember titles, author names, translations, or series exactly as they appear in the catalogue.

Bookbot uses a dedicated Manticore dictionary with roughly 4.52 million edition records for typo correction. It includes titles, authors, series, aliases, curated misspellings, localized fields, and German transliteration variants, while excluding identifiers such as ISBNs from correction candidates.

The dictionary proposes possible editions, but it does not decide what Bookbot can sell. The main inventory index still determines which physical copies are currently available and satisfy the user's filters.

This separation is useful because the two datasets have different jobs and different freshness requirements. The typo-correction dictionary can be refreshed on its own schedule, while the main inventory index follows live availability.

For readers, the result is more forgiving search without sacrificing the correctness of the actual offer.

One search engine across the life of a book

Customer search and Watchdogs are the clearest user-facing examples, but Bookbot uses Manticore in several other parts of the business.

Before a physical book can appear in the store, Bookbot needs to identify and catalogue it. A separate bibliographic metadata pool contains approximately 203.44 million reference records searchable by ISBN, title, and author.

Text such as titles, authors, and annotations remains full-text searchable, while structured metadata such as ISBN, publisher, publication year, language, and category IDs uses Manticore's columnar storage.

Internal search also runs on Manticore. Bookbot's backoffice indexes cover books, editions, authors, customers, orders, and accounting records; the internal book index used by pricing and lookup workflows contains about 24.7 million records.

Search extends into delivery as well. Another index contains approximately 197,000 pickup points, combining text lookup with geographic filtering and distance ordering using Manticore's geo-search capabilities.

This breadth matters because Bookbot develops its technology in-house. The team can reuse the same search engine for several very different workloads instead of treating customer search, reverse matching, large metadata lookup, internal search, and geo search as completely separate technical problems.

Manticore becomes part of the path a book follows through the business: identifying it, making it searchable, selecting the right physical copy, matching it with waiting readers, helping employees work with it, and finding a pickup point after it is sold.

“We use Manticore for storefront search, Watchdogs, a 203-million-record bibliographic dataset, internal search, and pickup-point lookup. Reusing the same search technology across these workloads keeps the architecture much simpler for our team.”

Production scale with a compact footprint

Bookbot runs four separate Manticore workloads on Kubernetes: customer search, backoffice search, Watchdogs, and the bibliographic metadata pool. They run on separate servers so each workload can be sized independently.

A production snapshot from September 5, 2026 looked like this:

Metric	Production scale
Main book index	~5.30 million documents
In-stock physical copies	~1.29 million
Precomputed title-level index	~3.59 million records
Stored Watchdog queries	~3.50 million
Bibliographic metadata pool	~203.44 million records
Instrumented listing operations	~20.2 million/day
Equivalent average operation rate	~234/sec
Successful item-selection query time	~7.7 ms mean
Successful item-selection queries within 50 ms	98.6%
Average CPU across four workloads	~6.6 cores
Average memory across four workloads	~40 GiB

The query-volume figure is a count of instrumented Manticore operations, not human searches. A single page request can issue several operations for results, item selection, and facets.

Likewise, the 7.7 ms measurement covers the application's Manticore call for successful item-selection queries; it is not complete page-load time or end-to-end inventory-update latency.

Across the four workloads, Bookbot observed no sustained CPU or memory saturation during the measured period. Customer-facing search used most of the CPU, while the 203-million-record metadata workload used less than 0.1 CPU core on average and about 18.6 GiB of memory.

The important point is not any single benchmark number. It is that Bookbot is using one search technology across live ecommerce search, reverse matching of millions of saved queries, a 203-million-record reference dataset, internal operations, and geo search-while keeping the infrastructure relatively compact.

Search that follows the life of a physical book

Bookbot's use of Manticore is unusual because search is closely tied to the lifecycle of each physical item.

A book arrives. Metadata search helps identify it. A real-time update makes the copy searchable. Filters make sure its price, edition, condition, and photograph stay together. Grouping gives the title one place in the results while selecting a real purchasable copy.

If no copy is available, a reader can leave a Watchdog behind. When another copy enters the system, Manticore can match that book against millions of saved searches and help Bookbot reconnect new supply with existing demand.

When the copy sells, search changes with the inventory and another suitable copy can take its place.

For readers, that means search that reflects what is actually available and remembers what they want when it is not.

For Bookbot, it means search contributes at several points in the customer journey-and, through Watchdogs, to a channel the company says generates roughly 10% of its revenue.

That is the value Manticore brings here: not simply finding text quickly, but giving Bookbot one search platform for fast-changing inventory, structured filters, grouped results, reverse search, large reference datasets, and geo search - and turning those capabilities into a better buying experience and measurable business value.

Better Vector Search for Long Documents: Chunking Inside Manticore Search

Sergey Nikolaev — Wed, 16 Sep 2026 09:00:05 +0000

Originally published on https://mnt.cr/go/cxcaaA on September 15, 2026

Better Vector Search for Long Documents: Chunking Inside Manticore Search

An embedding model reads only the first few hundred tokens of a document and silently drops the rest. Manticore Search now splits long documents for you at INSERT time: add chunk_strategy to the vector column and pick one of five strategies. No ingest pipeline, no splitter library. On our own manual, recall@5 for deep content went from 55% to 83%.

Say you are building search over your team's internal documentation — guides, runbooks, postmortems. You have a table with auto embeddings: you insert text, Manticore runs the model and fills the vector column for you. (If that is new to you, start with vector search in Manticore.) You load a 4,000-word document. The insert succeeds. The search works. Everything looks fine.

Except the model you picked has a 512-token input window, and that document is about 5,000 tokens long. The model read the first 380 words and threw away the other 3,600. Nothing in the document past that point can ever be retrieved, and nothing anywhere told you. The embedding may not represent the document as a whole either.

Until now, you would usually split the document into several pieces yourself, create embeddings for each one, and then work out how to combine the results if you wanted document search rather than chunk search. Manticore now handles this in the table definition: add chunk_strategy to the vector column in CREATE TABLE, and Manticore splits each document into chunks, embeds every chunk, and searches all of them:

DROP TABLE IF EXISTS docs;

CREATE TABLE docs (
  title text,
  content text,
  chunks float_vector_array knn_type='hnsw' hnsw_similarity='cosine'
    model_name='Xenova/all-MiniLM-L6-v2' from='title,content'
    chunk_strategy='sentence' max_tokens='256' overlap_tokens='32'
);

That is the whole feature. No ingest pipeline, no splitter library, no second table for chunks, no GROUP BY to fold chunk hits back into documents.

TL;DR

Five strategies: truncate (the old default), mean, fixed, recursive, sentence. Set with chunk_strategy on a model-backed vector column.
truncate and mean produce one vector per document and work on a float_vector column. fixed, recursive and sentence produce many, so they need a float_vector_array column.
A document is still one search result. Chunks compete individually, and Manticore returns the document once, with knn_dist() reporting the distance to its closest chunk. k counts documents, not chunks.
Tuning knobs: max_tokens (chunk size), overlap_tokens (shared tokens between neighbors), max_chunks (ceiling per document).
Measured on the Manticore manual (189 pages, ~298k words): for content buried past the model's window, recall@5 went from 55.1% → 83.3% and MRR from 0.44 → 0.70, at ~2.5× the RAM and ~4× the ingest time.
Queries are never chunked. A query is short enough to embed as a whole; only stored documents are split.

The problem, shown with a small example

Suppose you have four documents:

Backup and restore runbook — about 700 words, roughly 900 tokens. Backup schedules, retention, restore drills, credentials, capacity planning. The last section explains how to rotate the TLS certificate used by the replication port.
Monitoring and alerting guide — unrelated.
Getting started with the CLI — unrelated.
TLS and certificates for the HTTP API — a short page that is entirely about certificates, and never mentions rotation or replication.

You can create the table and add the documents using the commands below.

So, what we have is: one table, three vector columns with the same source text — one column per strategy. A single INSERT fills all three, so the comparison conditions are identical:

DROP TABLE IF EXISTS docs;
CREATE TABLE docs (
  title text,
  body text,
  v_truncate float_vector       knn_type='hnsw' hnsw_similarity='cosine'
    model_name='Xenova/all-MiniLM-L6-v2' from='title,body',
  v_mean     float_vector       knn_type='hnsw' hnsw_similarity='cosine'
    model_name='Xenova/all-MiniLM-L6-v2' from='title,body' chunk_strategy='mean',
  v_sentence float_vector_array knn_type='hnsw' hnsw_similarity='cosine'
    model_name='Xenova/all-MiniLM-L6-v2' from='title,body'
    chunk_strategy='sentence' max_tokens='128' overlap_tokens='32'
);

Insert the four documents

INSERT INTO docs (id, title, body) VALUES
  (1, 'Backup and restore runbook',
   'Nightly backups run at 02:00 UTC from the standby node. The job snapshots every table directory, writes a manifest, and uploads the result to object storage. Retention is thirty daily copies, twelve monthly copies, and one yearly copy. A restore drill runs on the first Monday of each month against a scratch cluster. The drill counts as passed only when a full-text search over the restored data returns the same document count as production. Anything less is treated as a failed drill and investigated the same week. Before a restore, freeze the target cluster so that no writes land while files are being replaced. Copy the manifest first and verify its checksum. If the checksum does not match, stop: a partial restore is worse than no restore, because the cluster will start and silently serve half the corpus. After the files are in place, unfreeze and let replication catch up. Watch the queue depth. If it does not drain within ten minutes, the node is probably still reading from cold storage and needs a warm-up pass before it can serve traffic. Backup failures page the on-call engineer. The three most common causes are an expired object storage credential, a disk that filled up while the snapshot was being written, and a table left frozen by a previous failed run. All three are recoverable without data loss. Check the job log first, then the disk, then the freeze state of every table. Capacity planning for backups is boring but it matters. A daily copy of the search cluster is roughly the size of the data directory plus fifteen percent for the manifest and metadata. Multiply by the retention count, add the transfer cost, and you have the monthly bill. Most teams discover too late that the yearly copies dominate the storage line. Object storage lifecycle rules do most of the retention work. Daily copies move to infrequent access after seven days and expire after thirty. Monthly copies move to archive after sixty days. Yearly copies never expire automatically; deleting one is a manual action that requires a second approver. Credentials for the backup job live in the secret manager and are issued to a role, not to a person. The role can write new objects and list the bucket. It cannot delete, and it cannot read objects older than the current day. That last restriction is the cheapest defence against a compromised backup runner turning into a data exfiltration path. Documentation for each table lives next to its schema: what the table is for, who owns it, how large it is expected to get, and whether it can be rebuilt from an upstream source. A table that can be rebuilt does not need thirty daily copies. Roughly half of most clusters turns out to be derived data that nobody had marked as derived. Verification is not the same as the job exiting zero. The job can succeed while producing an unusable copy: an empty table, a truncated upload, a manifest that references a file that was never written. The verification step reads the manifest back, checks every referenced object exists and matches its recorded size, and compares row counts on three sampled tables against production. Rotating the replication TLS certificate is a separate procedure and the step people most often get wrong. The certificate that secures the replication port is not the same as the one the HTTP API uses, and replacing one does not replace the other. Generate the new key and signing request on the node that will be rotated first, sign them with the cluster certificate authority, and place the files next to the existing ones rather than on top of them. Then update the node configuration to point at the new paths and reload. Do one node at a time and confirm that the cluster reports every peer as synced before moving on. A half-rotated cluster where two nodes trust different authorities will keep accepting writes on both sides and diverge quietly. When every node has been rotated, remove the old key material and revoke the retired certificate at the authority.'),
  (2, 'Monitoring and alerting guide',
   'Every node exports metrics over an HTTP endpoint that a scraper collects once per fifteen seconds. The dashboards are grouped into four rows: traffic, latency, saturation, and errors. Traffic is queries per second broken down by table. Latency is the ninety-fifth and ninety-ninth percentile of query time, measured server side. Alerting is deliberately thin. Paging alerts fire on sustained error rate above one percent for five minutes, on ninety-ninth percentile latency above two seconds for ten minutes, and on a node dropping out of the cluster. Everything else is a ticket, not a page. Teams that page on every anomaly stop reading pages within a month. Log retention is fourteen days hot and ninety days cold. The query log records the query text, the table, the match count, and the elapsed time. Turning it on costs a few percent of throughput and is almost always worth it, because most performance investigations start with a slow query nobody knew was being issued.'),
  (3, 'Getting started with the CLI',
   'The command line client connects over the MySQL wire protocol, so any MySQL client works and you do not need to install anything special. Point it at port 9306 and you get an interactive shell. The shell understands the usual conveniences: history, tab completion of table names, and vertical output when a row is too wide for the terminal. Start by listing tables, then look at one with SHOW CREATE TABLE. The output is the exact statement that would recreate the table, including every option that was applied implicitly, which makes it the fastest way to find out what a table actually does rather than what someone documented two years ago. Bulk loading from the shell is possible but rarely what you want. For anything above a few thousand rows, use the HTTP bulk endpoint or one of the log shipper integrations, both of which batch and retry for you.'),
  (4, 'TLS and certificates for the HTTP API',
   'The HTTP API can be served over TLS. You supply a certificate, a private key, and optionally a chain file, and the listener starts speaking HTTPS instead of HTTP. Clients that present a certificate of their own can be authenticated by it, which is the usual way to lock an internal API down without putting a password in every config file. Certificates for the HTTP API come from wherever your organisation gets certificates: a public authority, an internal authority, or an automated issuer. The file format is PEM. Both the certificate and the key must be readable by the user the server runs as, and the key must not be world readable or the listener refuses to start. Debugging TLS problems is mostly about reading the handshake. A client that reports an unknown authority is missing the chain. A client that reports a hostname mismatch is connecting by an address that is not in the certificate. A client that hangs is usually talking TLS to a plaintext port.');

Now ask a question whose answer lives in the runbook's last section, once per strategy:

SELECT title, knn_dist() FROM docs
WHERE knn(v_truncate, 4, 'how do I rotate the TLS certificate used for replication');

SELECT title, knn_dist() FROM docs
WHERE knn(v_mean, 4, 'how do I rotate the TLS certificate used for replication');

SELECT title, knn_dist() FROM docs
WHERE knn(v_sentence, 4, 'how do I rotate the TLS certificate used for replication');

Strategy	1st result	2nd result
`truncate` (default)	TLS and certificates for the HTTP API — 0.762	Backup and restore runbook — 0.936
`mean`	Backup and restore runbook — 0.656	TLS and certificates for the HTTP API — 0.762
`sentence`, 128 tokens, 32 overlap	Backup and restore runbook — 0.254	TLS and certificates for the HTTP API — 0.700

With truncate, the document that actually answers the question loses to a decoy that merely looks like it is about certificates. The runbook's single vector was built from its opening pages on backup schedules and restore drills, because that is all the model was allowed to read.

With sentence chunking, the runbook is stored as nine vectors instead of one:

SELECT id, title, LENGTH(v_sentence) AS chunks FROM docs ORDER BY id ASC;

+------+---------------------------------------+--------+
| id   | title                                 | chunks |
+------+---------------------------------------+--------+
|    1 | Backup and restore runbook            |      9 |
|    2 | Monitoring and alerting guide         |      2 |
|    3 | Getting started with the CLI          |      2 |
|    4 | TLS and certificates for the HTTP API |      2 |
+------+---------------------------------------+--------+

One of those nine is the certificate-rotation paragraph. It matches the query almost exactly, so the document wins by a wide margin: 0.254 against 0.700.

More about the chunking strategies

Strategy	Vectors per document	Column type	What it does
`truncate`	1	`float_vector`	Embeds as much as fits the model's window, drops the rest. The only mode available in older versions, and still the default.
`mean`	1	`float_vector`	Splits the whole document, embeds every piece, averages them into one vector.
`fixed`	N	`float_vector_array`	Fixed windows of `max_tokens` tokens.
`recursive`	N	`float_vector_array`	Splits on a separator hierarchy — paragraph, then line, then sentence, then space — keeping each piece within `max_tokens`.
`sentence`	N	`float_vector_array`	Sentence boundaries (Unicode UAX #29), packed up to `max_tokens`.

The important distinction is not how the text is cut. It is what a match means.

With one vector per document, search asks: "is this document, as a whole, similar to the query?" A single relevant paragraph is diluted by everything around it, and a document that covers five topics ends up not really matching any of them.

With one vector per chunk, search asks: "does this document contain something similar?" Each chunk competes on its own merits, and Manticore returns the document once, scored by its best chunk.

`truncate` — keep it when your documents are short

title text,
v float_vector knn_type='hnsw' hnsw_similarity='cosine'
  model_name='Xenova/all-MiniLM-L6-v2' from='title'

This is what you already have; chunk_strategy='truncate' is the default and you never have to write it. It's the right choice — and the fastest, and the smallest — whenever your text genuinely fits the model's window: product titles, short descriptions, tags, chat messages, log lines, search queries, commit subjects.

How much fits? More than most people assume, and less than they hope. all-MiniLM-L6-v2 takes 512 tokens, roughly 380 English words. text-embedding-3-small takes 8,192. If your 95th-percentile document is comfortably under the limit, stop reading and keep truncate.

When it hurts: anything long-form. Documentation pages, knowledge-base articles, contracts, transcripts, email threads, wiki pages, README files, incident postmortems.

`mean` — one vector, but the whole document

v float_vector knn_type='hnsw' hnsw_similarity='cosine'
  model_name='Xenova/all-MiniLM-L6-v2' from='title,content'
  chunk_strategy='mean'

Manticore splits the document, embeds every chunk, and averages the chunk vectors into a single normalized vector. Storage and search cost are identical to truncate — one vector per document, one HNSW node — but nothing is thrown away.

Use it when:

You want the tail to count but can't afford more vectors — a very large corpus where index RAM is the binding constraint.
The column is a plain float_vector and you can't change the type (for example you're adding the column to an existing table with ALTER, which multi-vector strategies don't support).
Your documents are about one thing, just long. A single product's full description, one recipe, one job posting.

Do not use it when a document covers several unrelated topics. Averaging a legal contract's indemnity clause with its payment terms produces a vector that sits between them and is close to neither. In our benchmark below, mean recovered about a third of the gap that chunking closes — a real improvement, and clearly not the same thing.

`fixed` — predictable, cheapest to reason about

v float_vector_array knn_type='hnsw' hnsw_similarity='cosine'
  model_name='Xenova/all-MiniLM-L6-v2' from='content'
  chunk_strategy='fixed' max_tokens='256' overlap_tokens='32'

Cut every max_tokens tokens, no matter what the text is doing at that point. Chunk count is a straight function of document length, so index size is predictable before you load anything.

Use it when the text has no reliable structure to exploit: OCR output, scraped HTML that lost its paragraphs, machine transcripts without punctuation, log dumps, minified content. Also a fine default when you simply want the cheapest thing that stops truncation.

The cost: a boundary can land mid-sentence, and a chunk that begins in the middle of a thought embeds badly. That is exactly what overlap_tokens is for — see below.

`recursive` — the best general default for prose

v float_vector_array knn_type='hnsw' hnsw_similarity='cosine'
  model_name='Xenova/all-MiniLM-L6-v2' from='title,content'
  chunk_strategy='recursive' max_tokens='256' overlap_tokens='32'

Same token budget as fixed, but each cut is pulled back to the nearest natural boundary: a blank line first, then a line break, then a sentence end, then a space. A chunk stops where the text stops, not where the counter runs out. The boundary is never dragged back past the midpoint of the chunk, so you don't get a stream of tiny fragments.

If you have used LangChain's RecursiveCharacterTextSplitter, this is the same idea, except it runs inside the database on the model's real tokens instead of characters, and there is nothing to install.

Use it for: Markdown and HTML documentation, wiki pages, knowledge bases, blog posts, README files, structured reports — anything written by a human in paragraphs. This scored highest on deep content in our benchmark.

`sentence` — when a chunk must be a complete thought

v float_vector_array knn_type='hnsw' hnsw_similarity='cosine'
  model_name='Xenova/all-MiniLM-L6-v2' from='content'
  chunk_strategy='sentence' max_tokens='256' overlap_tokens='32'

Detects sentence boundaries with Unicode UAX #29, then greedily packs whole sentences until the token budget is reached. A chunk never starts or ends mid-sentence. A single sentence longer than the budget is split by the token window, as a last resort.

Use it for: support tickets and email threads, chat and meeting transcripts, legal and policy text, news, customer reviews, medical and scientific abstracts — anything where a fragment of a sentence changes or destroys the meaning. It is also the strategy to pick when chunks will be fed to an LLM afterwards, because a chunk that ends mid-clause reads badly in a prompt.

sentence is a little more conservative than recursive: it produced fewer, cleaner chunks in our tests and scored about the same on recall@5.

The three knobs

chunk_strategy  = truncate | mean | fixed | recursive | sentence
max_tokens      = chunk size in tokens; 0 (default) = the model's own limit
overlap_tokens  = tokens shared between consecutive chunks; needs a non-zero max_tokens
max_chunks      = ceiling on vectors per document; 0 (default) = unlimited

max_tokens is capped at what the model can actually accept — ask for 4,096 on a 512-token model and you still get 512, not an error. Smaller chunks mean sharper matches and more vectors; larger chunks mean more context per vector and fewer of them. For English prose, 128–512 covers almost every use case; we used 256 throughout the benchmark.

overlap_tokens repeats the tail of each chunk at the head of the next, so a sentence that straddles a boundary still appears intact somewhere. 10–20% of max_tokens is the usual setting. Manticore guarantees forward progress: fixed and recursive cap the overlap at half the chunk size, and sentence re-seeds the next chunk with at most overlap_tokens worth of trailing whole sentences while always advancing by at least one sentence. It requires an explicit non-zero max_tokens — overlap against "whatever the model's limit happens to be" isn't a meaningful setting, so Manticore rejects it.

max_chunks limits the impact of unusually large documents. Without it, a 400-page PDF pasted into one row becomes thousands of HNSW nodes. With it, Manticore merges the overflow into the last kept chunk, then truncates it to the model's window when embedding:

-- a ~600-token document, chunked at 64 tokens
chunk_strategy='fixed' max_tokens='64'                  -- 22 vectors
chunk_strategy='fixed' max_tokens='64' max_chunks='3'   --  3 vectors

Use it as a guard rail against outliers, not as a way to save memory across the board.

What search looks like

Nothing about your query changes from before. There is no chunk table, no nested field, no join, no GROUP BY. Here is the complete example:

DROP TABLE IF EXISTS notes;
CREATE TABLE notes (
  title text,
  body text,
  chunks float_vector_array knn_type='hnsw' hnsw_similarity='cosine'
    model_name='Xenova/all-MiniLM-L6-v2' from='title,body'
    chunk_strategy='sentence' max_tokens='32'
);

INSERT INTO notes (id, title, body) VALUES
  (1, 'Certificate rotation',
   'The replication certificate is not the one the HTTP API uses. Generate the new key on the node being rotated and sign it with the cluster authority. Update the paths and reload, one node at a time, confirming every peer reports as synced before you move on.'),
  (2, 'Disk pressure',
   'When a data directory crosses eighty percent the merge scheduler stops compacting and the node starts refusing writes. Free space first, then trigger a manual OPTIMIZE. Adding a disk without draining the queue only postpones the problem.'),
  (3, 'Slow queries',
   'Turn the query log on before guessing. Most investigations end at a single query nobody knew was being issued, usually one that sorts on an unindexed attribute over the whole table.');

SELECT id, title, knn_dist() FROM notes
WHERE knn(chunks, 3, 'how do I replace an expiring certificate on every node');

+------+----------------------+------------+
| id   | title                | knn_dist() |
+------+----------------------+------------+
|    1 | Certificate rotation | 0.51039070 |
|    2 | Disk pressure        | 0.91703475 |
|    3 | Slow queries         | 1.02606630 |
+------+----------------------+------------+

max_tokens='32' is small on purpose here, so that these short notes actually split and you can see the multi-vector behaviour on a toy dataset. LENGTH() on the vector column shows how each document was divided:

SELECT id, title, LENGTH(chunks) AS n FROM notes ORDER BY n DESC LIMIT 5;

+------+----------------------+------+
| id   | title                | n    |
+------+----------------------+------+
|    1 | Certificate rotation |    2 |
|    2 | Disk pressure        |    2 |
|    3 | Slow queries         |    2 |
+------+----------------------+------+

Six vectors, three rows back. Search follows the rules described in Multiple vectors per document:

A document matches if any of its vectors is near the query vector.
Manticore returns each match exactly once. knn_dist() is the distance to its closest chunk.
k counts documents, not vectors. knn(chunks, 3, ...) means three documents.
A document with no vectors is never returned.

The same query over HTTP:

POST /search
{
  "table": "notes",
  "knn": {
    "field": "chunks",
    "query": "how do I replace an expiring certificate on every node",
    "k": 3
  },
  "_source": ["title"]
}

...
{
  "_id": 1,
  "_score": 1,
  "_knn_dist": 0.51039070,
  "_source": { "title": "Certificate rotation" }
}
...

Everything else on the KNN page keeps working as before: filtering, prefilter and postfilter strategies, quantization, early termination, and rescoring.

Does it actually help? Numbers on our own manual

We tested the feature on the Manticore English manual — 189 pages and about 298,000 words, ranging from a two-paragraph note to a 39,000-word changelog.

The query set is generated mechanically, not hand-picked. For every page we took its section headings, kept only headings that are unique across the whole manual, and split them in two:

Deep-content queries (419) — headings that appear after the first ~1,200 characters of their page. That is a deliberately conservative line: the model's window is 512 tokens, roughly 2,000 characters, so a few of these still point at text truncate can partly see. The gap below is therefore an understatement, not an exaggeration.
Head-content queries (88) — headings inside the first ~1,200 characters. The control group: content truncate can already see.

A query is a hit if KNN returns the page the heading came from, within the top k. Model: Xenova/all-MiniLM-L6-v2 (384 dims, 512-token window) running on Manticore's ONNX backend. Hardware: 32 threads. max_tokens='256', overlap_tokens='32' for the multi-vector strategies. Quality numbers are deterministic for a given index; the timings are a single run per strategy on an otherwise idle box.

Deep content — what chunking is for

Strategy	Vectors	Ingest	Index RAM	hit@1	hit@5	hit@10	MRR
`truncate`	189	21 s	4.2 MB	33.7%	55.1%	63.2%	0.44
`mean`	189	72 s	4.2 MB	43.9%	65.2%	74.7%	0.54
`fixed`	3,430	73 s	9.5 MB	56.3%	81.1%	86.2%	0.68
`recursive`	4,664	86 s	11.7 MB	58.7%	83.3%	89.5%	0.70
`sentence`	4,041	79 s	10.6 MB	55.4%	83.5%	89.0%	0.68

Chunking turns a coin flip into a working search. recall@5 goes from 55.1% to 83.3%, and the rank of the right answer improves just as much — MRR 0.44 → 0.70. Of the queries truncate could not answer in the top 5 at all, recursive recovers roughly two thirds.

mean lands where you would expect: it recovers about a third of the gap for free, because it costs exactly nothing extra to store or search.

Head content — the control group

Strategy	hit@1	hit@5	MRR
`truncate`	65.9%	86.4%	0.74
`mean`	59.1%	83.0%	0.69
`fixed`	60.2%	83.0%	0.71
`recursive`	58.0%	86.4%	0.70
`sentence`	56.8%	85.2%	0.69

For completeness, the control group is worth reading carefully. For content that the model could already see, truncate is still the most precise at rank 1 — 65.9% against 58.0% for recursive. A whole-document vector carries the page's overall topic, and when the query is about the page's opening subject, that context helps.

By rank 5 the difference is gone: recursive matches truncate exactly at 86.4%. So the trade is a few points of top-1 precision on content near the beginning, in exchange for +28 points of recall on everything else. For a documentation search, a help center, or any RAG retriever that feeds 5–10 passages to an LLM, that is not a close call.

Cost

Index RAM: 4.2 MB → 11.7 MB, about 2.5×, for ~25× as many vectors. Vectors are only part of what an RT table stores. The HNSW graph over those vectors also takes longer to build during chunk saves and OPTIMIZE, though Manticore builds it across all your cores.
Data load: 21 s → 86 s for 189 documents. Chunking means embedding the whole corpus instead of the first 380 words of each document, and the time scales with it. This is embedding cost, not chunking cost — the splitting itself is not measurable next to inference.
Query response time: 6.3 ms → 8.5 ms at p50. HNSW handles 4,664 vectors about as easily as 189 — see 2-pass HNSW, batched distances and AVX-512 for what carries that.

If you use a paid embedding API, read that ingest number as a bill: chunking sends your whole corpus to the model instead of the head of each document, and you pay for every token of it. Local ONNX models have no per-token cost, which is a large part of why we made them fast.

Recommendations for choosing a chunking strategy

Your data	Start with
Titles, names, short descriptions, tags, log lines	`truncate`
Long but single-topic; or RAM is the hard limit; or the column is an existing `float_vector`	`mean`
Documentation, wikis, knowledge bases, articles, READMEs	`recursive`, `max_tokens` 128–256
Support tickets, email, transcripts, legal text, reviews	`sentence`, `max_tokens` 128–256
OCR, scraped HTML, machine transcripts, unstructured dumps	`fixed`, `max_tokens` 256, plus overlap
Chunks will be passed to an LLM as context	`sentence`, `max_tokens` 384–512

Overlap is deliberately absent from most of those: our sweep below could not measure a benefit from it on structured prose, and it costs vectors. Add it when a thought routinely straddles a boundary — unstructured transcripts, OCR, long narrative without paragraph breaks.

How big should a chunk be?

Chunk size is the setting that actually affects your results. The trade is direct: a smaller chunk is a sharper match on one idea, a larger chunk carries more context but dilutes each idea inside it. A paragraph buried in a long document only becomes findable once the chunk size is small enough to give it a vector of its own.

We ran another test: recursive on the same 189-page manual, three chunk sizes × three overlap settings, and the same 419 deep queries. Two trends stand out: quality rises as chunks get smaller, while overlap adds cost without improving quality much.

Note that the Y axis starts at 78%, not zero — the whole spread is about six points, so a zero-based axis would flatten it into a straight line. The numbers behind the chart:

`max_tokens`	`overlap_tokens`	Vectors	Index RAM	deep hit@5	deep MRR
128	0	8,256	17.3 MB	85.2%	0.718
128	13	9,328	18.7 MB	85.7%	0.705
128	32	11,623	22.6 MB	85.2%	0.694
256	0	3,984	10.0 MB	83.1%	0.657
256	26	4,525	10.9 MB	84.5%	0.681
256	64	5,569	12.5 MB	83.5%	0.689
512	0	1,973	6.9 MB	80.2%	0.655
512	51	2,191	7.2 MB	79.2%	0.651
512	128	2,666	8.0 MB	79.7%	0.660

What we see:

Smaller chunks win, consistently. Going from 512 to 128 tokens buys about five points of recall@5 (80.2% → 85.2%) and a large jump in ranking quality (MRR 0.655 → 0.718). It costs 4× the vectors and 2.5× the index RAM. Below 128 the chunks stop containing a whole thought, so this is not a slope you ride forever — but on long technical prose, 128–256 beat 512 every time.

Overlap did essentially nothing for quality, and was not free. At 128 tokens, going from no overlap to 25% overlap moved recall@5 from 85.2% to 85.2% while adding 41% more vectors and 5 MB of RAM. The pattern holds at every size: the spread across overlap settings (±1.5 points) is within the noise of a 419-query set, while the cost is not. This lines up with Chroma's chunking evaluation, where plain recursive splitting at 200 tokens with no overlap scored 88.1% recall — within a few points of an LLM-driven splitter at 91.9% — and it is the opposite of the "always use 10–20% overlap" advice you will read in most RAG guides.

The honest caveat: this is one corpus, one model, and queries that look like section headings. Overlap earns its keep when a single fact routinely straddles a boundary — long unbroken narrative, transcripts without structure — and recursive already snaps cuts to paragraph and sentence boundaries, which does much of the same work. So treat "start at 128–256 with no overlap, add overlap only if you can measure it helping" as the default, and check it on your own data with the recipe below.

Comparing settings

You do not have to guess, and you do not need two tables. A table can carry several model-backed vector columns, each with its own strategy, all filled from the same fields on the same INSERT:

CREATE TABLE ab (
  title text,
  body text,
  sent_256 float_vector_array knn_type='hnsw' hnsw_similarity='cosine'
    model_name='Xenova/all-MiniLM-L6-v2' from='title,body'
    chunk_strategy='sentence' max_tokens='256' overlap_tokens='32',
  rec_128 float_vector_array knn_type='hnsw' hnsw_similarity='cosine'
    model_name='Xenova/all-MiniLM-L6-v2' from='title,body'
    chunk_strategy='recursive' max_tokens='128' overlap_tokens='16'
);

Load your corpus once, then run the same query against each column and compare. For example:

SELECT id, LENGTH(sent_256) AS sent_chunks, LENGTH(rec_128) AS rec_chunks FROM ab;
SELECT id, knn_dist() FROM ab WHERE knn(sent_256, 5, 'how do I rotate the replication certificate');
SELECT id, knn_dist() FROM ab WHERE knn(rec_128,  5, 'how do I rotate the replication certificate');

On a short runbook whose certificate section sits at the end, sentence/256 fits the whole document in a single chunk and answers at distance 0.515; recursive/128 splits it in two, isolates the certificate paragraph, and answers at 0.310. Same row, same model, same query — only the chunk size differs.

Build a dataset of real queries with answers you trust — even 50 is enough — and compare recall@5 across two or three columns, exactly as we did on the manual above. Then drop the losing column with ALTER TABLE ... DROP COLUMN and keep the winner.

Recipes

Documentation and help center search. Long Markdown pages, users asking questions in their own words. Chunk on structure and search across it:

CREATE TABLE docs (
  url string,
  title text,
  body text,
  chunks float_vector_array knn_type='hnsw' hnsw_similarity='cosine'
    model_name='Xenova/all-MiniLM-L6-v2' from='title,body'
    chunk_strategy='recursive' max_tokens='192'
);

Note from='title,body': the fields are joined before chunking, so the page title lands in the first chunk and gives it context. For a worked end-to-end example of this shape, see Vector search on GitHub.

Support tickets and email threads. A thread is a sequence of complete messages; cutting one mid-sentence loses the fact you need. Keep the chunk count bounded, because threads have no natural length limit:

CREATE TABLE tickets (
  ticket_id bigint,
  customer string,
  status string,
  thread text,
  chunks float_vector_array knn_type='hnsw' hnsw_similarity='cosine'
    model_name='Xenova/all-MiniLM-L6-v2' from='thread'
    chunk_strategy='sentence' max_tokens='256' overlap_tokens='32' max_chunks='64'
);

SELECT ticket_id, knn_dist() FROM tickets
WHERE knn(chunks, 10, 'customer was charged twice after upgrading')
  AND status = 'closed';

Filtering works exactly as it does for a single-vector column.

Contracts and policy documents. Clause-level retrieval is the entire point — nobody wants "the contract" back, they want the indemnity clause. Smaller chunks, generous overlap:

chunks float_vector_array knn_type='hnsw' hnsw_similarity='cosine'
  model_name='Xenova/all-MiniLM-L6-v2' from='body'
  chunk_strategy='sentence' max_tokens='128' overlap_tokens='32'

Product catalog with long descriptions. One product is one topic, and catalogs are large, so pay nothing extra:

embedding float_vector knn_type='hnsw' hnsw_similarity='cosine'
  model_name='Xenova/all-MiniLM-L6-v2' from='name,description'
  chunk_strategy='mean'

RAG: retrieval for an LLM. Whatever you retrieve gets pasted into a prompt, so chunks should read as prose — this is the retrieval half of conversational search. Larger chunks, sentence boundaries, and ask for more of them:

chunks float_vector_array knn_type='hnsw' hnsw_similarity='cosine'
  model_name='Xenova/all-MiniLM-L6-v2' from='title,body'
  chunk_strategy='sentence' max_tokens='512' overlap_tokens='64'

Adding chunking to a table you already have. Multi-vector columns can't be added by ALTER — existing rows have no vectors and there's no way to backfill them yet. A single-vector strategy can:

ALTER TABLE docs ADD COLUMN v2 float_vector knn_type='hnsw' hnsw_similarity='cosine'
  model_name='Xenova/all-MiniLM-L6-v2' from='title,body' chunk_strategy='mean';

ALTER TABLE docs REBUILD EMBEDDINGS v2;

For a multi-vector column, create the new table with the column in place and reindex into it.

How other engines handle this

Every vector engine now generates embeddings for you. Far fewer will split your document before doing it — and of those, most make you assemble it out of pipeline stages.

Engine	Embeds in-engine	Chunks in-engine	Strategies	One row per document at search time
Manticore Search	Yes — local + OpenAI / Voyage / Jina	Yes — `chunk_strategy` on the vector column	truncate, mean, fixed, recursive, sentence	Yes, native
Elasticsearch	Yes — inference endpoints	Yes	sentence (default), word, recursive (9.1+), none	Yes — `semantic_text` hides the chunks
OpenSearch	Yes — ML Commons	Yes — separate ingest processor	fixed_token_length, fixed_char_length, delimiter	Needs a nested field + nested query
Vespa	Yes — built-in embedders	Yes — indexing expression	fixed-length, sentence, custom	Yes
Azure AI Search	Yes — integrated vectorization	Yes — Split skill in a skillset	pages (chars), sentences	No — one row per chunk
PostgreSQL + pgai	Yes — background worker	Yes	character, recursive character	No — separate table, join and dedupe
Milvus / Zilliz	Yes — Function (2.6+)	No — app-side	—	—
Qdrant	Yes — Cloud Inference	No — app-side	—	—
Weaviate	Yes — vectorizer modules	No — app-side	—	—
Meilisearch	Yes — embedders	No — app-side	—	—
Typesense	Yes	No — open request	—	—
Apache Solr	Yes — LLM module (9.8+)	No	—	—
Pinecone	Yes — integrated inference	No — app-side	—	—
MongoDB Atlas	Yes — Automated Embedding	No — app-side	—	—

Every cell in the chunking column links to a source. A “yes” links to the feature's own documentation. An “app-side” links to that vendor's own guidance on chunking in your application — which is what they publish instead of an in-engine option. If we missed a feature, or one has shipped since, tell us and we will fix it.

Versions checked - 4 September 2026

The latest stable release of each product available that day:

Product	Version
Elasticsearch	9.5.3
OpenSearch	3.8.0
Vespa	8.750.13
Azure AI Search	REST API 2026-04-01
PostgreSQL + pgai	extension 0.11.2
Milvus / Zilliz	2.6.23
Qdrant	1.19.0
Weaviate	1.38.13
Meilisearch	1.53.1
Typesense	30.2
Apache Solr	10.0.0
Pinecone, MongoDB Atlas	hosted services, no version to pin

Two things stand out.

Chunking in-engine is still rare. Milvus, Qdrant, Weaviate, Pinecone, MongoDB Atlas, Typesense, Meilisearch and — since the 9.8 LLM module — Apache Solr will all run the embedding model for you, and every one of them will happily truncate your 4,000-word document without saying so. The splitting is your problem, in your application, in a language and a library that has no idea what tokenizer the model uses.

Where chunking exists, the plumbing usually leaks. OpenSearch gets you there with a text_chunking processor feeding a text_embedding processor writing into a nested field, queried with a nested query and a score mode. Azure AI Search wants a skillset with a Split skill, an embedding skill and index projections — and returns one result row per chunk, so grouping back to documents is on you. pgai Vectorizer writes chunks to a second table, so every query is a join plus a DISTINCT ON. Elasticsearch's semantic_text is genuinely close to Manticore's model: chunking settings on the inference endpoint, chunks hidden inside the field, one hit per document.

Manticore does the same thing with less surface area: the strategy is an option on the column, the chunks are the column's value, and search returns documents. If you are weighing the whole stack rather than this one feature, we have written up the comparison with Elasticsearch and with Turbopuffer too.

What chunking does not fix

Chunking solves one problem well — a document longer than the model's window is no longer half-invisible. It does not make retrieval perfect, and two known gaps are worth naming.

A chunk does not know where it came from. Split a document and you get a paragraph that says "do one node at a time and confirm every peer reports as synced" with no indication of what is being rotated, or which product it belongs to. Anthropic's contextual retrieval work put numbers on this: prepending a short, chunk-specific description of the surrounding document before embedding cut top-20 retrieval failures by 35%, and by 49% combined with a contextual BM25 index.

Manticore does not do this for you. FROM joins its fields with a space before chunking, so listing title first puts the title at the head of the text that gets split — which means it lands in the first chunk and only that one. Every chunk after it is on its own:

-- 'title' leads, so its words are in chunk 1; chunks 2..N never see them
from='title,body'

If you need every chunk to carry context, you have to build it into the stored text yourself before inserting — for example by repeating a short heading at the start of each section of body. There is no per-chunk prefix option today.

Chunk boundaries are decided before the model sees the text. Manticore splits, then embeds each piece independently — the standard approach, and what every engine with in-engine chunking in the table above does. An alternative called late chunking inverts it: run a long-context model over the whole document first, then pool the token embeddings into chunks, so each chunk vector carries context from the rest of the document. It needs a long-context model and more compute per document, and Manticore does not do it today. If your documents depend heavily on cross-paragraph context, it is worth knowing the option exists.

Neither gap changes the basic result: for long documents, chunked retrieval beats truncated retrieval by a wide margin, and reaching it means adding chunk_strategy to one column.

Limits and gotchas

max_chunks discards text. Manticore merges overflow into the last kept chunk, then truncates it to the model's window. Nothing warns you. It's a guard rail for outliers, not a way to save memory across the board.

Remote models chunk by bytes, not tokens. OpenAI, Voyage and Jina have no local tokenizer, so Manticore falls back to a deliberately conservative estimate of 3 bytes per token — a chunk lands under the provider's cap rather than over it. In practice max_tokens='N' becomes an N × 3-byte window. We measured it against a stub endpoint with a 3,599-byte document and the fixed strategy:

`max_tokens`	Byte window	Chunks produced
100	300	12
200	600	6
400	1,200	3

English prose runs closer to 4 bytes per token, so on a remote model you get chunks roughly a quarter smaller than the number you asked for — set max_tokens about 30% higher than you would for a local model to land in the same place. If exact boundaries matter, use a local model, where splitting is done on the model's real tokens.

Multi-vector columns can't be added with ALTER. ALTER TABLE ... ADD COLUMN and ALTER TABLE ... REBUILD EMBEDDINGS on a model-backed float_vector_array are rejected. Recreate the table instead. Both work normally on a float_vector, including with mean.

Chunking applies to auto embeddings only. Vectors you insert yourself are stored exactly as given — Manticore never re-cuts data you supplied. chunk_strategy without model_name is a DDL error, on purpose.

embeddings is a reserved word. EMBEDDINGS is a DDL keyword (ALTER TABLE ... REBUILD EMBEDDINGS), so a column literally named embeddings is a syntax error unless escaped. Use escaping if you need that name.

Queries are not chunked. A query is embedded whole, as a single vector. That is what you want: chunking exists to make a long document findable, not to split a fifteen-word question.

The DDL tells you when a combination is wrong, at CREATE TABLE time rather than at the first insert:

mysql> CREATE TABLE t (title text, v float_vector ... chunk_strategy='sentence');
ERROR 1064: chunk_strategy='sentence' produces several vectors per document
            and requires a float_vector_array attribute

mysql> ... chunk_strategy='fixed' overlap_tokens='32');
ERROR 1064: overlap_tokens requires an explicit non-zero max_tokens

mysql> ... chunk_strategy='paragraph');
ERROR 1064: unknown chunk_strategy 'paragraph'; expected truncate, mean, fixed,
            recursive or sentence

mysql> ... chunk_strategy='truncate' max_tokens='128');
ERROR 1064: chunk_strategy='truncate' ignores max_tokens, overlap_tokens and max_chunks

Try it

The shortest path to a working chunked semantic search:

DROP TABLE IF EXISTS docs;

CREATE TABLE docs (
  title text,
  content text,
  chunks float_vector_array knn_type='hnsw' hnsw_similarity='cosine'
    model_name='Xenova/all-MiniLM-L6-v2' from='title,content'
    chunk_strategy='recursive' max_tokens='256' overlap_tokens='32'
);

INSERT INTO docs (id, title, content) VALUES
  (1, 'Backup and restore runbook', 'Nightly backups run at 02:00 UTC ... ');

SELECT id, title, knn_dist() FROM docs
WHERE knn(chunks, 5, 'how do I rotate the replication certificate');

No model to download by hand, no splitter to pick, no pipeline to maintain. One column option, and the parts of your documents that used to be invisible start showing up in results.

Full reference: Chunking strategies and Multiple vectors per document in the manual. Questions and bug reports on GitHub.

Manticore Search 29.9.0: Chunked auto-embeddings and mmap columnar access

Sergey Nikolaev — Fri, 11 Sep 2026 16:05:04 +0000

Originally published on https://mnt.cr/go/c72RPs on September 11, 2026

Manticore Search 29.9.0: Chunked auto-embeddings and mmap columnar access

Manticore Search 29.9.0 adds chunked and multi-vector auto-embeddings, embedding input limits, UTF-8 identifiers, mmap columnar access by default, and fixes for hybrid search, KNN, bulk ingestion, and grouping.

[Manticore Search 29.9.0](https://mnt.cr/go/LfXmU1 has been released. The largest change is in auto-embeddings: long documents can now be split into searchable chunks, and a document can keep several vectors instead of compressing all of its content into one. This release also makes mmap the default access mode for columnar attributes, adds safer controls for embedding workloads, UTF-8 identifiers, better backup support, and fixes across hybrid search, KNN, bulk ingestion, grouped search, and schema changes.

This post covers everything shipped after 29.0.2, from 29.0.3 through 29.9.0.

❤️ We’d like to thank [@tudorvasinca](https://mnt.cr/go/Vn5RXy for their work on [PR #4857](https://mnt.cr/go/DdBMb7, [PR #4859](https://mnt.cr/go/jZePzd, and [PR #4873](https://mnt.cr/go/eQNGL6

Upgrade notes

There is no release-wide mandatory data migration. Existing tables and configuration can be upgraded normally, but a few fixes need follow-up if the earlier behavior already affected your data:

German sharp-s normalization is opt-in. Switching an existing table to lemmatize_de_v2 or lemmatize_de_v2_all changes indexed terms, so rebuild a plain table or replay documents into a new RT table.
Authenticated BACKUP introduces the backup authorization action. Before downgrading to 29.3.12 or earlier, remove any backup grants.
Columnar attributes now use mmap by default instead of buffered file reads. Set access_columnar_attrs='file' explicitly if you need to retain the previous access mode.

Long documents can keep more than one embedding

The old auto-embedding path produced one vector per document and truncated text that did not fit the model input window. That works for titles and short descriptions, but it means a relevant paragraph near the end of a long article may never reach the index.

Manticore Search now supports five [chunking strategies](https://mnt.cr/go/BXIJ8c

truncate keeps the previous behavior and remains the default.
mean embeds every chunk and averages the results into one vector.
fixed splits text into fixed-size token windows.
recursive prefers paragraph, line, sentence, and space boundaries in that order.
sentence groups complete sentences up to the configured limit.

The multi-vector strategies use [float_vector_array](https://mnt.cr/go/Y31JKu Each chunk competes independently during KNN search, but Manticore returns the document once and uses its closest chunk for knn_dist():

CREATE TABLE articles (
  title text,
  content text,
  chunks float_vector_array knn_type='hnsw' hnsw_similarity='cosine'
    model_name='Xenova/all-MiniLM-L6-v2' from='title,content'
    chunk_strategy='sentence' max_tokens='256' overlap_tokens='32'
);

INSERT INTO articles (id, title, content)
VALUES (1, 'Rotating certificates', 'A long guide with many sections ...');

SELECT id, knn_dist()
FROM articles
WHERE knn(chunks, 5, 'how do I rotate a certificate');

MAX_TOKENS, OVERLAP_TOKENS, and MAX_CHUNKS control chunk size, shared context at boundaries, and the maximum vector count. Declare a model-backed float_vector_array when creating the table: adding one later with ALTER TABLE ... ADD COLUMN and rebuilding its embeddings are not supported yet.

Put a ceiling on local embedding work

Long-context models can make a single large input unexpectedly expensive, especially on CPU. The new [MAX_INPUT_TOKENS](https://mnt.cr/go/sX4dok column option caps how much of each input is sent to a local embedding model:

ALTER TABLE articles
MODIFY COLUMN chunks MAX_INPUT_TOKENS='512';

The change applies to embeddings generated afterward; existing vectors stay as they are. Set it during CREATE TABLE or change it later without re-embedding the table. A value of 0, or leaving the option out, uses the model's own limit.

This release also fixes two less visible problems around this path. Models no longer remain cached after an embedding column is modified, and configurations with different API_TIMEOUT or MAX_INPUT_TOKENS values no longer collide and reuse the wrong cached model.

UTF-8 identifiers and better German matching

Table, field, and attribute names now follow one consistent [safe UTF-8 identifier syntax](https://mnt.cr/go/mfKS2T RT, percolate, distributed, template, and plain tables can use localized names such as Chinese or Cyrillic identifiers across DDL, expressions, field selectors, and inferred source schemas.

German AOT morphology also gains opt-in sharp-s normalization. With charset_table=non_cont,german and morphology=lemmatize_de_v2 or lemmatize_de_v2_all, forms such as Straße, Strasse, and STRAẞE match in ordinary whole-word searches. If index_exact_words=1 is enabled, exact-word queries can still distinguish the ß and ss forms.

Better load testing and backups

[manticore-load](https://mnt.cr/go/cnoLS6 can now benchmark through the HTTP JSON API with --http. Writes use /bulk, searches use /search, and --table selects the target table. Its reports now include local searchd RSS during the run plus peak RSS, disk, and CPU statistics at the end, including aggregate monitoring for multi-command workloads.

[Manticore Backup](https://mnt.cr/go/t4acW4 now works with authenticated Manticore Search installations through username/password or bearer-token credentials. SQL [BACKUP](https://mnt.cr/go/6dxdrT has a dedicated authorization action and checks read access to the selected tables.

S3 backups and restores can also use the AWS SDK credential provider chain when static keys are not set. That includes IRSA, shared credentials, ECS task roles, and EC2 instance profiles. Temporary credentials can supply AWS_SESSION_TOKEN.

Columnar attributes use mmap by default

Manticore Search now defaults [access_columnar_attrs](https://mnt.cr/go/sdnJ5l to mmap. The operating system maps and caches *.spc columnar-attribute files on demand, without prereading the whole file at startup. The previous buffered path remains available with access_columnar_attrs='file'.

ALTER TABLE also reopens replaced columnar storage with the configured access mode, so an altered table no longer falls back to a different reader than the one requested.

Vector and grouped-search fixes

Several fixes target queries that were valid but could return incomplete results or fail under a particular table layout:

Distributed and sharded KNN queries with a local shard no longer rescore merged 1-bit-quantized results twice, which could crash the coordinator or return the wrong nearest neighbor. ([Issue #4791](https://mnt.cr/go/LfXW1t
KNN queries with additional filters avoid a redundant knn_dist prefilter when HNSW already excludes documents without vectors. ([PR #4861](https://mnt.cr/go/bZtpIJ
LENGTH() on a float_vector_array now reports the number of vectors rather than its internal storage-word count. ([PR #4879](https://mnt.cr/go/8be2oV
Hybrid search with GROUP BY retains all buckets, including MVA groups, and follows both final and within-group ordering. ([Issue #4639](https://mnt.cr/go/b5Hmg8
Hybrid filters on weight() and expressions or aliases derived from it now run after fusion against the final text weight instead of being ignored. Weight-dependent filters inside OR trees remain unsupported and return an explicit error. ([Issue #4889](https://mnt.cr/go/OMu9qt
Multi-chunk RT grouping no longer risks duplicate groups, incorrect split counts, or a hang while ordering COUNT(DISTINCT ...) results. ([Issue #4856](https://mnt.cr/go/KtrFXK
Document-ID filters whose signed representation is negative now follow the lookup index's unsigned ordering. ([Issue #4774](https://mnt.cr/go/nEO1vc

There are crash fixes here too: a second hybrid-search statement in a multi-statement request ([PR #4864](https://mnt.cr/go/RJ5hGi, distributed JSON aggregation sorted by a string attribute ([Issue #4822](https://mnt.cr/go/Jdpl0M, and dropping a table during auto-embedding precommit ([Issue #4860](https://mnt.cr/go/Edp7gc are all handled safely now.

Bulk ingestion behaves predictably

Elasticsearch-compatible /_bulk requests now return HTTP 200 once a batch has been processed, while individual failures remain visible through errors: true and per-item statuses. Duplicate create actions return per-item 409 version_conflict_engine_exception errors, including duplicates within the same batch. This prevents clients such as Fluent Bit from retrying writes that already succeeded.

Fixed-length gzip-compressed /bulk bodies are also decoded correctly when they arrive across multiple socket reads. And if native bulk processing fails because the target table does not exist, the request can again reach Manticore's auto-schema fallback with valid NDJSON while preserving the expected bulk response envelope.

More reliability fixes

The rest of the release closes a broad set of operational and compatibility problems:

searchd --stopwait no longer hangs while a sharded table is being rebalanced after a node rejoins. ([Issue #3905](https://mnt.cr/go/ZVFkNe
Compatible older binlogs replay safely during upgrade and completed RT chunks are published before clean shutdown. ([Issue #4808](https://mnt.cr/go/O0ZRHJ Fatal replay diagnostics also name the relevant recovery flag. ([Issue #4811](https://mnt.cr/go/pzGwEm
indexer creates missing parent directories for plain-table paths when the nearest existing parent is writable. ([Issue #4793](https://mnt.cr/go/eIFSQf
UUID document IDs no longer cause stored text fields to come back empty. ([Issue #4833](https://mnt.cr/go/otelYC
ALTER TABLE ... RENAME preserves hidden remote embedding API keys without exposing them in SHOW CREATE TABLE. ([Issue #4842](https://mnt.cr/go/XkT4iw
Sequel Ace 5.3.1+ compatibility probes work again. ([Issue #4828](https://mnt.cr/go/zjvXbT
JSON /search keeps distances for negated NEAR and proximity operators. ([Issue #4784](https://mnt.cr/go/NgB0Q7
Internal string-sort helper columns no longer leak from LEFT JOIN results. ([Issue #4788](https://mnt.cr/go/rxNeFg
Malformed binary API SEARCH element counts are rejected instead of terminating searchd. ([PR #4790](https://mnt.cr/go/ufPVLK

For the complete list, see the [Version 29.9.0 changelog](https://mnt.cr/go/gl5v6N

Get Manticore Search 29.9.0

Install or upgrade Manticore Search with the [installation guide](https://mnt.cr/go/XPyNrv Review the upgrade notes above if you use columnar attributes, German AOT morphology, or authenticated backups.

Need help or want to connect?

Join our [Slack](https://mnt.cr/go/v4HPeK
Visit the [Forum](https://mnt.cr/go/R7u2vk
Report issues or suggest features on [GitHub](https://mnt.cr/go/CURvTu
Email us at contact@manticoresearch.com

Explore Manticore Data with OpenSearch Dashboards

Sergey Nikolaev — Tue, 08 Sep 2026 10:00:00 +0000

Originally published on https://manticoresearch.com/blog/manticore-opensearch-dashboards-integration/ on September 8, 2026

Explore Manticore Data with OpenSearch Dashboards

Connect OpenSearch Dashboards to Manticore Search, explore data in Discover, and build visualizations and dashboards using a familiar interface.

OpenSearch Dashboards provides a familiar visual interface for exploring data, building charts, and assembling interactive dashboards. Although it is normally paired with OpenSearch, you can also connect it directly to Manticore Search.

The integration lets you keep Manticore as the search and analytics backend while using Discover, Visualize, and Dashboards as the user interface. This is especially useful for log and event data: tools such as Logstash, Filebeat, Fluent Bit, and Vector can send data to Manticore, while OpenSearch Dashboards gives your team a convenient way to inspect it.

In this tutorial, we will configure the connection, add a small sample dataset, explore it in Discover, and create a visualization.

How the integration works

OpenSearch Dashboards communicates with its backend through an HTTP API. Manticore exposes a compatible API on its HTTP listener, which uses port 9308 by default.

The compatibility layer is provided by the EmulateElastic plugin in Manticore Buddy. It handles the requests that OpenSearch Dashboards needs for startup, index discovery, searches, filters, and supported aggregations. Buddy normally starts automatically with searchd, so a standard Manticore installation already has the required component.

This is an API compatibility integration, not an embedded OpenSearch cluster. Manticore stores and queries the data, and OpenSearch Dashboards provides the visual interface.

Prerequisites

For this walkthrough, you need:

A running Manticore Search instance with its HTTP endpoint available at http://localhost:9308
Manticore Buddy installed and running
OpenSearch Dashboards 3.4.0, the currently tested and recommended version
Manticore running in real-time mode

Other OpenSearch Dashboards versions may work, but they have not been tested to the same extent. The version reported by Manticore must also match the version of OpenSearch Dashboards.

Step 1: Configure Manticore

Open your Manticore configuration file and make sure that the HTTP listener is enabled. Set kibana_version_string to the version of OpenSearch Dashboards that you plan to run:

searchd {
    listen = 127.0.0.1:9308:http
    pid_file = /var/run/manticore/searchd.pid
    data_dir = /var/lib/manticore
    kibana_version_string = 3.4.0
}

The name kibana_version_string is retained for compatibility with the existing Elasticsearch-style API. OpenSearch Dashboards checks the backend version during startup, so a mismatch can produce warnings or prevent the application from starting.

Restart Manticore after changing the configuration.

If Manticore and OpenSearch Dashboards run on different hosts or in different containers, do not bind the HTTP listener only to 127.0.0.1. Bind it to an address that OpenSearch Dashboards can reach, and restrict access with your network or firewall configuration.

Step 2: Configure OpenSearch Dashboards

Open opensearch_dashboards.yml. In a tarball installation, it is usually located at config/opensearch_dashboards.yml; packages may place it at /etc/opensearch-dashboards/opensearch_dashboards.yml.

Point opensearch.hosts to Manticore's HTTP endpoint:

opensearch.hosts: ["http://localhost:9308"]

Manticore does not provide the OpenSearch Security plugin. The corresponding Dashboards plugin must therefore be disabled.

For a tarball installation, stop OpenSearch Dashboards and remove the plugin:

./bin/opensearch-dashboards-plugin remove securityDashboards

If you run OpenSearch Dashboards in Docker, provide both settings through environment variables:

environment:
  OPENSEARCH_HOSTS: '["http://manticore:9308"]'
  DISABLE_SECURITY_DASHBOARDS_PLUGIN: "true"

Here, manticore is the hostname or container name reachable from the OpenSearch Dashboards container. Using localhost inside that container would refer to the container itself, not to Manticore.

Start OpenSearch Dashboards and open http://localhost:5601.

Step 3: Add sample data to Manticore

If you already have a real-time table, you can skip this step. Otherwise, connect to Manticore through its MySQL-compatible port:

mysql -h127.0.0.1 -P9306

Create a table for a small set of application events:

CREATE TABLE app_events (
    message text,
    service string,
    status_code uint,
    response_time float,
    event_time timestamp
);

Add a few documents:

INSERT INTO app_events
    (message, service, status_code, response_time, event_time)
VALUES
    ('Request completed', 'catalog', 200, 0.12, 1786924800),
    ('Request completed', 'checkout', 200, 0.31, 1786928400),
    ('Upstream timeout', 'checkout', 504, 2.75, 1786932000),
    ('Product not found', 'catalog', 404, 0.08, 1786935600),
    ('Request completed', 'catalog', 200, 0.15, 1786939200),
    ('Payment rejected', 'payments', 422, 0.44, 1786942800);

This schema gives us text to search, dimensions to group by, metrics to aggregate, and a timestamp for time-based charts.

Step 4: Create an index pattern

OpenSearch Dashboards uses an index pattern to select the data source shown in Discover and visualizations. In this integration, the pattern matches a Manticore table name.

Open Management > Dashboards Management.
Go to Index Patterns and choose Create index pattern.
Enter app_events.
Select event_time as the time field.
Save the index pattern.

The app_events fields should now be available in OpenSearch Dashboards.

Step 5: Explore the data in Discover

Open Discover and select the app_events index pattern. You can now:

Inspect individual documents
Search the message field
Filter by fields such as service or status_code
Change the time range
Add and remove columns from the results

For example, add a filter where status_code is greater than or equal to 400 to focus on failed requests. You can then add service, message, status_code, and response_time as columns to get a compact error view.

Simple Dashboard Query Language searches work with the integration. Advanced DQL features—including nested field searches, regular expressions, fuzzy searches, proximity searches, and term boosting—may not be compatible with Manticore.

Step 6: Build a visualization

Open Visualize, create a new visualization, and choose the app_events index pattern. OpenSearch Dashboards can use the following bucket aggregations with Manticore:

terms
histogram
date_histogram
range
date_range

Supported metric aggregations include:

max
min
sum
avg

As a first chart, create a bar chart of events by service:

Use Count as the metric.
Add a Terms bucket using the service field.
Apply the changes.
Save the visualization as Events by service.

You can also create a time-series chart with a Date Histogram on event_time, or chart average latency by using the avg metric on response_time.

After saving your visualizations, open Dashboards, create a dashboard, and add them. Filters applied at dashboard level let you focus the whole view on one service, status code, or time range.

What is supported

The integration covers the main data exploration workflow:

Searching and filtering documents in Discover
Creating index patterns for Manticore tables
Building visualizations with supported bucket and metric aggregations
Saving visualizations and combining them into dashboards
Managing index patterns and saved objects in Dashboards Management

Manticore also emulates the stack-level requests needed during OpenSearch Dashboards startup, including node version information, cluster settings, configuration objects, and index listings.

Current limitations

OpenSearch Dashboards includes many features that depend on OpenSearch-specific APIs or field types. Those features are outside the scope of this integration.

Unsupported field types include:

Geographic and Cartesian fields such as geo_point, geo_shape, xy_point, and xy_shape
Range types such as integer_range, ip_range, and date_range
Specialized search types such as semantic, rank_feature, and percolator
OpenSearch vector types such as knn_vector and sparse_vector
Relational types such as nested and join
Advanced string types such as completion and search_as_you_type

Plain text and keyword fields are supported. Manticore's own vector search remains available through its SQL and JSON APIs, but OpenSearch Dashboards cannot represent it through OpenSearch's vector field type.

Nested aggregations—an aggs block inside another aggs block—are not supported. Metric functions are limited to those implemented by Manticore.

OpenSearch-specific applications and administration tools are also not available, including:

Geospatial visualizations
Observability and trace analytics
Alerting and Anomaly Detection
Security Analytics
Index State Management and Index Management
Performance Analyzer
OpenSearch Security plugin workflows

These limitations do not affect the core Discover, visualization, and dashboard workflow described above.

Bringing in real data

Once the connection works, you can replace the sample table with your own data pipeline. Manticore integrates with Logstash, Filebeat, Fluent Bit, and Vector. These tools can collect and transform logs or events before sending them to Manticore's Elasticsearch-compatible HTTP endpoint.

The resulting workflow is straightforward:

An agent or pipeline collects your data.
Manticore indexes it in real time.
OpenSearch Dashboards queries Manticore over HTTP.
Users explore the data and build dashboards in a familiar interface.

Conclusion

The OpenSearch Dashboards integration gives Manticore users a practical visual layer for log analysis and data exploration. With a small amount of configuration, you can search documents in Discover, build charts from supported aggregations, and combine them into dashboards while Manticore handles storage and query execution.

For the latest compatibility details and configuration notes, see the OpenSearch Dashboards integration documentation.

How to return top-N from each group with GROUP_CONCAT()

Sergey Nikolaev — Fri, 04 Sep 2026 10:00:05 +0000

Originally posted by Stanislav Klinov on https://manticoresearch.com/blog/group-concat-top-n-per-group/ on September 4, 2026

How to return top-N from each group with GROUP_CONCAT()

A practical example of how to select several most recent items from each group in Manticore Search and combine them into a single string.

Imagine a customer support screen. An agent searches for refund-related events and, instead of a long log, wants a short summary: one row per user, the total number of matches, and the five most recent events. If more details are needed, the application can load them by ID.

Two obvious approaches don't quite give us what we need:

regular GROUP_CONCAT() collects all values in the group together,
while GROUP N BY returns several rows per user.

Starting with Manticore Search 28.6.6, values inside GROUP_CONCAT() can be sorted and limited to the number you need:

GROUP_CONCAT(id ORDER BY event_ts DESC, id DESC LIMIT 5)

Let's see how it works. We'll use SphinxQL and an explicit GROUP BY - the new form of GROUP_CONCAT() doesn't work without it.

Events we'll work with

Let's create an activity table where each document is a single event. The event text is stored in body, the user in user_id, and the timestamp in event_ts.

CREATE TABLE activity (
    body text,
    user_id int,
    event_ts bigint,
    event_type string
);

INSERT INTO activity (id, body, user_id, event_ts, event_type) VALUES
    (1001, 'refund requested for order 501',       101, 1770000010, 'requested'),
    (1002, 'user logged in',                       101, 1770000020, 'login'),
    (1003, 'refund approved for order 501',        101, 1770000030, 'approved'),
    (1004, 'refund email sent for order 501',      101, 1770000040, 'email'),
    (1005, 'refund status checked for order 501',  101, 1770000050, 'checked'),
    (1006, 'refund payout queued for order 501',   101, 1770000060, 'queued'),
    (1007, 'refund webhook retried for order 501', 101, 1770000060, 'retried'),
    (2001, 'refund requested for order 601',       202, 1770000015, 'requested'),
    (2002, 'refund approved for order 601',        202, 1770000025, 'approved'),
    (2003, 'shipping address changed',             202, 1770000035, 'shipping'),
    (2004, 'refund payout queued for order 601',   202, 1770000045, 'queued'),
    (2005, 'refund completed for order 601',       202, 1770000055, 'completed'),
    (3001, 'refund requested for order 701',       303, 1770000012, 'requested'),
    (3002, 'invoice downloaded',                   303, 1770000022, 'invoice'),
    (3003, 'refund rejected for order 701',        303, 1770000032, 'rejected');

A search for refund will find six events for user 101, four for user 202, and two for user 303. We deliberately gave events 1006 and 1007 the same timestamp: later you will see why sorting also needs id.

What's wrong with the old approaches

Let's start with regular GROUP_CONCAT(). The response format is what we need - one row per user:

SELECT
    user_id,
    COUNT(*) AS matched_events,
    GROUP_CONCAT(id) AS all_event_ids
FROM activity
WHERE MATCH('refund')
GROUP BY user_id
ORDER BY matched_events DESC, user_id ASC;

But for user 101, the string will contain all six IDs, for example 1001,1003,1004,1005,1006,1007, while we only need the five most recent ones. Also, without internal sorting, the order of the values is not guaranteed.

+---------+----------------+-------------------------------+
| user_id | matched_events | all_event_ids                 |
+---------+----------------+-------------------------------+
|     101 |              6 | 1001,1003,1004,1005,1006,1007 |
|     202 |              4 | 2001,2002,2004,2005           |
|     303 |              2 | 3001,3003                     |
+---------+----------------+-------------------------------+

Another option is to ask GROUP N BY to select the five most recent documents from each group:

SELECT
    id,
    user_id,
    event_ts
FROM activity
WHERE MATCH('refund')
GROUP 5 BY user_id
WITHIN GROUP ORDER BY event_ts DESC, id DESC
ORDER BY user_id ASC;

The oldest event for user 101 disappears, but each remaining document is returned as a separate row. So instead of three rows, we get eleven: 5 + 4 + 2. This is convenient when the client needs the documents themselves, but it doesn't work for our compact summary.

+------+---------+------------+
| id   | user_id | event_ts   |
+------+---------+------------+
| 1007 |     101 | 1770000060 |
| 1006 |     101 | 1770000060 |
| 1005 |     101 | 1770000050 |
| 1004 |     101 | 1770000040 |
| 1003 |     101 | 1770000030 |
| 2005 |     202 | 1770000055 |
| 2004 |     202 | 1770000045 |
| 2002 |     202 | 1770000025 |
| 2001 |     202 | 1770000015 |
| 3003 |     303 | 1770000032 |
| 3001 |     303 | 1770000012 |
+------+---------+------------+

Combining only the five most recent IDs

Now let's combine both steps: sort the documents directly inside GROUP_CONCAT() and limit the list to five values there as well.

SELECT
    user_id,
    COUNT(*) AS matched_events,
    GROUP_CONCAT(
        id
        ORDER BY event_ts DESC, id DESC
        LIMIT 5
    ) AS recent_event_ids
FROM activity
WHERE MATCH('refund')
GROUP BY user_id
ORDER BY matched_events DESC, user_id ASC;

+---------+----------------+--------------------------+
| user_id | matched_events | recent_event_ids         |
+---------+----------------+--------------------------+
|     101 |              6 | 1007,1006,1005,1004,1003 |
|     202 |              4 | 2005,2004,2002,2001      |
|     303 |              2 | 3003,3001                |
+---------+----------------+--------------------------+

First, MATCH('refund') selects refund events, then GROUP BY user_id groups them by user. COUNT(*) counts all matching events, while GROUP_CONCAT() takes only the first five from each group after sorting.

The second sort key - id DESC - is especially important here. Events 1006 and 1007 have the same event_ts, so without it their relative order would be undefined. Sorting by ID ensures that event 1007 always comes first.

Notice that the query has two ORDER BY clauses. The one inside GROUP_CONCAT() determines the order of IDs in the string. The final ORDER BY sorts the completed rows: first by the number of matches, then by user_id.

The inner LIMIT does not change COUNT(*) or affect pagination of the overall result. So user 101 still has six matches even though only five IDs are shown next to it. That's exactly what we need for this summary.

One more detail: GROUP_CONCAT() always returns a string, even when it contains numeric IDs. If the API needs to return an array of numbers or objects with several fields, the string has to be parsed on the client side, or a different response format should be used.

The query works the same way with distributed tables. Manticore collects candidates from all local and remote tables, then selects the overall top-N for each group.

If a comma doesn't work

By default, values are separated by commas. Sometimes a more readable string is useful - for example, showing the event type next to its ID. You can set a custom separator with SEPARATOR:

SELECT
    user_id,
    GROUP_CONCAT(
        CONCAT(event_type, ':', TO_STRING(id))
        ORDER BY event_ts DESC, id DESC
        SEPARATOR ' / '
        LIMIT 3
    ) AS recent_events
FROM activity
WHERE MATCH('refund')
GROUP BY user_id
ORDER BY user_id ASC;

+---------+----------------------------------------------+
| user_id | recent_events                                |
+---------+----------------------------------------------+
|     101 | retried:1007 / queued:1006 / checked:1005    |
|     202 | completed:2005 / queued:2004 / approved:2002 |
|     303 | rejected:3003 / requested:3001               |
+---------+----------------------------------------------+

In this syntax, SEPARATOR comes before LIMIT. Manticore doesn't escape anything or add quotes: the result is a regular string, not a JSON array.

This works well for event_type because those values are controlled by the application. Be more careful with arbitrary text: if the separator appears in the data itself, the result can no longer be parsed reliably. In that case, it's better to return separate rows or use a structured format.

Where the new approach doesn't work

This form of GROUP_CONCAT() has several limitations. It works only in SQL queries with an explicit GROUP BY and does not support:

DISTINCT, OFFSET, or combining multiple expressions at once;
JOIN, FACET, outer SELECT queries, or table functions;
KNN and hybrid queries, or scroll;
implicit grouping or equivalent aggregation syntax in the JSON API.

Its alias cannot be used in HAVING or the final ORDER BY. The groups themselves can still be sorted by the grouping key and regular aggregates - for example, by user_id and COUNT(*), as in the query above.

Memory usage is another consideration. For each such expression, Manticore stores a separate top-N for every group that remains in the result. The more groups there are, the higher N is, and the more expressions you use, the more memory is required. The size of the values and sort keys also affects memory usage, so it is best not to set an unnecessarily high limit "just in case".

Full documentation for the new functionality is available here.

More examples

Support events are just one possible scenario. The new mode can be useful anywhere a document already contains a group key, the value you want to collect, and a field to sort by:

Product images: collect the first few IDs or paths for each product_id, sorted by display_order.
Priority tasks: return up to six task IDs for each assignee_id, sorted by a precomputed priority.
Server errors: show the latest N errors for each server while keeping their total count separately.

If you need a short list of IDs, names, or paths, GROUP_CONCAT(... ORDER BY ... LIMIT N) now lets you get it in a single query. We hope you find this useful.

How to search for long hashes and IDs with dict='keywords_32k'

Sergey Nikolaev — Wed, 02 Sep 2026 07:30:01 +0000

Originally posted by Manticore Search on https://manticoresearch.com/blog/dict_keywords_32k/ on August 31, 2026

How to search for long hashes and IDs with dict='keywords_32k'

A practical guide to searching long hashes, event IDs, message IDs, and email addresses in Manticore Search: limits, exact matching, wildcard search, tokenization, migration, and limitations.

Full-text search usually works with ordinary words: product names, titles, comments, and descriptions. Such tokens are rarely longer than a few dozen characters.

Logs and technical data are different. A SHA-256 hash is 64 characters long, while message IDs, correlation keys, event identifiers, and some email addresses can be even longer. The value is often meaningful only as a whole: if its tail is lost, one ID can easily be confused with another.

Manticore Search provides dict='keywords_32k' for these cases.

keywords_32k is available starting with Manticore Search 27.1.1. Version 27.1.5 or newer is recommended when converting an existing table from keywords.

The problem with the regular dictionary

By default, Manticore uses dict='keywords'. Its maximum token length is 42 bytes after normalization.

The limit is measured in bytes, not characters. One ASCII character takes one byte, but a UTF-8 character can take several bytes.

If a token exceeds 42 bytes, Manticore truncates it:

when indexing a document;
when processing a search query.

As a result, querying the complete long value does not necessarily return zero results. Because the query is truncated too, the document may be found-but only by the first 42 bytes.

This creates a more serious problem: two different IDs with the same first 42 bytes become indistinguishable to full-text search. It is also impossible to find a value by a fragment located after the 42nd byte.

What `keywords_32k` changes

dict='keywords_32k' increases the maximum normalized token length to 32768 bytes, or 32 KB.

Behavior	`dict='keywords'`	`dict='keywords_32k'`
Maximum token length	42 bytes	32768 bytes
Token exceeding the limit	Truncated	Skipped with a warning
Prefix and infix search	Supported	Supported
Morphology for tokens longer than 42 bytes	Token is already truncated	Not applied
RT tables	Supported	Supported
Plain tables	Supported	Supported

The setting applies to the entire table:

CREATE TABLE events (
  message text,
  event_id text
)
dict='keywords_32k';

Regular short words in the same table continue to use the configured morphology. Tokens longer than 42 bytes are stored in their original normalized form, without stemming or lemmatization.

For machine identifiers, this is usually exactly what you need: a hash or event ID has no useful word stem.

When to use `keywords_32k`

Use it when both conditions are true:

The value can exceed 42 bytes after tokenization.
It must be searchable through MATCH(), by prefix, or by substring.

Typical examples include:

SHA-256 and other long hashes;
event IDs and message IDs;
request IDs, trace IDs, and other technical identifiers;
long record keys;
email addresses with a long local or domain part;
technical values from logs;
long identifiers containing separators.

However, keywords_32k is not required for every ID lookup.

If you only need exact equality

If the application always receives the complete ID and only needs to check exact equality, a string attribute is sufficient:

CREATE TABLE events (
  message text,
  event_id string
);

Then use a regular filter:

SELECT id, message
FROM events
WHERE event_id =
  '9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08';

If you need both exact matching and full-text search

Use string attribute indexed:

CREATE TABLE events (
  message text,
  event_id string attribute indexed
)
dict='keywords_32k';

In this case, Manticore:

stores the original value as a string attribute;
lets you filter it with WHERE;
also indexes it for MATCH() and wildcard search.

This is usually the most convenient schema for technical identifiers.

Searching by the complete token

Create a table and add a 64-character SHA-256 hash:

DROP TABLE IF EXISTS events;

CREATE TABLE events (
  message text,
  event_id string attribute indexed
)
dict='keywords_32k';

INSERT INTO events (id, message, event_id) VALUES
(
  1,
  'delivery accepted',
  '9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08'
);

Use the complete normalized token in a full-text search:

SELECT id, message
FROM events
WHERE MATCH(
  '@event_id 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08'
);

The @event_id operator restricts the search to the required field. Without it, Manticore searches for the value in all full-text fields of the table.

Quotes are not needed around a single simple alphanumeric token here. Quotes denote a phrase search; they do not turn MATCH() into a byte-for-byte comparison of the original string.

A full-text match is not the same as exact equality

MATCH() operates on the result of tokenization and normalization. It can be affected by:

charset_table;
conversion to lowercase;
blend_chars;
ignore_chars;
word forms and other text-processing settings.

For a strict comparison of the stored value, use a string attribute:

SET collation_connection='binary';

SELECT id, message
FROM events
WHERE event_id =
  '9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08';

binary enables byte-for-byte string comparisons in the current SQL session. This setting does not affect full-text search behavior.

The practical rule is simple:

WHERE event_id = ... - a strict comparison of the stored string;
MATCH('@event_id ...') - a normalized-token search;
MATCH('@event_id prefix*') - a prefix search;
MATCH('@event_id *fragment*') - a substring search.

How to inspect tokenization

Before loading a large volume of data, check that Manticore actually sees the value as a single token:

CALL KEYWORDS(
  '9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08',
  'events'
);

The normalized column should contain the complete 64-character hash.

CALL KEYWORDS is especially useful for values containing periods, hyphens, @, colons, slashes, and characters from different writing systems. This lets you inspect the actual token boundaries before indexing the data.

Prefix and substring search

To search inside a token, enable min_infix_len:

DROP TABLE IF EXISTS events_infix;

CREATE TABLE events_infix (
  message text,
  event_id string attribute indexed
)
dict='keywords_32k'
min_infix_len='4';

Prefix search:

SELECT id, message
FROM events_infix
WHERE MATCH('@event_id 9f86d081*');

Substring search:

SELECT id, message
FROM events_infix
WHERE MATCH('@event_id *b2b0b822*');

A positive min_infix_len also enables prefix search. This example uses 4 to prevent excessively short and broad patterns.

For production systems: do not allow excessively short fragments, choose min_infix_len based on real data, use expansion_limit, test performance with a production-sized dictionary, and restrict search to a specific field with @field.

Email addresses and other values with separators

keywords_32k changes only the maximum token length. It does not determine where a token begins and ends.

By default, a period, @, hyphen, and other characters may split a value into multiple parts. If an email address or message ID must also be indexed as a whole, you can use blend_chars:

DROP TABLE IF EXISTS mail_events;

CREATE TABLE mail_events (
  sender string attribute indexed,
  subject text
)
dict='keywords_32k'
blend_chars='., @, -'
min_infix_len='4';

Blended characters are indexed in two ways: as part of the complete token and as separators between regular parts of the value. This makes it possible to search both the complete email address and individual words within it.

How to convert an existing table

For an RT table, you can change the setting with ALTER TABLE:

ALTER TABLE events dict='keywords_32k';

However, this affects only documents added or replaced after the setting is changed. Existing documents are not automatically tokenized again. Their long tokens remain in the old truncated form until the documents are reindexed.

Follow these steps:

Update dict.
Verify the setting with SHOW CREATE TABLE.
Reindex or reload the existing documents.
Check several long values with CALL KEYWORDS and MATCH().

Why `dict='crc'` does not solve this problem

dict='crc' stores keyword checksums instead of their original text. However, it does not increase the allowed token length. The exception to the regular 42-byte limit is implemented specifically by dict='keywords_32k'.

Current limitations

At the time of publication, dict='keywords_32k' has several limitations:

CALL SUGGEST and CALL QSUGGEST are not supported;
it cannot be used in percolate tables;
tokens longer than 42 bytes are not highlighted in snippets or highlights;
indextool --dumpdict cannot dump this type of dictionary;
the full-text REGEX operator works with dict='keywords', but not with keywords_32k.

Do not index secrets

Being able to search a long value does not mean that you should store it in a search index. Do not index API keys, bearer tokens, session cookies, private keys, passwords, reset tokens, or other data that grants access to the system unless absolutely necessary.

If a secret must be matched by its exact value, it is safer to calculate a suitable hash in advance and store only the hash.

Quick checklist

Before enabling keywords_32k, check:

Is the normalized token actually longer than 42 bytes?
Do you need full-text or wildcard search rather than only WHERE value = ...?
Does CALL KEYWORDS see the entire value as one token?
Do you need blend_chars for periods, hyphens, @, and other separators?
Is min_infix_len large enough?
Is the number of wildcard expansions limited?
Have the old documents been reindexed?
Is the field free of secret data?
Does the application avoid relying on highlighting, SUGGEST, percolate, or full-text REGEX?

Summary

dict='keywords_32k' solves a specific problem: it allows a full-text index to store normalized tokens up to 32768 bytes long instead of the regular 42 bytes.

It is well suited to long hashes, event IDs, message IDs, email addresses, and other machine identifiers. Keep three points in mind:

keywords_32k increases the token length but does not change tokenization rules;
MATCH() on a complete token is not the same as a strict comparison of the original string;
existing documents must be reindexed after the setting is changed.

If you only need exact equality, use a string attribute. If you need both exact matching and partial-value search, use string attribute indexed together with dict='keywords_32k'.

Documentation

How KupujemProdajem searches 5.6 million ads at ~10 ms with Manticore Search

Sergey Nikolaev — Mon, 31 Aug 2026 07:00:01 +0000

Originally posted by Manticore Search on https://manticoresearch.com/blog/kupujemprodajem/ on August 31, 2026

How KupujemProdajem searches 5.6 million ads at ~10 ms with Manticore Search

How Serbia's largest online classifieds marketplace uses Manticore Search to search 5.6 million active ads at ~10 ms, process real-time updates, and prepares for hybrid search.

For an online marketplace, having millions of listings is useful only if buyers can find the right one.

At KupujemProdajem, Serbia's largest online classifieds marketplace, search is the primary way users discover ads. Buyers search from the web and mobile applications, while internal teams use search for moderation and support. Because of that, both relevance and latency directly affect user engagement and sellers' ability to be discovered.

Today, KupujemProdajem uses Manticore Search to search approximately 5.6 million active ads, while separate administrative indexes contain around 70 million rows of current and historical data. The production cluster handles around 150 search requests per second on average, with an average Manticore query time of approximately 10 ms.

And the same search infrastructure is now giving the team a path toward something more ambitious: hybrid lexical and semantic search designed specifically for Serbian-language marketplace content.

Search is part of the core product

KupujemProdajem maintains two main kinds of search indexes. The first powers the customer-facing experience. It contains approximately 5.6 million active classified ads, including titles, descriptions, categories, and structured attributes. Users search these indexes while browsing the marketplace on web and mobile.

The second set of indexes is used internally. In addition to current data, these indexes retain part of the history of deleted ads, allowing moderation and support teams to search information that may no longer be visible on the public marketplace.

This makes search important on both sides of the product. For buyers, it determines how quickly they can get from an idea of what they want to a relevant listing. For sellers, search affects whether their ads are discovered. Internally, it gives employees access to historical marketplace data when investigating problems.

As Dimitrije Marinković, Backend Developer at KupujemProdajem, puts it:

“Search is the primary discovery mechanism on the platform, so relevance and latency directly affect user engagement and seller success.”

Choosing a search engine that could grow with the product

KupujemProdajem had already been running a dedicated search engine for years. When its previous engine, Sphinx, stopped being actively developed, the team needed a maintained alternative that would fit the existing architecture without requiring the entire search integration to be rebuilt.

But compatibility was only part of the decision. The new engine also needed to support:

consistently low query latency;
real-time changes to millions of ads;
replication;
SQL that worked naturally with the team's MySQL-centric PHP stack;
full-text search together with structured filters;
flexible relevance tuning;
and a path toward vector and hybrid search.

Manticore provided those capabilities in one system. That last requirement is becoming increasingly important.

KupujemProdajem plans to combine traditional lexical search with vector search using its own embedding model. Because Manticore supports KNN search alongside full-text search, the team can build that functionality inside the search infrastructure it already operates instead of synchronizing its data with a separate vector database.

Keeping millions of changing ads searchable in real time

MySQL remains the source of truth for marketplace data. When an ad is created, edited, or removed, the corresponding change is pushed to Manticore in real time. The production workload averages approximately 40 REPLACE operations per second, with additional UPDATE and DELETE operations.

This is particularly useful for classifieds. Marketplace inventory changes continuously. Sellers publish new ads, update prices and descriptions, change status, and remove products that are no longer available.

A search system therefore has two jobs at once:

return relevant results quickly;
make sure those results reflect the current marketplace.

KupujemProdajem uses Manticore's real-time updates to keep the searchable representation of the marketplace synchronized with MySQL as these changes happen.

More than matching words

Marketplace search also has very different requirements from searching a collection of plain documents. A listing contains text, but it also has structured information such as category, price, location, status, and other properties. KupujemProdajem combines full-text matching with filters over these typed attributes in Manticore.

The team also controls how different parts of the listing affect relevance. The title, for example, receives significantly more weight than the description or category. This makes intuitive sense for classified ads: a product named directly in the title is normally a stronger signal than the same word appearing somewhere in a longer description.

On top of this, KupujemProdajem uses a custom ranker tuned for its relevance requirements and groups results by ad. This gives the team the ability to encode marketplace-specific relevance rules rather than relying entirely on a generic ranking formula.

A small replicated cluster

The entire Manticore deployment runs on four virtual machines. Three nodes form the production replication cluster, while a fourth VM serves as a backup. All writes are sent to one node and replicated to the others. Read requests are distributed among all three production nodes by application-level logic.

The architecture is relatively simple: MySQL → real-time updates → Manticore write node → replicated Manticore nodes → web/mobile/admin search

That simplicity matters operationally. The team does not need a large distributed data platform merely to support marketplace search. The same cluster handles ingestion, full-text retrieval, structured filtering, custom ranking, replication, and the foundation for future vector search.

5.6 million active ads, 70 million admin rows

The user-facing indexes currently contain around 5.6 million active ads and occupy approximately 9 GB. The larger administrative indexes contain approximately 70 million rows, including some historical deleted-ad data, and occupy around 90 GB.

Across the three-node cluster, KupujemProdajem sees approximately:

Metric	Production scale
Active ads	~5.6 million
Admin index rows	~70 million
User-facing index size	~9 GB
Admin index size	~90 GB
Search traffic	~150 queries/sec average
Search traffic per node	~50 queries/sec average
`REPLACE` traffic	~40/sec
Average query time	~10 ms
Infrastructure	3 production VMs + 1 backup

The cluster remains stable under normal operation, without stuck or long-running queries. Short latency spikes of approximately 150 ms occur occasionally, while the average query execution time remains around 10 ms. Those numbers are particularly useful because this is not a synthetic benchmark. They describe an actual production workload behind the primary discovery experience of a large marketplace.

What users get from this architecture

Infrastructure metrics matter, but only because of what they enable at the product level. For buyers, Manticore helps KupujemProdajem keep search fast even while querying millions of listings and applying relevance logic and structured filters. Because changes are continuously pushed into the index, search can also reflect changes in marketplace inventory quickly instead of relying on occasional large reindexing jobs.

For sellers, this infrastructure supports the main mechanism through which buyers discover their listings. For moderation and support teams, the much larger internal indexes make historical ad information searchable even when it is no longer part of the live marketplace.

The next challenge: understanding Serbian-language intent

The next project for the KupujemProdajem team is hybrid search. Traditional full-text search works particularly well when the words entered by a user also appear in an ad. But marketplace users and sellers do not always describe the same thing in exactly the same way.

That problem becomes even more interesting in Serbian. The team highlights several challenges:

rich morphology;
both Latin and Cyrillic scripts;
synonyms;
different ways buyers and sellers can express the same intent.

KupujemProdajem is therefore preparing its own embedding model for Serbian marketplace content. The plan is to retrieve results using both lexical matching and KNN vector search, then combine the two sets of signals at query time.

The user benefit is straightforward. Someone may search for an item using words that never literally appear in the seller's listing, while the listing is nevertheless highly relevant. Semantic retrieval can provide another signal for recognizing that relationship.

As Dimitrije describes the expected result:

“Semantic matching will let users find relevant ads even when their wording doesn't literally match the listing.”

Perhaps just as important for the engineering team, they do not need to introduce another database to do it. The vector representation can live alongside the text and structured attributes already searchable in Manticore, giving the team one engine in which to combine lexical relevance, filters, and semantic similarity.

A foundation for better marketplace search

KupujemProdajem already uses Manticore for one of the most important workflows on its marketplace. Approximately 5.6 million active ads are searchable with an average engine query time of around 10 ms. Updates arrive continuously. Replication provides multiple search nodes. Internal teams search a much larger historical dataset using the same technology.

But the more interesting part may be what comes next. The team can now experiment with semantic retrieval and hybrid relevance without replacing the search infrastructure or operating an additional vector database.

For KupujemProdajem, that means Manticore is not only keeping today's marketplace search fast. It is also giving the team a practical way to make tomorrow's search better.

Full-Text Search Still Works. It Just Doesn’t Get You to an Answer

Sergey Nikolaev — Mon, 24 Aug 2026 10:34:39 +0000

Originally posted by Klim Todrik on https://manticoresearch.com/blog/conversational-search/ on Aug 20, 2026

Imagine a typical online shoe store.

A shopper opens search and types:

I need black waterproof running shoes for daily runs on wet pavement. What would you recommend?

A few years ago, almost no one expected this from a search box. The query would have been shortened to something like:

black waterproof running shoes

Then the shopper would open several product cards, compare descriptions, materials, intended use, and price, and make the decision alone.

Today, people increasingly expect search itself to do part of that work. Not because full-text search has become worse. It still performs very well with exact names, SKUs, product codes, brands, and keywords. What has changed is what people expect to be able to ask a search system.

For example, Google reported in May 2026 that AI Mode had surpassed one billion monthly users. The company notes that people are asking longer, more complex questions that previously did not fit into conventional search. (blog.google)

The same thing happens in an online store.

A query such as:

I need black waterproof running shoes for daily runs on wet pavement.

only looks like one sentence. For a search system, it contains several tasks.

It needs to extract constraints such as color and waterproofing; understand that this is about running rather than walking; account for price, size, and availability; find suitable products; and, if the user asks “which are better,” explain the differences.

No single algorithm solves all of that.

Full-text search, vector search, filters, hybrid search, and a language model each solve different parts of the problem. It is far more effective to combine them than to choose between them.

That is exactly why Manticore has Conversational Search.

Word search, semantic search, and conversation are different tasks

Start with a simple query:

Nike Pegasus 41 black

Here, the system barely needs to interpret the user’s intent. Full-text search handles it directly.

Or something even simpler:

SKU 123456

Semantic methods are not needed here.

Now consider another example:

light shoes for long summer walks

A product card may not contain the words “summer” or “long walks,” but it may include details such as “breathable material” or “lightweight construction.”

This is where vector search becomes useful.

Real queries often fall between these extremes:

black Gore-Tex shoes for everyday running

Some parameters — black and Gore-Tex — need to be preserved. Everyday running describes the user’s intent rather than an exact attribute.

For such cases, Manticore uses hybrid search, combining full-text and vector search through result ranking.

But even hybrid search returns only a list of results.

At that point, search considers its job done. The user usually does not.

It does not answer questions such as:

Which of these models are better suited to rain?

And it certainly does not handle a follow-up such as:

Which of those cost less than $120?

That is a conversation. It needs another layer.

What we built

To test this in practice, we used ConvApparel, a dataset of conversations about choosing apparel. After cleanup, it contained 82,524 products: footwear, pants, tops, and outerwear. Each product has a description, category, images, and attributes. We built Manticore Apparel Shop on this data.

For example, you can type:

I need black waterproof running shoes for jogging

The system first finds suitable products, then a language model generates an answer using them as context, while the interface shows the products themselves.

Try the demo: Manticore Apparel Shop generates a random product and a query that should retrieve it, then demonstrates that the same query does retrieve that product through Manticore.

It is important to keep the connection between the answer and the data. If the system claims that a model is suitable for rain, the user should be able to open the product and verify the source of that claim.

In this approach, the language model does not replace search. It interprets its results.

How it works

Two main commands are used:

CREATE CHAT MODEL

and

CALL CHAT(...)

First, you create a Conversational Search model and set the rules it follows.

CREATE CHAT MODEL assistant (
    model='openrouter:google/gemma-4-26b-a4b-it',
    timeout=60,
    retrieval_limit=5,
    max_document_length=3000,
    custom_prompt='You are a context-only shopping assistant.

Answer using only the provided context.
Do not use outside knowledge or unsupported assumptions.

Recommend only products supported by the retrieved context.
For every recommended product, briefly explain why it matches the request.

End every recommendation with the corresponding
context source ID in the format [ref:<id>].

If none of the retrieved products support the request,
say that you do not have enough information.'
);

retrieval_limit determines how many documents enter the context. max_document_length limits the amount of text from each document.

If there is too little context, the model will not see the right products. If there is too much, latency and query cost increase. Like a person, a language model does not become smarter just because it has been given everything to read.

You can then run a query:

CALL CHAT(
    'I need black waterproof running shoes for jogging',
    'convapparel_products',
    'assistant',
    'demo-session-001',
    'embedding_vector'
);

You can then continue the conversation:

CALL CHAT(
    'Which of these are better for daily use?',
    'convapparel_products',
    'assistant',
    'demo-session-001',
    'embedding_vector'
);

The system uses conversation history, so the user does not need to repeat the context.

Through the HTTP API

Conversational Search is also available through the JSON API:

{
  "chat": {
    "query": "I need black waterproof running shoes for jogging",
    "table": "convapparel_products",
    "model_name": "assistant",
    "conversation_uuid": "demo-session-001",
    "vector_field": "embedding_vector"
  }
}

The request is sent to /search.

What the system returns

The response contains:

conversation_uuid
user_query
search_query — the search query generated by the system
response
sources

search_query is particularly important.

If the user writes:

Which of these would work better in rain?

On its own, this query makes no sense without context. The system therefore forms a complete search query using the conversation history.

This also simplifies debugging: you can trace the entire chain from query to answer.

How retrieval works

Conversational Search uses vector search over an embedding field. The flow looks like this:

user question
→ conversation history
→ search query
→ vector search
→ retrieved documents
→ language model
→ answer and sources

Where search ends and conversation begins

Full-text search works well for exact queries. Vector search works with semantic ones. Hybrid search works with their combination. Conversational Search is needed when the result must be explained, compared, or refined.

Quality

We tested this on our conversational search quality benchmark, using 200 deterministic shopping queries from ConvApparel. The benchmark evaluates the product IDs returned as sources, rather than the wording of the generated answer.

In the current run, Manticore scored 0.3650 Hit@3, 0.4250 Hit@5, 0.5250 Hit@10, and 0.2790 MRR — the best result on each of those metrics among the engines tested. The full repository includes the dataset-building rules, engine configuration, smoke tasks, and raw results.

Conclusion

Search in modern systems is not one algorithm but several layers:

full-text search
vector search
hybrid search
Conversational Search

Each solves its own task. A language model does not replace search; it works on top of it.

Without good search, you do not get a smart assistant; you get a very talkative consultant that barely knows its own catalog.

Want to test the approach in practice? Try Manticore Apparel Shop: choose a random product, ask a question based on it, and confirm that the same product is found and suggested in the answer.

If you want to run it locally or explore the code, check the GitHub repository.

UUID in Manticore: A Practical Guide

Sergey Nikolaev — Thu, 06 Aug 2026 05:20:04 +0000

In the overview article we explained why it makes sense to use the same UUID in search and in the primary database, when one already exists. Here we will go straight to practice: create a table, run the core operations through SQL and the JSON API, and then load several documents through /bulk.

All examples are for Manticore Search 28.5.0 or later. The <generated UUID> value in the responses means the UUID that Manticore creates when processing the request. You do not need to copy this string into the next request: substitute the actual id from your own response.

Table for all examples

To make UUID the document ID, first declare the field as id uuid. Aside from the id type, the RT table schema does not change:

CREATE TABLE products_uuid (
    id uuid,
    title text,
    sku string,
    price int
);

DESC products_uuid will show that the id field has the uuid type. Both SQL and the HTTP API use this value as the document ID, so there is no need to copy the UUID into a string attribute.

Working with UUID through SQL

First, insert a product with a prebuilt UUID:

INSERT INTO products_uuid (id, title, sku, price)
VALUES (
    '550e8400-e29b-41d4-a716-446655440000',
    'Mechanical keyboard',
    'KB-001',
    149
);

In SQL, the UUID must be enclosed in single quotes. You can find the document with a regular WHERE id = '...' condition:

SELECT id, title, sku, price
FROM products_uuid
WHERE id = '550e8400-e29b-41d4-a716-446655440000';

Manticore can also generate the UUID. To do that, simply do not pass any id value:

INSERT INTO products_uuid (title, sku, price)
VALUES ('USB microphone', 'MIC-001', 89);

SELECT LAST_INSERT_ID();

LAST_INSERT_ID() will return the UUID created by this request:

+--------------------------------------+
| last_insert_id()                     |
+--------------------------------------+
| <generated UUID>                     |
+--------------------------------------+

INSERT for multiple documents and the @@session.last_insert_id variable are covered in detail in the documentation section on adding documents.

Let's add another document with a known ID to check IN:

INSERT INTO products_uuid (id, title, sku, price)
VALUES (
    '550e8400-e29b-41d4-a716-446655440001',
    'USB-C dock',
    'DOCK-001',
    119
);

SELECT id, sku, price
FROM products_uuid
WHERE id IN (
    '550e8400-e29b-41d4-a716-446655440000',
    '550e8400-e29b-41d4-a716-446655440001'
);

You can change attributes with a regular UPDATE. The id itself stays the same:

UPDATE products_uuid
SET price = 139
WHERE id = '550e8400-e29b-41d4-a716-446655440000';

An INSERT with an already existing UUID will not overwrite the document. Manticore will return a duplicate error:

INSERT INTO products_uuid (id, title, sku, price)
VALUES (
    '550e8400-e29b-41d4-a716-446655440000',
    'Duplicate keyboard',
    'KB-DUP',
    1
);

To create a new version of the document with the same UUID, use REPLACE:

REPLACE INTO products_uuid (id, title, sku, price)
VALUES (
    '550e8400-e29b-41d4-a716-446655440000',
    'Mechanical keyboard, revised',
    'KB-001',
    129
);

To change only the price, UPDATE is enough. Full-text fields and columnar attributes require REPLACE: it marks the old version of the document with the same ID as deleted and writes the new one. If that ID does not exist yet, Manticore simply adds the document. More details are in the documentation: UPDATE and REPLACE.

Delete the document by the same UUID:

DELETE FROM products_uuid
WHERE id = '550e8400-e29b-41d4-a716-446655440001';

Let's check the current state of the document whose UUID we set on insert:

SELECT id, title, sku, price
FROM products_uuid
WHERE id = '550e8400-e29b-41d4-a716-446655440000';

The SQL client sends and receives a UUID as a string. Pass it as a string parameter, and read the id from SELECT as a string. There is no need to convert it to a number or BINARY(16). Code written for a numeric document ID will need to be adjusted.

The same operations through the JSON API

When writing through the JSON API, the id field is passed alongside table, not inside doc. In the first example, we use an uppercase UUID:

curl -sS http://localhost:9308/insert \
  -H 'Content-Type: application/json' \
  -d '{
    "table": "products_uuid",
    "id": "AAAAAAAA-AAAA-4AAA-8AAA-AAAAAAAAAAAA",
    "doc": {
      "title": "Wireless keyboard",
      "sku": "JSON-KB-001",
      "price": 159
    }
  }'

Manticore accepts UUID in uppercase, stores it in lowercase, and returns it lowercased in the response:

{
  "table": "products_uuid",
  "id": "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa",
  "created": true,
  "result": "created",
  "status": 201
}

For automatic generation, remove the id field entirely:

curl -sS http://localhost:9308/insert \
  -H 'Content-Type: application/json' \
  -d '{
    "table": "products_uuid",
    "doc": {
      "title": "Portable speaker",
      "sku": "JSON-SPK-001",
      "price": 79
    }
  }'

The response contains the ID that should be saved for subsequent operations:

{
  "table": "products_uuid",
  "id": "<generated UUID>",
  "created": true,
  "result": "created",
  "status": 201
}

In /search, you can use UUID in the equals filter. If you request id in _source, the result will contain the same UUID in both _id and _source.id:

curl -sS http://localhost:9308/search \
  -H 'Content-Type: application/json' \
  -d '{
    "table": "products_uuid",
    "query": {
      "equals": {
        "id": "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa"
      }
    },
    "_source": ["id", "title", "sku", "price"]
  }'

{
  "timed_out": false,
  "hits": {
    "total": 1,
    "total_relation": "eq",
    "hits": [
      {
        "_id": "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa",
        "_score": 1,
        "_source": {
          "id": "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa",
          "title": "Wireless keyboard",
          "sku": "JSON-KB-001",
          "price": 159
        }
      }
    ]
  }
}

_id is part of the search result metadata, while _source.id is the document field. For a UUID table, they contain the same string.

Now let's call the remaining endpoints for modifying data one by one. UPDATE changes only the price:

curl -sS http://localhost:9308/update \
  -H 'Content-Type: application/json' \
  -d '{
    "table": "products_uuid",
    "id": "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa",
    "doc": {"price": 149}
  }'

To replace the document completely, call /replace:

curl -sS http://localhost:9308/replace \
  -H 'Content-Type: application/json' \
  -d '{
    "table": "products_uuid",
    "id": "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa",
    "doc": {
      "title": "Wireless keyboard, revised",
      "sku": "JSON-KB-001",
      "price": 139
    }
  }'

Now delete the replaced document:

curl -sS http://localhost:9308/delete \
  -H 'Content-Type: application/json' \
  -d '{
    "table": "products_uuid",
    "id": "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa"
  }'

After /delete, there is no document with this UUID left in the table. A /search by the same ID will return total: 0:

curl -sS http://localhost:9308/search \
  -H 'Content-Type: application/json' \
  -d '{
    "table": "products_uuid",
    "query": {
      "equals": {
        "id": "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa"
      }
    }
  }'

{
  "timed_out": false,
  "hits": {
    "total": 0,
    "total_relation": "eq",
    "hits": []
  }
}

Examples of requests for deleting by ID and by condition are collected in the documentation section on deleting documents.

Where to generate UUID

If the UUID is already issued by the primary database, just pass it to Manticore as id. If the request is repeated, the UUID will stay the same. INSERT will report a duplicate, and REPLACE will write a new version of the document under the same ID. The same rule applies to insert and replace in /bulk.

Manticore can also generate the UUID itself: just do not pass id. But resending such a request will create another document, so for automatic retries it is better to set the UUID explicitly.

For an explicit ID, any UUID version from v1 to v8 is suitable. For automatic generation, Manticore uses its own UUIDv8 structure, but it does not convert UUIDs received from the client into it.

Batch loading through `/bulk`

/bulk accepts NDJSON: each line contains a separate operation. The UUID is passed in the id field, just like in the other JSON API requests:

POST /bulk
Content-Type: application/x-ndjson

{"insert":{"table":"products_uuid","id":"bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb","doc":{"title":"USB hub","sku":"BULK-HUB-001","price":49}}}
{"insert":{"table":"products_uuid","id":"bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbc","doc":{"title":"Laptop stand","sku":"BULK-STAND-001","price":39}}}

After the last data line, a trailing newline is required. With curl, it is convenient to pass the body through --data-binary so it does not strip newlines.

Both operations belong to one table, so Manticore executes them in a single transaction. The shortened response shows how many documents were added and how the entire batch finished:

{
  "items": [
    {
      "bulk": {
        "created": 2,
        "status": 201
      }
    }
  ],
  "current_line": 2,
  "skipped_lines": 0,
  "errors": false
}

On an error, current_line shows the line where processing stopped, and skipped_lines shows the number of skipped lines. If an empty line or a table switch split the request into multiple transactions, Manticore will not roll back the ones that already completed.

When reloading a document, it is important to choose the right operation. If the UUID already exists, insert will fail with a duplicate error. replace will write the document again, and if no such ID exists, it will add a new document.

The id field in /bulk can also be omitted, and Manticore will generate a UUID. But the /bulk response contains only the overall transaction result, without separate results for each inserted row. If you need to keep the ID of each document, it is more convenient to generate the UUID before the batch request or add documents one by one through /insert.

Validation and common errors

Manticore validates the UUID before adding the document. SQL, the JSON API, and /bulk return errors differently, so do not bind your code to the exact wording of the message.

What you sent	What Manticore will do
`550e8400-e29b-41d4-a716-446655440000`	Add the document
`AAAAAAAA-AAAA-4AAA-8AAA-AAAAAAAAAAAA`	Add the document, store the UUID, and return it in lowercase
A string without hyphens or with an invalid character	Return an error because the UUID format is invalid
`00000000-0000-0000-0000-000000000000`	Return an error: zero UUID cannot be used
A number, including `0`	Return an error: `id` must be a string
A UUID with a version outside the 1-8 range or an RFC-invalid `variant` value	Return a validation error
`INSERT` or `insert` in `/bulk` with an already existing ID	Return a duplicate error

A few points:

The canonical form consists of 36 characters split into 8-4-4-4-12 groups. UUID versions from 1 to 8 are valid; in the RFC variant position, one of these characters must appear: 8, 9, a, or b.
Manticore validates only the ID format. Correct generation is the application's responsibility: for UUIDv4, randomness matters, and for UUIDv7, the time component and compliance with the rules matter. Manticore does not convert passed v4 and v7 values into v8.
In a table with UUID IDs, id = 0 does not trigger automatic generation unlike numeric IDs: the request will fail. But if you do not pass id, Manticore will generate its own UUIDv8 structure and return it in the response. Keep in mind that it can be used as a document ID, but not as, for example, an access token.
You can validate the UUID with a standard library on input, but keep in mind that Manticore still performs its own validation.
In the /bulk response, check errors, current_line, and skipped_lines: they show where processing stopped and which part of the batch could not be committed. In SQL batch requests, the entire request is rolled back if an error occurs.

Columnar RT and replication

UUID can also be used as the document ID in RT tables with columnar storage. Only the engine declaration changes:

CREATE TABLE products_uuid_columnar (
    id uuid,
    title text,
    sku string,
    price int
) engine='columnar';

For regular and columnar RT tables, INSERT, REPLACE, and DELETE, exact-match searches, and IN conditions are written the same way. The client code does not depend on the table engine. But UPDATE does not change columnar attributes, so, for example, the price in such a table can only be updated together with the whole document through REPLACE.

Replication also supports UUID. Suppose the catalog cluster has already been created, the nodes have joined, and the local products_uuid table exists. Let's add it to the cluster:

ALTER CLUSTER catalog ADD products_uuid;

In SQL, a colon goes between the cluster name and the table name:

INSERT INTO catalog:products_uuid (id, title, sku, price)
VALUES (
    '550e8400-e29b-41d4-a716-446655441000',
    'Replicated keyboard',
    'REPL-KB-001',
    169
);

SELECT id, sku, price
FROM catalog:products_uuid
WHERE id = '550e8400-e29b-41d4-a716-446655441000';

In the JSON API, the table name is passed unchanged, while the cluster goes in a separate field. For example, the following operation will update a document previously added through SQL:

curl -sS http://localhost:9308/update \
  -H 'Content-Type: application/json' \
  -d '{
    "cluster": "catalog",
    "table": "products_uuid",
    "id": "550e8400-e29b-41d4-a716-446655441000",
    "doc": {"price": 159}
  }'

After replication, the document keeps the same UUID on all nodes. You can run exact-match searches, UPDATE, REPLACE, and DELETE by it.

What to consider before rollout

The uuid type can be assigned only to the id field. It is supported in RT tables, including those with columnar storage and replication, but not in plain, percolate/PQ, or shard tables.
An existing table cannot be switched from a numeric ID to a UUID or back with ALTER TABLE. This decision has to be made when creating the new schema.
A UUID can be used in = and IN conditions. The <, <=, >, and >= ranges, as well as arithmetic on id, are not supported.
UUIDv7 contains time, but filtering id by range is still not allowed. For date-based selection, add an attribute such as created_at timestamp and filter by it.
The document ID cannot be changed with UPDATE. To give an object a different UUID, you will need to create a document with a new ID and delete the old one separately.

The behavior of SQL sessions and the full list of limitations are described in the documentation: UUID document IDs. Support for UUID as a document ID appeared in Manticore Search 28.6.6.

DEV Community: Sergey Nikolaev

Closing the KNN rescoring gap between columnar and row-wise storage

Closing the KNN rescoring gap between columnar and row-wise storage

How slow can rescoring from columnar storage be?

Why the defaults make this important

The graph search is the same; the vector access is different

Why columnar storage still matters

A new access path for columnar vectors

Fit-in-memory result: most of the gap disappears

Out-of-core result: the taxi benchmark

Cold runs

Hot runs

Why non-KNN search remains close

Conclusion

Columnar access configuration

How search works in Kiva's SaaS, where every customer has their own database

How search works in Kiva's SaaS, where every customer has their own database

Search is part of the platform, not just a search box

Why MySQL full-text search was not enough

Why Kiva moved to Manticore

A separate search table for every customer

No dedicated search servers required

Less load on MySQL, more resources for the application

Next step: hybrid search

Searching data with a changing structure

How wine.co.za made an archive of about 145,000 records easier to search with Manticore Search

How wine.co.za made an archive of about 145,000 records easier to search with Manticore Search

An archive of the South African wine industry

Before Manticore: visitors had to know what to type

MariaDB remains the source of truth

Better suggestions as visitors type

Years of reliable search

Better search without a big engineering project

How Bookbot powers 3.5 million saved searches and ~10% of revenue with Manticore Search

How Bookbot powers 3.5 million saved searches and ~10% of revenue with Manticore Search

When every copy is different

One title in the results, many physical copies underneath

Turning unavailable books into future demand

Search that helps readers even when they do not type perfectly

One search engine across the life of a book

Production scale with a compact footprint

Search that follows the life of a physical book

Better Vector Search for Long Documents: Chunking Inside Manticore Search

Better Vector Search for Long Documents: Chunking Inside Manticore Search

TL;DR

The problem, shown with a small example

More about the chunking strategies

truncate — keep it when your documents are short

mean — one vector, but the whole document

fixed — predictable, cheapest to reason about

recursive — the best general default for prose

sentence — when a chunk must be a complete thought

The three knobs

What search looks like

Does it actually help? Numbers on our own manual

Deep content — what chunking is for

Head content — the control group

Cost

Recommendations for choosing a chunking strategy

How big should a chunk be?

Comparing settings

Recipes

How other engines handle this

What chunking does not fix

Limits and gotchas

Try it

Manticore Search 29.9.0: Chunked auto-embeddings and mmap columnar access

Manticore Search 29.9.0: Chunked auto-embeddings and mmap columnar access

Upgrade notes

Long documents can keep more than one embedding

Put a ceiling on local embedding work

UTF-8 identifiers and better German matching

Better load testing and backups

Columnar attributes use mmap by default

Vector and grouped-search fixes

Bulk ingestion behaves predictably

More reliability fixes

Get Manticore Search 29.9.0

Need help or want to connect?

Explore Manticore Data with OpenSearch Dashboards

`truncate` — keep it when your documents are short

`mean` — one vector, but the whole document

`fixed` — predictable, cheapest to reason about

`recursive` — the best general default for prose

`sentence` — when a chunk must be a complete thought

What `keywords_32k` changes

When to use `keywords_32k`

Why `dict='crc'` does not solve this problem

Batch loading through `/bulk`