DEV Community

Nao San for AWS Community Builders

Posted on Edited on

[AWS] Celebrating Hybrid Search Support in S3 Vectors! Reorganizing the Search Methods [S3Vectors]

This article is a machine translation of the contents of the following URL, which I wrote in Japanese:

【AWS】祝、S3 Vectorsのハイブリッド検索対応!検索方法をあらためて整理【S3Vectors】 #rag - Qiita

はじめに RAGやエージェントのメモリで、ベクトル検索を使う場面が増えました。ベクトル検索は意味の近さで探せる一方で、型番、エラーコード、固有名詞のように「文字列そのものが一致しているか」が重要なクエリは苦手とされています。 Amazon S3 Vectorsは、2026...

favicon qiita.com

Introduction

Vector search is increasingly used for RAG and agent memory. Vector search can find results by closeness of meaning, but it is said to struggle with queries where "whether the string itself matches" is what matters, such as model numbers, error codes, and proper nouns.

On November 16, 2026, Amazon S3 Vectors added support for full-text search. You can also use hybrid search, which combines vector search and full-text search in a single query. Until now, S3 Vectors was often described as a store for vector search, and when you needed full-text search, you often designed the system around combining it with another service. With this update, you can use three search methods (vector search, full-text search, and hybrid search) with S3 Vectors alone.

In this article, I organize the content of this update using the official documentation as the primary source, and explain what each of the three search methods is suited for. I also verify, with fictional Japanese manuals, points that the official announcement alone does not answer, and I show the results. The points I checked are these three: whether it really works for Japanese data, what changes between the Japanese analyzer (ja) and the standard analyzer, and what happens when you add full-text search to an existing index later. I first compared the methods with 30 documents and 22 queries. Because no difference appeared between the methods, I ran an additional comparison with 204 documents (including many similar ones) and 50 queries.

Prerequisites

This article assumes that you know the basics of S3 Vectors (vector buckets, vector indexes, PutVectors, and QueryVectors) and the idea behind vector search (embeddings and cosine distance). I ran the verification with S3 Vectors in the Tokyo Region (ap-northeast-1). For embeddings, I used Titan Text Embeddings V2 (1,024 dimensions) on Amazon Bedrock in the same Tokyo Region.

Overview of the Three Search Methods

First, I line up the three search methods side by side. "Questions it is said to be good at" is the official explanation, and "Results in this verification" are my own observations. The details of each method are in the later sections.

Item Vector search Full-text search Hybrid search
How it searches By closeness of embeddings (vectors), that is, closeness of meaning By whether the words in the query appear in the document (BM25) Merges the two rankings into one with RRF
Questions it is said to be good at (official explanation) Meaning and paraphrases Specific words, such as model numbers, error codes, and proper nouns Questions that involve both meaning and specific words
What you need to use it Generating embeddings Text to search, and a full-text search configuration (an ENHANCED index) Both of those
Questions it tends to struggle with Questions where matching a specific word matters (official explanation; not reproduced in this verification) Paraphrases (this verification) Affected by the choice of analyzer (this verification)
Results in this verification Hit@1 is 1.00 for codes and model numbers, and 0.50 for paraphrases Hit@1 is 0.20 for paraphrases. For queries with a sentence added to the code, it is 0.00 with the standard analyzer and 1.00 with ja Standard is the best on Hit@1 across the whole additional round, at 0.94 (on Recall@5 across the whole round, vector only is the best, at 0.96). ja has a Hit@1 of 0.20 for paraphrases

What Use Cases Fit?

First, I organize the situations where hybrid search is useful. The axis for judging is the nature of the question you want to search.

The Nature of the Question and the Search Method That Fits

Nature of the question Example (from this article's verification data) Search method I would expect to fit
Searching for a specific string itself Model numbers and error codes such as "E1234" and "TST-X420" Full-text search
Searching by meaning or paraphrase 冬に配管が壊れるのを防ぎたい ("I want to prevent pipes from breaking in winter"; the document says 凍結対策, "freeze protection") Vector search
The two above are mixed in one question "E1234 センサー交換" (a code, plus what you want to do: replace the sensor) Hybrid search

This table is my own translation of the official explanation (full-text search is strong for codes and proper nouns, and vector search is strong for meaning) into concrete examples (my inference). Whether it really holds is what I check in the verification later in this article.

Hybrid Search Use Cases Listed in the Official Documentation

The official documentation lists the following four use cases where hybrid search is useful. The right column contains concrete situations that I imagined (my inference).

Use case listed in the official documentation Example situation
RAG and knowledge base search over documents An internal help desk. Questions that include a code ("Error E1234 appeared") and questions that only describe a symptom ("the pump does not run") are mixed
Agent memory When recalling past interactions, an agent uses both a hint about meaning ("that conversation about the setting") and a hint in the form of an identifier, such as a ticket number or order ID
Catalog and enterprise document search (relying on IDs, citations, and codes) In a product catalog, some people type a model number directly, and others search with a sentence such as "a small one that can be used outdoors"
Media and content discovery You want to search by both a match in titles and descriptions and closeness in concepts

For agent memory, the official documentation says the following.

"Agent memory, where an agent recalls past interactions by both meaning and exact identifiers"
—— Hybrid search with Amazon S3 Vectors

Cases Where It Does Not Fit or May Not Be Needed

There are also cases where hybrid search is not needed (the following includes my own inference). When your questions are almost all natural sentences and matching a specific word does not matter (for example, a FAQ that is close to small talk), vector search alone may be enough. That is because the full-text search configuration and the constraints that are hard to change later (described below) add extra work.

Likewise, if you can search by an exact match on an ID alone, a metadata filter may be enough, and you do not need to use vector search or full-text search at all.

The other case is when you need a consistently high query rate or low latency. The official AWS blog explains that S3 Vectors is intended for storing large-scale vectors at low cost with infrequent queries. It is said to fit when you can accept responses of about 100 ms or more, or fast-growing data with a moderate query rate. In the comparison table of the same blog, the latency (P95) is listed as more than 100 ms, and the QPS per index as "low hundreds", contrasted with "thousands" for Amazon OpenSearch Service (source: AWS vector solutions: Build agentic AI where your data lives). I could not confirm in the official documentation whether this new feature changes that assumption.

Three Questions to Judge Whether It Fits Your Use Case

  1. Do the documents you search contain words where an exact match matters, such as model numbers, IDs, codes, and proper nouns? If not, vector search alone may be enough.
  2. Do users' questions include ones written by meaning or paraphrase? If not, full-text search alone may be enough.
  3. Does it fit what S3 Vectors is intended for (a low to moderate query rate, and responses of about 100 ms or more being acceptable)? If not, other services are also candidates.

Hybrid search comes into play when both 1 and 2 apply. The verification data in this article is built so that both are mixed.

How It Works and How to Use It

The Three Types of Queries

On the same index, you use three types of queries depending on the combination of arguments to QueryVectors.

Type Arguments to specify Basis for ranking
Vector only queryVector only Vector distance
Full-text only queryTexts only BM25 score
Hybrid Both queryVector and queryTexts RRF score

"Both present runs a hybrid search. S3 Vectors runs a full-text search for each query text field and a vector search, fuses the rankings with Reciprocal Rank Fusion (RRF), and returns one ranked list of your top K results."
—— Querying with full-text and vector search

When you search multiple text fields at the same time, BM25 runs for each field, the scores are summed into one full-text score, and then that result is merged with the vector side by RRF.

Design Considerations

From here, I organize the points that require judgment in design and operation, based on the official documentation. In the text, "(official)" means the statement is in the official documentation itself, "(my inference)" means it is my own reasoning, and "(confirmed in verification)" means it is an observation from this verification.

An ENHANCED Index Is Required

"Full-text and hybrid search are supported on ENHANCED indexes only, not on CLASSIC indexes."
—— Hybrid search with Amazon S3 Vectors

The difference between the two modes is when the metadata filter is applied (official). ENHANCED applies it before the vector search, and CLASSIC applies it during the vector search. For a CLASSIC index, you convert it to ENHANCED with UpdateIndexMode and then enable full-text search. The conversion takes effect in place, without re-ingestion or changes to queries (official, S3 FAQ).

In addition, you can set CLASSIC only on indexes in vector buckets created before September 30, 2026 (official). From this, new buckets created after that date appear to assume ENHANCED, but the documentation does not state this explicitly (my inference). In this verification, the indexes in the newly created vector bucket were all ENHANCED when I created them without specifying a mode (confirmed in verification).

The Full-Text Search Configuration Cannot Be Changed After Creation

"After the index is created, you can add new full-text search fields or remove full-text search fields, but you can't change an existing full-text search configuration (for example, its analyzer)."
—— Configuring full-text search

After an index is created, what you can and cannot do is as follows (official).

Operation Allowed?
Add a full-text search field (add in UpdateMetadataConfiguration) Yes
Remove a full-text search field (delete) Yes
Change the settings of an existing field (such as the analyzer) No. Delete the key's configuration and add it again. The full-text index for that key is then rebuilt
Change nonFilterableMetadataKeys No

Filterable metadata is limited to 2 KB per vector. For that reason, long text needs to be put in nonFilterableMetadataKeys when you create the index. Because this cannot be changed later, adding a long field to an existing index as a full-text search target later may require recreating the index, depending on the design (my inference).

Enabling Full-Text Search on Existing Data (Backfill)

When you enable full-text search on an existing field, indexing of the existing data runs in the background. The official description is as follows.

Situation Behavior
Status Changes from BUILDING to ACTIVE. It takes from a few minutes to a few hours
Queries during BUILDING Full-text and hybrid queries that target that field are rejected
Writes during BUILDING PutVectors works. Vectors that arrive are indexed
Other configuration changes during BUILDING Result in a ConflictException. A backfill cannot be canceled, and a field under construction cannot be removed

"While a field is BUILDING, full-text or hybrid queries that target that field are rejected. The field becomes searchable when its status is ACTIVE."
—— Configuring full-text search

If you already know which fields you want to enable, put them together in a single UpdateMetadataConfiguration call. You can check progress with GetIndex. The measurements on a small dataset are summarized later, in "Backfill Behavior".

Text Analyzers (Japanese)

An analyzer is the mechanism that splits text into the units used for search (tokens). The same analyzer is applied both when writing data and when querying (official).

Type Description
Standard analyzer (default) Splits at Unicode word boundaries, removes most punctuation, and lowercases
Language-specific analyzer You specify a language per field. It applies stemming and stopword removal
Supported languages English, French, Spanish, Portuguese, Italian, German, Dutch, Chinese, Japanese, and Korean (en fr es pt it de nl zh ja ko)

The official documentation does not describe how the Japanese analyzer splits words. In this article, I observe its behavior in the verification.

Here is the S3 console showing "Metadata configuration" for idxja. title and body are registered as full-text search metadata, the analyzer setting is "Japanese" (日本語), and the status is "Active" (アクティブ). The screenshot is from the Japanese console.

The S3 console showing the metadata configuration of idxja. The body and title fields are active as full-text search metadata with the analyzer set to Japanese

Permissions (IAM)

A QueryVectors call that uses any of queryTexts, a metadata filter, or returnMetadata requires s3vectors:GetVectors in addition to s3vectors:QueryVectors.

"If you use any of these options without the s3vectors:GetVectors permission, Amazon S3 Vectors returns an AccessDeniedException."
—— QueryVectors

If an existing policy allows only QueryVectors, full-text and hybrid queries may not go through (my inference). I ran this verification in an environment with broad permissions, so I did not confirm this point.

Limits (an Excerpt Related to Full-Text Search)

Item Limit
Number of metadata keys targeted for full-text search (per index) 10
Size of one full-text search field 100 KB
Total metadata per vector 100 KB (ENHANCED) / 40 KB (CLASSIC)
Number of full-text fields searched in one QueryVectors call 10
Length of one queryTexts entry 1,024 characters
topK of QueryVectors 10,000
Number of vectors per PutVectors call 500

Source: Limitations and restrictions

Because queryTexts is limited to 1,024 characters, a design that passes a long question straight to the full-text side needs to truncate it (my inference).

About the Weights in RRF

I could not find a parameter in the official documentation for adjusting the weights between the vector side and the full-text side. If you want your own ranking, the official documentation shows a method: issue a full-text-only query and a vector-only query as separate calls, and merge them in your application.

"If you have your own re-ranking or score-normalization step, run a full-text-only query and a vector-only query as separate QueryVectors calls with returnScores enabled."
—— Querying with full-text and vector search

Verifying with Japanese Data

What the Official Documentation Says About Accuracy

The official documentation says that hybrid search can improve recall over either method alone, and it recommends that you verify this with your own data.

"Since actual performance might vary depending on your specific use cases, we recommend conducting your own tests with representative data and queries to validate that S3 Vectors meets your recall requirements."
—— Querying with full-text and vector search

I could not find official accuracy figures, so in this article I check with Japanese data, albeit on a small scale.

Verification Design

This is the overall setup of the verification. From my local PC, I created embeddings for documents and queries with Bedrock, wrote them to S3 Vectors indexes, and queried with five methods.

Item Details
Data 30 documents from a fictional pump controller manual, written in Japanese (title and body). All company names, product names, and model numbers are fictional. I deliberately mixed in similar model numbers (TST-X420 and TST-X402, TSP-3300 and TSP-3300B) and similar error codes (E1234 and E1243)
Queries 22 queries: code (model numbers and codes) 6, term (Japanese terms) 4, semantic (paraphrases and natural sentences) 6, variant (notation variants) 4, and mixed 2. Each query has the ID of its correct document
Methods compared Five: vector only, full-text only (standard), full-text only (ja), hybrid (standard), and hybrid (ja)
Indexes Two: one with the standard analyzer and one with ja. I put the same vectors and the same text in both
Full-text search targets Two fields, title and body (body is in nonFilterableMetadataKeys)
Metrics Recall@5 (the share of correct answers that appeared in the top 5), MRR (the reciprocal of the rank of the first correct answer), Hit@1 (the share of queries whose top result is correct; used in the additional round), and the top 5 results for each query
Embeddings Titan Text Embeddings V2, 1,024 dimensions, Bedrock in the Tokyo Region
Environment boto3 1.44.4, Windows 11, run on November 17, 2026

The correspondence between the five methods, the query arguments they use, and the analyzers is as follows. "Standard" and "ja" in the method names refer to the type of analyzer used for full-text search.

Method Arguments to specify Analyzer (index used)
Vector only queryVector only None (run on the index for the standard analyzer; the vectors in the two indexes are the same)
Full-text only (standard) queryTexts only Standard analyzer (the standard index)
Full-text only (ja) queryTexts only Japanese analyzer (the ja index)
Hybrid (standard) queryVector and queryTexts Standard analyzer (the standard index)
Hybrid (ja) queryVector and queryTexts Japanese analyzer (the ja index)

I call these 30 documents and 22 queries the "first round". In the first round, vector only reached the ceiling (a Recall@5 of 1.00) on every query, so I could not measure any difference between the methods. So after seeing the results, I designed an "additional round" with conditions where differences could appear. I report the results of the first round without changing them.

Item Details of the additional round
Documents 204. I added 120 error code documents and 60 model number documents to the 30 documents of the first round
Error code documents Four-digit codes with swapped digits or a single digit changed, in documents whose body is almost a template. The combination of component and symptom differs for each document
Model number documents TST-X500 to TST-X559. Documents that differ only in the model number and rated values
Queries 50: code (code only) 20, code_ctx (code plus a sentence) 10, model (model number only) 10, and semantic (a paraphrase of the symptom, with no code) 10. Each query has exactly one correct document
Reproducibility The random seed is fixed

Here are examples of first-round queries. The queries are in Japanese because the data is in Japanese.

Type Query Correct answer What I wanted to see
code E1234 The document for error code E1234 Whether it gets pulled toward a similar code (E1243)
code TSP-3300B The two documents for TSP-3300B Whether documents for TSP-3300 get mixed in when the model number is split at the hyphen
term テストクラウド (Test Cloud) The document for テストクラウド Whether full-text search can pick up a Japanese proper noun
semantic 冬に配管が壊れるのを防ぎたい ("I want to prevent pipes from breaking in winter") The freeze protection document (凍結対策) Whether vector search can find it even though the words for "winter" and "break" do not appear in the document
variant 監視サーバ ("monitoring server", without the long vowel mark) Both the documents that write サーバー and サーバ Whether the difference in the long vowel mark (ー) is absorbed
variant E1234 (full-width digits) The document for E1234 Whether the difference between full-width and half-width characters is absorbed

Limitations of the Verification

The first round has 30 documents and 22 queries, with 2 to 6 queries per type. The additional round also has only 10 to 20 queries per type. Because a change of one rank can move an average a lot, I use the contents of the top 5 results for each query as the main basis, not only the size of the numbers. The data is fictional and differs from the distribution of real business documents. The documents in the additional round follow a template that I made, so they are artificially more similar to each other than real documents would be. In addition, I designed the additional round after seeing the results of the first round, and did not decide it in advance. I used only one embedding model, so with a different model, the results on the vector side could change. Latency is a client-side value measured from my local PC, and is different from official performance figures. Even when the results differed from my expectations, I did not adjust the evaluation set, and I report them as they are.

Verification Results

First, here is how to read the results.

  • The values in the tables range from 0 to 1. A larger value means the method puts correct answers higher in the results
  • The three metrics are used as follows
    • Hit@1: the share of queries whose top result is correct. It shows whether the first result you look at is a hit
    • Recall@5: the share of correct answers that appeared in the top 5. It shows whether you reach a correct answer if you look at up to 5 results
    • MRR: how high the first correct answer ranked. It is 1 for rank 1, 0.5 for rank 2, and 0.33 for rank 3
  • The columns of the tables are the types of queries. "all" is the value over all queries
  • In the first round, I look at Recall@5 and MRR. In the additional round, each query has exactly one correct answer, so I added Hit@1 and look at Hit@1 and Recall@5
  • The number of queries per type is small, so a change of one query moves a value a lot. Pay attention to which types of queries show differences, not only to how large the numbers are

First Round (30 Documents, 22 Queries)

Recall@5 (the share of correct answers that appeared in the top 5).

Method code term semantic variant mixed all
Vector only 1.00 1.00 1.00 1.00 1.00 1.00
Full-text only (standard) 1.00 1.00 0.83 0.75 1.00 0.91
Full-text only (ja) 1.00 1.00 0.75 1.00 1.00 0.93
Hybrid (standard) 1.00 1.00 1.00 1.00 1.00 1.00
Hybrid (ja) 1.00 1.00 1.00 1.00 1.00 1.00

MRR (the reciprocal of the rank of the first correct answer).

Method code term semantic variant mixed all
Vector only 1.00 1.00 1.00 1.00 1.00 1.00
Full-text only (standard) 1.00 1.00 0.67 0.75 1.00 0.86
Full-text only (ja) 1.00 1.00 0.64 1.00 1.00 0.90
Hybrid (standard) 1.00 1.00 1.00 1.00 1.00 1.00
Hybrid (ja) 1.00 1.00 1.00 1.00 1.00 1.00

Vector only scored 1.00 on every type, so hybrid search had no room to do better. Even so, the top 5 results for each query show differences between the methods.

For the model number "TSP-3300B" (the correct answers are the two TSP-3300B documents), full-text only returned the two correct documents plus a document for a different model number, TSP-3300. Both the standard analyzer and ja seem to split the model number around the hyphen and match on the shared "TSP" or "3300" (an inference from the behavior). For the full-width query "E1234", full-text only (standard) returned 0 results, while full-text only (ja) put the correct E1234 document at rank 1. For "電気代を減らしたい" ("I want to reduce my electricity bill"), full-text only (ja) returned 0 results. Standard put the correct answer at rank 4, but returned 5 results that included unrelated documents. Note that full-text only can return fewer than 5 results. This is probably because full-text search returns only documents that contain words from the query.

Additional Round (204 Documents, 50 Queries)

Hit@1 (the share of queries whose top result is correct).

Method Code only (code) Code + sentence (code_ctx) Model number only (model) Paraphrase (semantic) all
Vector only 1.00 1.00 1.00 0.50 0.90
Full-text only (standard) 1.00 0.00 1.00 0.20 0.64
Full-text only (ja) 1.00 1.00 1.00 0.20 0.84
Hybrid (standard) 1.00 1.00 1.00 0.70 0.94
Hybrid (ja) 1.00 1.00 1.00 0.20 0.84

In the Hit@1 chart, the only types where the methods differ are code + sentence and paraphrase. For code only and model number only, all methods scored 1.00.

Recall@5 (the share of correct answers that appeared in the top 5).

Method Code only (code) Code + sentence (code_ctx) Model number only (model) Paraphrase (semantic) all
Vector only 1.00 1.00 1.00 0.80 0.96
Full-text only (standard) 1.00 1.00 1.00 0.60 0.92
Full-text only (ja) 1.00 1.00 1.00 0.40 0.88
Hybrid (standard) 1.00 1.00 1.00 0.70 0.94
Hybrid (ja) 1.00 1.00 1.00 0.70 0.94

For paraphrases, the ranking of the methods flips between Hit@1 and Recall@5. Hybrid (standard) beat vector only on Hit@1 (0.70 versus 0.50). On Recall@5, however, vector only is ahead (0.80 versus 0.70). Hybrid (standard) is strong at putting the correct answer at rank 1, but it did not match vector only at getting it into the top 5. These are results from 10 queries, so a difference of one query equals 0.10.

code_ctx queries add a sentence to the code, like "E2675 が出たときの対処" ("how to handle it when E2675 appears"). For full-text only (standard), the correct answer was not at rank 1 for any of the 10 queries (it was at rank 2 or 3). I think the phrase "が出たときの対処" matches characters that appear in almost every document, which disturbs the ranking. The next observation supports this.

Observing Analyzer Behavior

Using full-text search only, I queried a few strings directly and compared the number of matching documents (on the indexes of the additional round, up to 100 results).

Query Standard analyzer ja analyzer What it suggests
が出たときの対処 (particles and a verb ending, plus "handling") 100 results (the upper limit) 0 results The standard analyzer also matches on characters shared by many documents, such as particles. ja removes function words, so nothing matches
対処 ("handling") 3 results (freeze protection, noise countermeasures, water hammer countermeasures, and so on) 0 results The standard analyzer seems to treat 「対処」 character by character, so documents that contain 「対」 matched
E1234 (full-width) 0 results 1 result (the correct one) ja absorbs full-width characters. The standard analyzer does not
e1234 (lowercase) 1 result 1 result Both ignore the difference between uppercase and lowercase
エコ ("eco") 0 results 2 results (the エコフロー documents) ja splits katakana compound words and matches parts of them
サーバ / サーバー (without and with the long vowel mark) One different document each One different document each Neither absorbs the difference in the long vowel mark
モータ ("motor", without the long vowel mark) 0 results 0 results Same as above. It does not match 「モーター」
ウォーターハンマ (truncated "water hammer") 0 results 0 results There is no prefix-match-like behavior for input with the ending cut off

The "What it suggests" column is my own inference from the counts, and is not a description of official specifications.

Latency

This is the round-trip time of QueryVectors measured from my local PC (in milliseconds). It is not an official performance figure.

Method First round: average (max) Additional round: average (max)
Vector only 107 (443) 81 (137)
Full-text only (standard) 99 (166) 84 (175)
Full-text only (ja) 103 (186) 83 (145)
Hybrid (standard) 84 (95) 79 (85)
Hybrid (ja) 90 (120) 79 (89)

Hybrid search did not tend to be slower than vector only. The maximum values are affected by outliers such as the first call.

Backfill Behavior

I enabled full-text search later on idxbackfill (created without full-text search, with 30 documents already written) and checked the behavior.

Situation Result
Full-text query before enabling ValidationException: "One or more metadataKeys in queryTexts are not configured for text search on this index"
Full-text query right after enabling (during BUILDING) ValidationException: "One or more metadataKeys in queryTexts cannot be queried until their status is ACTIVE"
Another configuration change during BUILDING ConflictException: "An index metadata configuration update is already in progress"
Time until ACTIVE With 30 documents, both title and body were ACTIVE at the check after 5 seconds
Full-text query after ACTIVE Succeeded

As the official documentation says, full-text queries during BUILDING were rejected, and another configuration change conflicted. With only 30 documents, I could not reproduce the official description of "a few minutes to a few hours".

Guidelines for Choosing a Method

These are guidelines for choosing a method within the scope of this verification. The numbers are small, so please read them as tendencies.

When Method that fits Basis in this verification
Searching short documents where model numbers and error codes are written directly in the body Vector only was enough Even among 204 similar documents, Hit@1 was 1.00 for code and model
You want exact matches on codes to be reliable, independent of the quality of the embeddings Hybrid (or add full-text search) No difference appeared in this verification. This is my inference about a use case that needs a guarantee of exact matches
Searching by paraphrase or natural sentence Vector only, or hybrid with the standard analyzer For semantic, Hit@1 is 0.50 for vector and 0.70 for hybrid (standard). However, Recall@5 flips: 0.80 for vector and 0.70 for hybrid (standard). Hybrid (ja) has a Hit@1 of 0.20 and a Recall@5 of 0.70
Searching Japanese sentences that contain a code Full-text with ja, or hybrid For code_ctx, full-text only with the standard analyzer scored 0.00, and full-text only with ja and hybrid scored 1.00
Full-width and half-width characters are mixed Specify ja Full-width digits matched only with ja
The long vowel mark (ー) varies Cover it on the vector side (for example, hybrid) Full-text search did not absorb it with either analyzer

Conclusion

S3 Vectors now supports full-text search and hybrid search, in addition to vector search. This means you can build a search that mixes questions where matching specific words matters and questions that search by meaning, with S3 Vectors alone. Organizing it again, here are the guidelines for choosing a method (within the scope of this verification).

First, for short documents where model numbers and codes are written directly in the body, vector search alone found them well enough in this verification. The situation the official documentation describes, that "vector search is weak on model numbers", was not reproduced under these conditions. Whether adding full-text search is worth it depends on the length of your documents and the embedding model, so you need to check with your own data.

Next, if you use full-text search on Japanese, there is a reason to specify ja instead of the standard analyzer. ja was better at absorbing full-width digits and at queries where a sentence is added to a code. However, ja is not always better. For paraphrase queries, hybrid (ja) scored lower than vector only. Because some parts of the full-text search configuration cannot be changed later, it is safer to try your own queries on a small dataset first and then design.

References

Top comments (0)