DEV Community: Jon Handler

Dial in your search with User Behavior Insights: from clicks to accurate results

Jon Handler — Tue, 13 Oct 2026 13:20:06 +0000

Users grade your search results by acting on them. Write down the grades yourself, and use them to make the results better.

Every action a user takes on a search result, a click, a hover, an add-to-cart, a purchase, tells you whether the ranking was right. Most search systems let that signal fall on the floor. Capture it instead and you get a continuous stream of relevance judgments, generated by the people who actually know what they were looking for, at zero labeling cost. That stream is what turns relevance tuning from guesswork into measurement.

The problem is that most teams have no structured record of what their users do. They know the query volume and maybe the zero-result rate, but they cannot answer the question a data scientist actually asks: for this query, which results did we show, and which ones did the user click? Without the causal link between a query, the results it returned, and the actions taken on those results, every relevance decision is an opinion.

User Behavior Insights (UBI) is the OpenSearch answer to that gap. Introduced in OpenSearch 2.15 and built on an open specification, UBI is a schema and a collection path for recording search behavior: the queries users submit, the results shown, and the actions taken on those results, all linked together. This post covers what UBI captures and how to collect it in Amazon OpenSearch Service. More to the point, it shows how to turn that data into results that move the numbers you care about: more of the purchases, answers, and signups your search is supposed to produce, fewer dead-end queries that send customers elsewhere, and retrieval your agents can actually trust.

What UBI actually records

UBI separates two kinds of records and ties them together. Query records capture what was searched and what the engine returned, with an identifier for the query. Event records capture what the user did next: a click, a scroll, an add-to-cart, a conversion, each tagged with the same query identifier so you can trace an action back to the exact result set that produced it. Plenty of analytics tools count clicks, but counting is not the same as explaining. UBI records which query showed which results, in which positions, and which result the user acted on, so you can judge whether a ranking did its job.

Because the schema is an open specification rather than a proprietary log format, the data you collect is portable and interoperable across tools that speak UBI. That matters when you want the behavior data to feed more than one consumer: a relevance dashboard, an offline evaluation harness, and a tuning workflow can all read the same records without a bespoke export for each.

Collecting behavior data in Amazon OpenSearch Service

The client side is where behavior originates, so UBI ships an optional JavaScript collector that manages the query identifier for a search and records events against it as the user interacts with the results. Your web page or app initializes the collector, associates it with each query, and sends click and action events as they happen. The collector handles the lifecycle of tying events back to the originating query so you do not hand-roll that plumbing.

On the server side, the open-source UBI plugin can record query data directly, avoiding a round trip to the client for the query and result payload. The plugin is part of open-source OpenSearch and is not bundled into Amazon OpenSearch Service, but you do not need the plugin to adopt UBI. Amazon OpenSearch Service documents a supported path for collecting UBI-formatted data by sending query and event records to UBI indices directly, so the managed service stores and analyzes the same schema. The data lands in OpenSearch indices, which means you query and visualize it with the same tools you already use for search.

Turning behavior into relevance

Once the events are flowing, the metrics that were previously anecdotes become queries you can run. Zero-result rate, click-through rate by position, the share of sessions that refine a query and search again, and the rank of the result users actually click all come straight out of the UBI indices, through query DSL or SQL in the Query Workbench. A dashboard over those indices turns the stream into a picture of where search is winning and where it is losing.

The behavior data also drives OpenSearch features built on top of the UBI specification. The open-source Search Relevance Workbench uses judgments derived from behavior to evaluate and compare ranking configurations, and its Hybrid Search Optimizer uses the same data to sweep normalization and weighting settings for hybrid search rather than making you guess at them. Those same behavior-derived judgments feed Learning to Rank: the Workbench turns UBI events into implicit judgment lists, and Learning to Rank, available in Amazon OpenSearch Service, trains an XGBoost or RankLib model on them to rescore results with behavioral signals that BM25 alone never sees. The click that a user made yesterday becomes an input to the ranking they see tomorrow. That is the feedback loop that separates a search system that improves from one that just runs.

Behavior data for agentic search

Your users are no longer the only ones searching. Agents now run retrievals too, and they leave behavior signals of their own. When an agent runs a retrieval, reads the results, and picks which passages to use in its answer, that selection is a relevance judgment in the same way a human click is. Record agent interactions in the UBI schema and you get a behavior stream for agent traffic alongside the human one, so you can see which retrievals actually fed answers and which the agent threw away. The stakes are higher than they look: an agent that retrieves the wrong passages produces a confidently wrong answer, and without behavior capture you hear about it from a user complaint instead of your own data.

Capturing all of this behavior is worth it on its own, but it also feeds something that pays it back. Agentic relevance tuning, delivered through the OpenSearch Relevance Agent introduced as an experimental feature in OpenSearch 3.6, takes the complexity out of relevance tuning by working directly from your UBI data. Through a natural-language chat in OpenSearch Dashboards, it reads the behavior you have captured to find where ranking falls short, proposes tuning strategies from what it finds, and validates them against offline evaluation sets before anything ships. Tuning that used to take a search expert weeks becomes a conversation, and it keeps you as the decision-maker rather than changing your ranking behind your back. The behavior stream this article is about is the fuel: the richer your UBI data, the more the agent has to reason over.

Start recording

The behavior data you need to improve search is being generated right now, on every query your users and agents run, and it disappears the moment you fail to record it. UBI is the path to keeping it: an open schema, a client collector, and a supported collection path in Amazon OpenSearch Service, all landing in indices you already know how to query. Instrument the queries that matter, watch what users and agents do with the results, and let the grades they have been giving you all along drive what you ship next.

From 1.3 TB to 66 GB: On-Disk Vector Search in OpenSearch

Jon Handler — Thu, 08 Oct 2026 16:21:14 +0000

The last turn of the dial. Move the full-precision vectors to disk, stack a few more savings, and pay for the latency you actually need.

The first two articles in this series shrank the vectors and kept them in memory. 1.3 TB of RAM for 100M Vectors? Shrink Them Before They Hit OpenSearch covered pre-quantized vectors, where you convert to FP16, byte, or binary before indexing. Native Quantization: Let OpenSearch Service Compress Your Vectors handed that conversion to the engine at index time, from 16-bit scalar quantization down to 32x with optimized scalar quantization. Both kept the working set in RAM, where search answers in milliseconds.

This article takes the last turn of the dial: it moves the full-precision vectors to disk. Amazon OpenSearch Service keeps a compressed copy in memory and searches it first, then reads the full-precision vectors from disk to rescore the best candidates. For a replicated index of 100 million embeddings at 1,536 dimensions, that drops resident memory from about 1.3 TB, the best-latency deployment with everything in RAM, to roughly 66 GB. The price is some added search latency. A few stacking techniques, covered later, trim the footprint further.

On-disk mode: compressed in memory, full precision on disk

Disk-based vector search, introduced in OpenSearch 2.17, runs a search in two phases. The compressed vectors stay resident in memory, and OpenSearch searches them first to gather more candidates than you asked for. Then it loads the full-precision vectors for those candidates from disk and rescores them, returning the top k by true similarity. The large full-precision copy lives on the secondary storage tier, not in RAM, which is where the memory savings come from. Rescoring is on by default, with an oversample factor of 2.0 at the default 32x compression, so recall stays strong despite the compressed first pass.

You enable it with a single field-level parameter, mode set to on_disk. By default that selects the Faiss engine, the HNSW method, and 32x compression, which routes through the optimized scalar quantizer OpenSearch uses at that level, bringing a replicated 100-million-vector index down to roughly 66 GB resident. The mode works with the float and half_float data types. To trade some of the savings back for recall, lower the compression with compression_level; at 4x, OpenSearch uses the Lucene engine.

PUT /my-ondisk-index
{
  "settings": { "index.knn": true },
  "mappings": {
    "properties": {
      "embedding": {
        "type": "knn_vector",
        "dimension": 1536,
        "space_type": "l2",
        "data_type": "float",
        "mode": "on_disk"
      }
    }
  }
}

Latency is the thing to measure. In-memory search runs every comparison at RAM speed. On-disk search adds a second pass that reads full-precision vectors from the storage tier, so SSD throughput sets the pace. In the OpenSearch project's disk-based search benchmark on 113 million 1,024-dimension vectors, on-disk search at 32x answered in about 100 milliseconds at p90 with a single client, compared with about 24 milliseconds for the uncompressed in-memory index. Latency climbs as concurrency rises, so test at your real query load. For RAG, agent, and batch workloads that is usually a good trade. For the tightest user-facing latency budgets, keep the working set in RAM with one of the in-memory options from the earlier articles.

A useful pattern runs two indexes on one cluster: an in-memory quantized index for interactive, user-facing queries, and an on-disk index for batch or background work such as overnight RAG precomputation or periodic analytics over the full corpus. Each is sized for its own latency target, so you are not paying RAM prices for queries that can wait.

Derived source: cut disk storage, not just RAM

Derived source saves disk, not memory. Everything else in this series shrinks what sits in RAM; this setting shrinks what sits on disk. Without it, OpenSearch keeps a full copy of every vector in the stored _source field, alongside the vector data the search structures already hold, so each vector is stored twice. Derived source drops that copy from _source and rebuilds the values on demand, so the update, update_by_query, and reindex APIs keep working. Less data on disk also makes search faster: in the OpenSearch benchmarks for derived vector source, storage dropped by 3x, force merges ran about 10% faster, and search latency on the Lucene engine fell by 90%, because less unneeded vector data moves through the page cache. Derived source is on by default for vector indexes on OpenSearch 3.0 and later, and it matters most for on-disk indexes, where the full-precision vectors already occupy the storage tier. For an index created before the default, check index.knn.derived_source.enabled; the setting is static, so turning it on means reindexing.

Serving indexes larger than memory, handled for you

You do not have to load an entire Faiss index into memory to search it. Memory-optimized search, introduced in OpenSearch 3.1, memory-maps the index file and lets the operating system file cache serve reads, so a node can hold indexes larger than its physical memory and repeated queries come from cache. When you choose on-disk mode, or 32x compression with the optimized scalar quantizer, OpenSearch applies it for you behind the scenes. You can also turn it on explicitly for any Faiss HNSW index with a single index setting. On instances with large local memory and NVMe storage, memory-mapping raises effective capacity per node, with occasional p99 variability when a read misses the cache. The setting is static, so for an existing index you close the index, update the setting, and reopen it. It applies to Faiss HNSW only; IVF and product quantization indexes still load into memory.

PUT /my-vector-index
{
  "settings": {
    "index.knn": true,
    "index.knn.memory_optimized_search": true
  },
  "mappings": {
    "properties": {
      "embedding": {
        "type": "knn_vector",
        "dimension": 1536,
        "space_type": "l2",
        "method": { "name": "hnsw", "engine": "faiss" }
      }
    }
  }
}

IVF with product quantization for billion-scale

HNSW is the default graph for most workloads, but at billion scale even a quantized graph can exceed the memory budget. Faiss offers an alternative: an inverted file (IVF) index combined with product quantization. IVF partitions the vector space into clusters with k-means and searches only the clusters near the query, while product quantization encodes each vector into a compact set of codes. There is no navigation graph to hold, so the footprint is smaller than HNSW, at the cost of a training step to learn the cluster centroids and codebooks, and typically lower recall than HNSW at the same latency. Reach for IVF with product quantization when you have billions of vectors and cost is the binding constraint. IVF is available only on the Faiss engine; the Lucene engine supports HNSW and flat indexes but not IVF. Like all product quantization, it requires training before you index.

Reminders from earlier in the series

A few techniques from the earlier articles stack with everything here. When a filter narrows the candidates to roughly 100,000 vectors or fewer, exact k-NN skips the graph entirely and returns perfect recall with no graph to hold in memory. On nodes dedicated to vector search, raising the k-NN memory circuit breaker, knn.memory.circuit_breaker.limit, above its 50% default fits more vectors per node before you add hardware. And GPU-accelerated graph construction cuts build times for full-corpus rebuilds and embedding-model upgrades; see the GPU acceleration write-up.

The full spectrum, end to end

Here is the whole dial across the three articles, measured against the same baseline: 100 million vectors at 1,536 dimensions, HNSW with m of 16, one replica. RAM figures are the resident footprint with the replica. Dollar figures are order-of-magnitude on demand before reserved-instance or savings-plan discounts, meant to show relative magnitude, not quote a price.

Method	Resident RAM (with replica)	p50 latency	Recall
FP32 baseline	~1.3 TB	single-digit ms	reference
FP16 / SQ 16-bit	~656 GB	single-digit ms	indistinguishable from FP32
INT8 / byte	~341 GB	single-digit ms	0.99 on Euclidean sets
SQ 4-bit (8x)	~184 GB	single-digit ms	0.94-0.98 with rescoring
SQ 2-bit (16x)	~105 GB	single-digit ms	moderate, benchmark it
OSQ 1-bit (32x)	~66 GB	single-digit ms	0.94-0.98 with rescoring
Product quantization	~42 GB	single-digit ms	moderate, codec-dependent
On-disk (32x default)	~66 GB resident + FP32 on disk	~100 ms p90 (benchmark)	strong, two-phase rescoring

Dimensionality reduction stacks on every row. If your embedding model emits shorter vectors, or you apply principal component analysis or Matryoshka truncation, cutting 1,536 dimensions to 768 roughly halves each RAM figure again.

Wrapping up the series: pick your point on the dial

Across all three articles, the choice comes down to a tradeoff among cost, latency, and accuracy. You can hold any two steady by giving up some of the third. Every step down the table saves memory and pays for it in recall or in latency, and oversampling with rescoring buys recall back at a latency cost. No setting is right for every workload; the right one is the point where your recall and latency requirements still hold at the lowest cost.

Two settings make good middle grounds. FP16, either pre-quantized half_float vectors or 16-bit scalar quantization in the engine, halves memory to about 656 GB with recall effectively indistinguishable from full precision and no change in latency. It is the lowest-risk first step. At the other end, 32x compression on disk takes the same index to about 66 GB resident, around a twentieth of the baseline, and two-phase rescoring against the full-precision vectors keeps recall strong, around 95% recall@100 in the OpenSearch disk-based search benchmark. The cost is latency, about 100 milliseconds at p90 in that benchmark, which suits RAG, agent, and batch workloads. Between the two, you can cover a wide range of workloads without exotic configuration.

You do not have to walk the dial by hand. Auto-optimize for Amazon OpenSearch Service takes your vectors from Amazon S3, along with the recall and latency you need, and evaluates HNSW parameters, quantization, and rescoring settings against them. It returns up to three recommended configurations, typically within 30 to 60 minutes. The jobs run on separate infrastructure that OpenSearch Service manages, so they don't consume resources on your domain, and each job is billed at a flat rate. That makes it a simple, cost-effective way to explore the dial on your own data before you commit to a configuration.

Whichever setting you choose, measure it. Start from a full-precision baseline on your own evaluation queries, record recall and p50 and p95 latency, and compare every option against it. The right point moves as the corpus grows, the embedding model changes, and the accuracy requirement shifts, so build recall measurement into the deployment pipeline and re-check when the inputs change.

Native Quantization: Let OpenSearch Service Compress Your Vectors

Jon Handler — Tue, 06 Oct 2026 21:51:48 +0000

Send FP32, get 2x to 32x compression, and leave your ingestion pipeline untouched. The engine does the work.

The previous article in this series put the quantization work on you. You convert vectors to a reduced precision before indexing, and Amazon OpenSearch Service stores exactly what you send. That gives you full control, and it gives you a standing job: pick a method, run the conversion in your pipeline, and re-check accuracy every time your embedding model changes.

There is another way. OpenSearch Service can compress at index time. You send the full-precision FP32 vectors your embedding model already produces, and the engine converts each one to a lower-bit representation during ingestion and stores the compressed vectors in the index. Both the Faiss and Lucene engines offer native quantization, each with its own methods. Your ingestion pipeline does not change. The control over the accuracy tradeoff moves out of your code and into index configuration.

The baseline is the same one from the previous article. At 100 million vectors of 1,536 dimensions, full-precision FP32 needs about 643 GB for one copy, roughly 1.3 TB with a replica. That figure is the best-latency deployment, holding every vector in memory; with memory-optimized loading you can serve the same index in less RAM and page from disk at some latency cost. Every option here reduces the footprint while leaving ingestion identical. This is the second of three articles on the quantization spectrum in OpenSearch Service. The first covered pre-quantized vectors. The third covers on-disk mode and cost stacking.

Scalar quantization: the engine compresses each dimension

Faiss scalar quantization is the native path you reach for first. You set the encoder on the knn_vector field to sq and give it a bits value, and OpenSearch Service converts each FP32 dimension to the target precision during ingestion, storing the quantized vectors in the index. Valid bit widths are 16, 4, 2, and 1, spanning the range from near-lossless to aggressive compression.

At 16 bits, scalar quantization halves memory, the same reduction as pre-quantized FP16, dropping a replicated 100-million-vector index to about 656 GB, without a conversion step in your pipeline. OpenSearch stores the 16-bit vectors and builds the HNSW graph over them, and at search time it computes distances directly on the 16-bit values, accelerated by SIMD instructions on recent processors, so there is no round trip back to full precision. The type parameter selects the 16-bit format: fp16, the default, for the highest precision within a plus or minus 65,504 range, or bf16 (introduced in OpenSearch 3.9) for the full 32-bit value range at slightly lower precision. A companion clip parameter governs fp16 only: left at its default it rejects any vector with a value outside the fp16 range, and set to true it rounds out-of-range values to the limits instead. Both type and clip apply to 16-bit quantization only.

PUT /my-sq-index
{
  "settings": { "index.knn": true },
  "mappings": {
    "properties": {
      "embedding": {
        "type": "knn_vector",
        "dimension": 1536,
        "space_type": "l2",
        "method": {
          "name": "hnsw",
          "engine": "faiss",
          "parameters": {
            "m": 16,
            "ef_construction": 256,
            "encoder": { "name": "sq", "parameters": { "bits": 16 } }
          }
        }
      }
    }
  }
}

The low-bit end: 1, 2, and 4-bit quantization

Below 16 bits, scalar quantization reaches the aggressive end of the range. Multi-bit scalar quantization added two-bit (16x) and four-bit (8x) options for Faiss in OpenSearch 3.9, and one-bit (32x) has been available since OpenSearch 3.6. These bit widths are supported for the HNSW method. You configure the multi-bit options by setting the sq encoder bits to 2 or 4, or compression_level on the field to 8x or 16x. For the 32x end, set compression_level to 32x and let OpenSearch choose the algorithm, which since 3.7 is the optimized scalar quantizer described below. At 32x, a replicated 100-million-vector index drops to about 66 GB, from the 1.3 TB FP32 baseline. Collapsing each dimension toward a single bit discards magnitude, so benchmark against your own data; at 32x the optimized scalar quantizer recovers much of that recall for you, as the next section describes.

PUT /my-sq-2bit-index
{
  "settings": { "index.knn": true },
  "mappings": {
    "properties": {
      "embedding": {
        "type": "knn_vector",
        "dimension": 1536,
        "space_type": "l2",
        "method": {
          "name": "hnsw",
          "engine": "faiss",
          "parameters": {
            "encoder": { "name": "sq", "parameters": { "bits": 2 } }
          }
        }
      }
    }
  }
}

The 32x path: optimized scalar quantization on both engines

At the 32x end of the dial, OpenSearch Service uses optimized scalar quantization (OSQ), a RaBitQ-derived technique that compresses float32 vectors down toward a bit per dimension while holding recall far better than naive binary rounding. Where naive binary quantization centers each vector on the global centroid and rounds, OSQ adds a calibration step: it computes statistics, optimizes the quantization intervals, and keeps small corrective factors with each vector that sharpen the distance estimates. The payoff is a better compression-to-quality ratio at the same 32x reduction. You do not pick the algorithm by hand. Starting in OpenSearch 3.7, asking for 32x compression selects OSQ automatically, and OpenSearch applies it across both the Faiss and Lucene engines, handling the underlying mechanism for you. Set the compression level and let the engine choose.

The reason OSQ holds recall at roughly a bit per dimension is a set of techniques OpenSearch applies for you, with no knobs to turn. It keeps small corrective factors with each quantized vector so the first-pass distances stay close to the true ones. It uses asymmetric distance computation, keeping the query vector at full precision and rescaling it to compare meaningfully against the compressed document vectors, so the query side keeps information the documents gave up. And it applies a random rotation that spreads variance across dimensions, so more signal survives the compression. Each of these was once a separate option to wire up by hand; at 32x, OpenSearch does all of it under the covers.

PUT /my-osq-index
{
  "settings": { "index.knn": true },
  "mappings": {
    "properties": {
      "embedding": {
        "type": "knn_vector",
        "dimension": 1536,
        "space_type": "l2",
        "compression_level": "32x"
      }
    }
  }
}

Product quantization, briefly

Product quantization (PQ) is the one native method that works differently enough to call out. Instead of reducing the precision of each dimension, PQ splits each vector into m subvectors and encodes each one against a learned codebook, reaching compression ratios beyond what scalar quantization offers. The cost is a training step: PQ runs k-means over a representative sample to build the codebooks before you can index, which scalar quantization does not require. That training requirement, plus a larger accuracy gap at high compression, makes PQ a fit for very large corpora, hundreds of millions to billions of vectors, where the savings justify the added operational step. For the parameters and the training workflow, see the OpenSearch product quantization documentation.OpenSearch product quantization documentation

Oversampling to recover accuracy

The low-bit methods search over reduced-precision vectors, so the first-pass scores are approximate. Across these methods, you raise recall at query time with oversampling: set an oversample_factor and OpenSearch retrieves that multiple of k candidates before ranking, trading latency for accuracy, with 2x to 4x a reasonable starting point. The same rescore.oversample_factor parameter applies whether you are running scalar quantization or the optimized scalar quantizer at 32x. Measure recall and latency together on your own evaluation set, because the right oversampling depends on your data and your bit width.

GET /my-sq-index/_search
{
  "size": 10,
  "query": {
    "knn": {
      "embedding": {
        "vector": [0.1, 0.2],
        "k": 10,
        "method_parameters": { "ef_search": 100 },
        "rescore": { "oversample_factor": 3.0 }
      }
    }
  }
}

When exact k-NN is the better call

In-engine quantization still builds and searches an HNSW graph, and that graph is what holds memory. When your queries pre-filter to a small candidate set, you can skip the graph entirely. Exact k-NN scores the query against every vector that passes the filter, with no graph to build or hold in RAM, and when a structured filter narrows the field to roughly 100,000 vectors or fewer it returns perfect recall at latency competitive with the approximate path. It stacks with quantization too: a quantized exact scan over a filtered subset is both small and precise. Reach for exact k-NN when the access pattern is search within this tenant, this category, this date range rather than search the whole corpus. The graph earns its memory only when the candidate set stays large.

Pick the method, keep the pipeline

The native methods trade a little configuration for a large cut in memory while your ingestion stays exactly as it was. Scalar quantization at 16 bits halves memory at recall close to full precision, and the 2- and 4-bit settings take you to 16x and 8x. At the 32x end, you set the compression level and the optimized scalar quantizer takes over on either engine, recovering recall with corrective factors, asymmetric distance computation, and random rotation that you never configure. Product quantization serves the largest corpora, and oversampling tunes recall across all of them. When your queries filter down to a small set, exact k-NN sidesteps the graph cost altogether.

The next article moves from compressing the vectors to moving them off memory. On-disk mode keeps a compressed graph in RAM and the full-precision vectors on disk, with a two-phase search that rescores against those full-precision vectors, and cost-stacking techniques layer on top of any method here for more savings.

1.3 TB of RAM for 100M Vectors? Shrink Them Before They Hit OpenSearch

Jon Handler — Fri, 02 Oct 2026 13:51:05 +0000

FP16, INT8, and binary. Pick your spot on the accuracy-cost dial before Amazon OpenSearch Service ever sees the vector.

If you use vectors for search, you are always trading off cost, latency, and accuracy. OpenSearch Service gives you a set of knobs to adjust that balance, depending on where you want the emphasis. The one that dominates the bill is memory, because the approximate-nearest-neighbor graph that makes vector search fast lives in RAM, and RAM is the resource you provision the most of. How much you need comes down to one choice you make before indexing a single document: how much precision each vector carries.

At 100 million vectors of 1,536 dimensions, you need about 643 GB for a single copy. Run it with one replica, which most production deployments do, and the resident footprint is about 1.3 TB. That full footprint is what you deploy to hold every vector in memory for the best latency; with memory-optimized loading you can run in less RAM and page from disk on demand, trading some latency for a smaller bill. That number is not fixed either way. You decide how much of it to pay for, by choosing how much precision each vector carries before OpenSearch Service ever stores it, and you make that choice knowing what accuracy it costs.

Quantization is a dial with a continuous range of settings, each trading a measured amount of accuracy and latency for a lower memory bill. This is the first of three articles walking the full spectrum available in Amazon OpenSearch Service. Here you shrink the vectors yourself before they reach the engine. The next two cover OpenSearch-native quantization, where the engine compresses for you, and on-disk mode, where a compressed copy stays in memory and the full-precision vectors move to disk for a second pass that restores accuracy.

The baseline: full-precision FP32

Every compression option is measured against the full-precision baseline, so start there. A single FP32 vector at 1,536 dimensions holds 1,536 values of 4 bytes each, which is 6,144 bytes. The OpenSearch estimate for Faiss HNSW adds the graph that connects those vectors and multiplies the whole by a storage factor: 1.1 times the quantity 4 times dimension plus 8 times m. The 4 times dimension term is the raw vector, 4 bytes per FP32 dimension. The 8 times m term is the graph: m is how many neighbor links HNSW keeps per vector (m defaults to 16), and each link is stored as an 8-byte node identifier, so the graph adds 8 times m bytes per vector. The 1.1 is the source-to-index ratio, the stored index running about ten percent larger than the raw input.

memory_per_vector = 1.1 x (4 x dimension + 8 x m)
                  = 1.1 x (4 x 1,536 + 8 x 16)
                  = 1.1 x (6,144 + 128)
                  = ~6,899 bytes/vector

Per copy:        100,000,000 x 6,899 bytes = ~643 GB
With 1 replica:  643 GB x 2               = ~1,285 GB (~1.3 TB)

The raw vectors dominate that number. A replica holds a full second copy of the graph in memory for availability and query throughput, so every footprint figure in this article is per copy, and you double it for one replica. 1.3 TB is the resident anchor for a replicated 100-million-vector index. Every option below reduces it.

Turn 1.3 TB into instances to see the bill. Take the r8gd.2xlarge, with 64 GB of RAM. By default OpenSearch Service gives half of that to the JVM heap, leaving 32 GB off-heap, and the k-NN memory circuit breaker (knn.memory.circuit_breaker.limit) caps native vector memory at 50% of what remains, about 16 GB per instance. At 16 GB usable, 1.3 TB needs roughly 80 instances. If the node is dedicated to vector search and not running heavy text queries that lean on the heap, raise the circuit breaker to 75% of off-heap memory, which lifts usable vector memory to about 24 GB per instance and brings the count down to roughly 54 instances. On demand, 54 to 80 r8gd.2xlarge instances runs on the order of $31,000 to $46,000 per month, before reserved-instance or savings-plan discounts. Treat the dollars as order-of-magnitude. The point is that full precision at this scale is a five-figure monthly line item, and the dial below moves it.

Pre-quantized vectors: you shrink, OpenSearch stores

The first category of compression happens before your vectors reach OpenSearch Service. You, your embedding model, or a step in your ingestion pipeline produce vectors in a reduced-precision format, and OpenSearch Service stores and searches exactly what you send. The engine does no conversion here. The appeal is control: you pick the method, you measure recall against your own evaluation set, and nothing about the accuracy tradeoff is hidden. Three formats are available, and they sit at different points on the dial: FP16, INT8, and binary. The footprints below are per copy; double each for a replica, the same as the baseline.

FP16: half the bytes, nearly identical recall

Half-precision floating point stores each dimension in 16 bits instead of 32. Per-vector size drops by exactly half, from 6,144 to 3,072 bytes, and the 100-million-vector footprint falls from about 643 GB to roughly 328 GB per copy, about 655 GB with a replica. Many embedding models carry little useful signal in the low-order bits of each FP32 dimension, so truncating to FP16 costs almost nothing in quality. OpenSearch benchmarks put FP16 recall at effectively indistinguishable from full precision, because most datasets do not use the full 32-bit range. Some models emit FP16 natively, in which case there is no conversion step to add.

In OpenSearch Service, the data type for native FP16 is half_float, introduced in OpenSearch 3.9 and supported for the HNSW method on both the Faiss and Lucene engines. You ingest and query half_float vectors the same way as float vectors, and each dimension must fall within the representable range of roughly plus or minus 65,504. The mapping:

PUT /my-vector-index
{
  "settings": { "index.knn": true },
  "mappings": {
    "properties": {
      "embedding": {
        "type": "knn_vector",
        "dimension": 1536,
        "data_type": "half_float",
        "space_type": "l2",
        "method": {
          "name": "hnsw",
          "engine": "faiss"
        }
      }
    }
  }
}

INT8 byte vectors: a quarter of the footprint

Byte vectors store each dimension as a signed 8-bit integer in the range -128 to 127. At one byte per dimension, a 1,536-dimension vector occupies 1,536 bytes, a quarter of the FP32 baseline, and the footprint drops to roughly 170 GB per copy, about 341 GB with a replica. The cost is a conversion step: you map your FP32 embeddings into the byte range before indexing. A linear scaling from the observed min and max across a representative sample works for many models, and calibrated or per-channel approaches narrow the accuracy gap further. OpenSearch benchmarks on byte vectors report recall@100 of 0.99 on Euclidean datasets, matching full precision, with angular datasets measured by cosine similarity giving up more and benefiting most from careful calibration. Measure it against your own queries before you commit.

PUT /my-byte-index
{
  "settings": { "index.knn": true },
  "mappings": {
    "properties": {
      "embedding": {
        "type": "knn_vector",
        "dimension": 1536,
        "data_type": "byte",
        "space_type": "l2",
        "method": {
          "name": "hnsw",
          "engine": "lucene"
        }
      }
    }
  }
}

Binary vectors: 32x smaller, a different bargain

Binary is the aggressive end of the pre-quantization dial. Each dimension collapses to a single bit, one if the value is non-negative and zero otherwise. A 1,536-dimension vector becomes 1,536 bits, which is 192 bytes, 32 times smaller than FP32, and the 100-million-vector footprint lands near 33 GB per copy, about 66 GB with a replica, small enough to fit on a single large instance either way. The tradeoff is that binary discards magnitude entirely. For high-dimensional embeddings whose vectors are well separated by angle, which is common for modern text embeddings, binary retains a surprising share of the original recall. OpenSearch large-scale tests with rescoring hold recall@100 in the 0.94 to 0.98 range across 8x, 16x, and 32x compression, close to the in-memory baseline. For embeddings that lean on subtle magnitude differences, the drop is steeper. Always benchmark against your own data.

When you convert to binary yourself and store the result, you pack the bits into bytes: eight bits become one int8 value, so the dimension you declare must be a multiple of 8, and the engine compares vectors with Hamming distance. This pre-converted path is Faiss-only. If you would rather hand FP32 vectors to the engine and let it compress them, OpenSearch does that for you at the aggressive end of the dial through the optimized scalar quantizer, which is the subject of the next article.

A production pattern pairs a fast binary first pass with a full-precision second pass that restores ordering. OpenSearch Service implements that pattern for you in on-disk mode rather than leaving you to wire up a separate full-precision field, so the mechanics are covered in the third article of this series.

When exact k-NN beats the graph entirely

Quantization assumes you are running approximate search over an HNSW graph, and that graph is the thing eating your RAM. If your queries pre-filter to a small candidate set, you may not need the graph at all. Exact k-NN scores the query against every vector that passes the filter, with no graph to build, hold in memory, or rebuild on merge. When a structured filter narrows the field to roughly 100,000 vectors or fewer, exact search returns perfect recall at latency that is competitive with the approximate path, and it stacks cleanly with quantization: a byte or binary exact scan over a filtered subset is both inexpensive and precise. Reach for exact k-NN when your access pattern is 'search within this tenant, this category, this date range' rather than 'search the whole corpus.' The graph earns its memory only when the candidate set stays large.

Dimensionality reduction stacks with all of it

Precision is one axis. Dimension count is another, and the two multiply. Reducing 1,536-dimension embeddings to 768, whether through a model that emits shorter vectors or a reduction step such as principal component analysis, halves memory on its own and composes with any format above. FP16 at 768 dimensions is half of FP16 at 1,536. Binary at 768 is half of binary at 1,536. The dial is really a surface, precision on one axis and dimensionality on the other, and you can move along both.

Pick your spot on the dial

Pre-quantized vectors are the easiest segment of the spectrum to reason about, because you own the conversion and you measure the result. For many workloads FP16 alone halves the memory bill at recall indistinguishable from full precision, and that is often the end of the story. Byte vectors take you to a quarter of the baseline when a few points of recall are acceptable. Binary takes you to a thirty-second when your embeddings tolerate it and a rescoring pass covers the rest. And when your queries filter down to a small set, exact k-NN sidesteps the graph cost altogether.

When you want more compression without managing the conversion yourself, OpenSearch Service will do the work. The next article covers OpenSearch-native scalar and binary quantization, where you send full-precision vectors and the engine compresses them at index time, with oversampling to recover accuracy. After that, on-disk mode keeps a compressed copy in memory and moves the full-precision vectors to disk for a second-pass rescore, the lowest-cost end of the dial.

Migrating From Keyword Search to Vector Search on OpenSearch: A Guide for Teams Still on Lexical-Only

Jon Handler — Tue, 29 Sep 2026 22:43:14 +0000

Your users learned to ask questions somewhere else, and now your agents are asking too. Your search bar is the last place that still wants keywords.

A citizen types "can I renew my passport if it expired two years ago" into a government services portal, and the search bar hands back every page that contains the words "passport," "renew," and "expired," ranked by how often those words appear. The one page that answers the question sits on the third results screen, if it surfaces at all. The citizen gives up and calls the contact center, which costs the agency real money on a query the website should have answered for free.

If you run search for an organization, you have watched some version of this play out. Your users arrive from ChatGPT, Perplexity, and Claude, where they ask full questions and get direct answers. Then they reach your search box and have to translate the question back into the two or three keywords they hope your index contains. The friction is not subtle, and it shows up in your analytics as abandoned searches, repeated queries, and support tickets for answers that already exist in your content.

The reason to move to vector search is not that the technology is new or interesting. The reason is that the cost of staying on keyword-only search now shows up on a budget line, and it is about to show up again as you put agents in front of your content. An agent answering a user's question runs its own searches, and it hits the same wall your users do when the index only matches exact terms. This guide covers when keyword search is actually failing you, what vector and hybrid search on Amazon OpenSearch Service change for both people and agents, and how to stage the migration without ripping out the system you already run.

When keyword search is actually failing you

Keyword search still does real work, and moving to Amazon OpenSearch Service does not mean giving it up. Exact product codes, known document titles, specific error strings, faceted navigation, and structured filters all run on BM25 ranking that returns the right result quickly and transparently, and OpenSearch Service runs that lexical path natively. The point is not to replace keyword search. It is to stop asking keyword search to answer the queries it was never built for.

The failure mode appears when queries stop looking like keywords and start looking like questions. A search for "vehicle hire" against a catalog that only says "car rental" returns nothing, because the two strings share no terms even though a person would treat them as the same request. Keyword search matches the characters the user typed against the characters in your index. When those two vocabularies diverge, the query fails silently, and the user rarely tells you. They just leave.

Pull your search logs and sort by zero-result and low-engagement queries, and the failures line up in a clear pattern. They cluster on natural-language phrasings, multi-part questions, and searches that use different words than your content authors used. Those are the queries bleeding engagement, and they are exactly the queries that keyword matching cannot rescue no matter how carefully you tune the ranking.

What vector and hybrid search change

Vector search matches on the company words keep rather than the exact characters a user typed. An embedding model maps text to coordinates so that phrases appearing in similar contexts across the training corpus land near each other. "Vehicle hire" and "car rental" end up close because they show up in overlapping surrounding text, not because a model comprehends either phrase. The practical effect is that a query retrieves relevant documents even when the author and the user chose different words for the same thing. Correlation over word usage, not understanding, is what closes the vocabulary gap.

Vector search alone overcorrects. Ask it for an exact product code and it may return a cluster of conceptually adjacent items while burying the precise match the user actually wanted. This is why production search combines both approaches. Hybrid search runs the keyword path and the vector path together and merges the results, so exact matches keep ranking high while related-but-differently-worded documents fill in the results a keyword-only system would have missed. OpenSearch Service runs hybrid search natively, and the balance between the two paths is a setting you tune to your traffic rather than a rewrite you commit to.

Increasingly this is not one-shot retrieval-augmented generation but agent-driven question answering. An agent decides how to answer a question: it can run several retrievals, refine them based on what came back, pull from more than one index, and compose an answer grounded in what it found. OpenSearch Service supports this directly with agentic search, where the agent chooses the lexical or vector path per query based on the query itself, using exact matching for a date filter or a product code and vector retrieval for a conceptual question. Grounding the agent in content it retrieves from your own indices is what keeps the answer built from your documents rather than from the model's training data alone. Retrieval quality sets the ceiling on answer quality, so the move to better search pays off twice: once in direct results and again in every agent you build on top of them.

How to stage the migration

Treat the move as an augmentation, not a rip-and-replace. Your keyword index keeps running and keeps serving the exact-match and filter traffic it already handles well. You add a vector path alongside it and route more query types through the hybrid pipeline as your confidence grows. Nothing about this requires a flag-day cutover, and the staged approach means you can measure each step against the system it replaces.

Start where keyword search fails hardest, which your zero-result logs already identified. Stand up an embedding pipeline in OpenSearch Service so documents get vectorized as they are indexed, point a hybrid query at the failing query set, and compare relevance against your current system on the same queries. The Search Relevance Workbench in OpenSearch Dashboards runs keyword and neural queries side by side against the same index, so you can see the difference on your own queries rather than take it on faith, and it is enough of a topic on its own that it deserves its own article. Amazon OpenSearch Service can host the embedding model inside the engine and vectorize both documents at index time and queries at search time, so you are not standing up a separate model-serving tier to get started. Once the hybrid pipeline wins on the queries that were failing, widen its coverage.

Measure the transition on user behavior, not just offline relevance scores. The catch is that most teams do not capture user behavior in any structured way, so the signal that would prove the migration worked never gets recorded. This is what User Behavior Insights (UBI) addresses: a standard schema that ties each query to the results shown and the actions the user took on them, so zero-result rate, search refinement, and click-through depth become data you can query rather than anecdotes. You can collect UBI-formatted data with Amazon OpenSearch Service and analyze it in the same engine that serves the search. Those metrics also make the business case for the next phase, because they translate search quality into the operational costs your organization already tracks, like the support contacts that duplicate content the site should have surfaced.

The cost of waiting

The gap between what your users expect and what your search bar delivers widens every day they spend in a conversational interface somewhere else. When one organization in your sector ships natural-language search, the expectation resets for everyone your users interact with next, including you. The migration stops being a project you might schedule and becomes a gap your users feel on every visit.

Amazon OpenSearch Service gives you a path that does not start with tearing down what you have. Keep the keyword search that works, add vector and hybrid search where questions have replaced keywords, and let agents run retrieval across your indices when you are ready to return answers instead of links. The technology is production-ready today. The queries your users and your agents are already typing are the ones telling you where to start.

OpenSearch Unifies the Retrieval Layer for Structured, Time Series, and Full Text Data

Jon Handler — Thu, 24 Sep 2026 20:07:35 +0000

Text, time series, and structured data in one engine. One query interface. One place for agents to look.

For a decade, search quality meant one thing: getting the right link into the top slot. Ten blue links. Click-through rate. Precision at rank one. The entire search industry organized itself around a single user staring at a results page, deciding which link to click. We got very good at that problem.

Agents changed the question. An agent does not click links. An agent synthesizes. It pulls a fact from a research document, a number from a time-series index, a constraint from a structured database, and weaves them into a response. The agent does not care whether the data came from a full-text index, a vector embedding, or an aggregation pipeline. It cares that the data is accurate, that the retrieval was fast, and that it did not have to call four different backends with four different query languages to assemble one answer.

That shift, from person-as-consumer to agent-as-consumer, changes what a search engine needs to be. It is no longer enough to do one type of retrieval well. The engine needs to unify text retrieval (structured and unstructured), time-series analytics, and hybrid search behind a single query interface. Amazon OpenSearch Service does all three.

The financial analyst's agent

Consider a financial services company building an agent to help its research analysts. The agent needs to answer questions like "What is the market sentiment on Acme Corp following their Q3 earnings, and how has the stock performed since the product launch in June?" That question touches four data types: news articles (unstructured text), internal research reports (semi-structured documents), transaction records (structured data), and stock price history (time series). Each data type lives in a different system today. The agent has to call each one, reconcile the results, and hope the latency budget survives.

OpenSearch Service consolidates those retrievals. The text data, the structured records, the time-series metrics, and the vector embeddings all live in the same engine. The agent can choose the right retrieval strategy per question: a pure lexical query when the analyst asks for a specific company by name, a semantic query when the question is conceptual, a hybrid query that combines both when the question has precise and fuzzy elements, or a multi-clause query that pulls text and time-series data together in a single call. The engine supports all of these through one query interface. The integration complexity that used to live in the agent's orchestration layer moves into the search engine, where it belongs.

Text retrieval: lexical, semantic, and hybrid together

Search engines have been refining text retrieval for more than thirty years. Inverted indices, BM25 scoring, boolean query algebra, stemming, synonyms, phrase matching, reranking: the toolbox is deep and battle-tested. OpenSearch inherits all of that. Lexical search through BM25 gives you precise term matching, boolean queries, function_score for custom relevance logic, and integration with reranking models. When the analyst searches for "Acme Corp revenue guidance," the lexical path finds documents that contain those exact terms. Lexical search remains the foundation: it handles exact matching, structured field filters, and queries where the user's words map directly to the vocabulary in the index.

Semantic search through vector embeddings finds results where the surrounding text contexts are similar, even when the exact words differ. If a user searches for "laptop" and your index contains "notebook computer," lexical search returns nothing because the terms do not overlap. Semantic search surfaces the match because "laptop" and "notebook computer" appear in overlapping lexical contexts across the training corpus. The match is based on distributional patterns in how words co-occur, not on shared terms. When the analyst searches for "market outlook," semantic search surfaces documents about "forward guidance" and "earnings projections" for the same reason. OpenSearch supports both exact and approximate nearest-neighbor search, with quantization methods and tiered storage that let you optimize vector costs as the collection grows. Auto-optimization tunes the model and indexing parameters so you do not have to.

Hybrid search combines both paths in a single query with per-clause weighting. This is the default for production retrieval today, and for good reason: the lexical leg catches exact terminology while the semantic leg catches contextual relevance. For agents, the per-clause weighting is especially useful. The agent can emphasize the lexical clause when the user asks for a specific company name and emphasize the semantic clause when the user asks a conceptual question. One query, adjustable emphasis.

Semantic search does not paper over weak lexical search. Builders today often hope that adding a vector path will compensate for a poorly tuned BM25 configuration, but hybrid search amplifies both signals. If the lexical side is noisy, the combined results are noisy. In our testing, tuning the lexical side of hybrid search produced significantly better results overall than tuning the semantic side alone. Both legs have to be good for hybrid to deliver.

Back to the financial analyst: the agent searches for information about Acme Corp's innovation strategy. The lexical clause matches documents mentioning "Acme Corp" by name. The semantic clause surfaces research reports about the company's R&D investments and patent filings, even when those reports never use the word "innovation." The combined result set gives the agent the full picture without requiring two separate queries.

Time series retrieval: aggregation and analysis

Time-series data gives agents the ability to aggregate, trend, and correlate. Stock prices, server metrics, transaction volumes, sensor readings: these are all sequences of values over time, and agents need to query them the same way they query text. OpenSearch Service provides an optimized engine for time-series data with fast ingestion, sub-second aggregations, and tiered storage for data that regulatory requirements demand you keep for years.

Agent observability is a natural fit here. The financial company wants to control its AI costs and understand how its agents perform. OpenSearch's observability stack ingests OpenTelemetry traces and metrics from the agent infrastructure. The company builds KPI dashboards around agent interactions: latency per tool call, token consumption per query, success and failure rates across agent-to-agent handoffs. The same engine that stores the analyst's research documents also stores the operational data about how the agent system itself is performing.

Hybrid queries can bring time-series data into the same result set as text retrieval. The analyst wants to see Acme Corp's stock performance over the last six months and correlate price movements with product launch dates. The time-series clause returns the aggregated price data. The text clause returns the news articles and press releases around each launch. The agent receives both in a single response and synthesizes the correlation without orchestrating separate backend calls.

Why OpenSearch fits

The information agents need spans every data type in the stack: unstructured text, structured records, vector embeddings, and time-series metrics. Most retrieval architectures treat these as separate concerns, each with a dedicated system. OpenSearch fits naturally because it handles all of them natively. Full-text search, vector search, hybrid search, sub-second aggregations over time-series data, and structured field queries all run inside the same engine, against the same indices, through the same query API.

That native breadth is what makes OpenSearch a natural retrieval layer for agents. The agent does not need to learn four query languages or stitch together results from four response formats. The agent issues one query with clauses that target different data types, and the engine handles the internal fan-out, score normalization, and result fusion. The agent gets back one ranked list and spends its compute on reasoning, not plumbing.

The value compounds as the data grows. Adding a new data type to the retrieval layer means adding an index and a query clause, not integrating a new backend. The financial company that started with research documents and stock prices can add regulatory filings, earnings call transcripts, and agent telemetry data to the same engine without changing the agent's retrieval architecture.

The engine backs that breadth with production-grade performance. Amazon OpenSearch Service delivers single-digit-millisecond query latencies, sustains thousands to tens of thousands of queries per second, and scales vector storage into the hundreds of billions of embeddings. Time-series ingestion runs at hundreds of thousands of events per second with sub-second aggregation response times. These are not theoretical limits. They are the operating parameters of production workloads running on the service today. When an agent needs to retrieve across text, vectors, and time series in one call, the latency budget has to hold for all three data types. OpenSearch Service holds it.

Where this is heading

The shift from people to agents as search consumers is still early, but the retrieval patterns are taking shape. Agents need unified retrieval across data types, not a patchwork of specialized backends stitched together with orchestration code. They need a single query interface that handles text, vectors, time series, and structured data with per-clause weighting so the agent can tune the emphasis on each data type per query.

OpenSearch Service delivers that unified interface at production scale. Lexical search, semantic search, hybrid search with reciprocal rank fusion, time-series analytics, observability, and tiered storage all run inside the same engine with single-digit-millisecond latencies and thousands of queries per second. The retrieval layer does not need to be assembled from parts. It is one service, one query language, one place for your agents to look.

Hybrid Search Explained: Combining Lexical and Semantic Search in OpenSearch

Jon Handler — Tue, 22 Sep 2026 16:23:51 +0000

Your keyword search is precise. Your vector search is contextual. Use both.

Hybrid search in Amazon OpenSearch Service combines BM25 keyword matching with vector similarity in a single query. You get the precision of lexical search (exact terms, filters, structured fields) and the contextual reach of semantic search (vocabulary mismatch handled, related terms surfaced) without choosing between them. As of OpenSearch 2.11, score normalization and combination run natively inside the search pipeline, so you do not need to build the merge logic yourself.

I ran an experiment recently that made the case for hybrid search more clearly than any architecture diagram could.

One Query, Three Approaches

I indexed a product catalog and searched for "women's shoes." With pure lexical search, the results included two women's shoes (matched on both words) and two men's shoes (matched on "shoes" alone). Technically correct: the query terms appeared in all four documents. But the user's intent was clearly gendered, and lexical search had no way to capture that.

With pure semantic search, all four results were boots. The vector space placed boots close to "shoes" based on co-occurrence patterns in the training corpus, which is accurate as far as distributional similarity goes. But the results lost variety entirely. If the user wanted sandals, flats, or sneakers, semantic search had narrowed the options to a single style.

With hybrid search, the top result was a boat shoe (strong lexical match on the query terms) and the remaining results included boots and other styles (semantic similarity expanding the candidate set). The user got what they searched for at the top, plus contextually related options below. Precision where it matters, discovery where it helps.

Score Normalization Is the Hard Part (and OpenSearch Handles It)

Combining lexical and semantic scores is not as easy as adding them together. BM25 returns scores from 1 to infinity. FAISS vector similarity returns scores between 0 and 1. Without normalization, the BM25 scores dominate every time, and the semantic signal disappears.

OpenSearch 2.11 introduced a score normalization processor in search pipelines that handles this at the coordinator level, globally across all shards. You configure the normalization technique (min-max or L2), the combination method (arithmetic mean, geometric mean, or harmonic mean), and the weight distribution between lexical and semantic scores. The OpenSearch team's benchmarks on public datasets show min-max normalization with arithmetic mean delivers strong results across a range of datasets, but the right settings depend on your data and your users.

I found equal weighting (0.5 lexical, 0.5 semantic) provided the best balance of relevance and discovery in my tests. Higher semantic weights pushed results toward the pure semantic outcome, which is useful when user intent is ambiguous but loses precision when the user knows exactly what they want. Lower semantic weights approach pure lexical behavior. The weight is a dial, and the right setting is a product decision, not a technical one.

Neither Lexical Nor Semantic Is Enough Alone

Lexical search is fast, interpretable, and works without any ML model. It handles exact matches, filters on structured fields, and generalizes across domains without fine-tuning. For queries where the user types the exact term that exists in your data, lexical search is already optimal. The failure mode is vocabulary mismatch: the user says "beach shoes" and your catalog says "water-resistant footwear."

Semantic search via vector embeddings handles vocabulary mismatch by correlating terms that co-occur in similar contexts across the training corpus. But vector search comes with costs: you need an embedding model (either hosted or via a service like Amazon Bedrock), the vectors consume memory, and the retrieval can over-cluster results around a narrow region of the vector space, as the boots example showed. Pure semantic search also loses the ability to do exact filtering and structured field matching that lexical search handles natively.

Hybrid search gives you both. Exact matches rank high because BM25 rewards them. Semantically related results fill in below because vector similarity surfaces them. Filters on structured fields (price range, category, availability) work through the lexical path. The combined result set is richer than either approach alone.

Implementing Hybrid Search in OpenSearch Service

The implementation runs through search pipelines. You create a pipeline with a normalization processor that specifies the normalization technique, combination method, and weights. At query time, you send a hybrid query that includes both a BM25 text match and a k-NN vector query. OpenSearch Service runs both queries in parallel, normalizes the scores using your configured method, combines them with your specified weights, and returns a single ranked result set.

The key configuration choices are the normalization technique and the weight distribution. For normalization, min-max rescales both score sets to a 0-1 range. L2 normalizes by the Euclidean magnitude of the score vector. In practice, min-max with arithmetic mean is a strong default. For weights, start at 0.5/0.5 and adjust based on how your users search: if most queries are specific product names, lean lexical; if most queries are natural language descriptions, lean semantic.

On the semantic side, you need an embedding model. OpenSearch Service integrates with Amazon Bedrock for embedding generation, or you can host your own model on SageMaker. The ingest pipeline with a text_embedding processor converts documents to vectors at index time. At query time, the same model converts the user's query to a vector for the k-NN search leg of the hybrid query.

Where Hybrid Search Fits

E-commerce search is the obvious case: users switch between exact product names and natural language descriptions within the same session. Internal knowledge bases are another strong fit, where users search for concepts they cannot name precisely. RAG retrieval pipelines benefit from hybrid search because the lexical path catches exact terminology (error codes, API names, product IDs) while the semantic path catches conceptual relevance.

If you are running pure lexical search today, adding the semantic path is where the lift comes from. If you have already invested in semantic search, adding the lexical path back in restores the precision you may have lost. Either way, hybrid search is the baseline for production search experiences now, not an advanced feature.

Start by running your current query set through a hybrid pipeline alongside your existing implementation. Measure not just relevance metrics but user behavior: click-through rates, time to conversion, search refinement patterns. The difference is usually visible within days.

Semantic Search Setup in Amazon OpenSearch Service: From Zero to Vector Search in 15 Minutes

Jon Handler — Thu, 10 Sep 2026 19:10:44 +0000

Tight integration brings the embedding model to the search engine. Vectors happen automagically.

If you have been considering semantic search, you already know the list of unknowns. Which embedding model do you choose? How do you host it? How do you get your documents converted to vectors and keep those vectors in sync as your data changes? How do you handle vectorization at query time without adding latency? And once you have vectors, how do you wire everything together so that your search index, your embedding model, and your ingest pipeline all talk to each other without a custom integration layer in between?

Those questions can take weeks of work to resolve, and for a lot of organizations they represent a huge hill to climb. Selecting and deploying an embedding model on SageMaker. Writing batch processing jobs to convert content into vectors. Building orchestration code to keep the ML pipeline and the search index in sync. Managing two separate systems that both need to know when your content or your model changes. The cognitive overhead alone can be a barrier, before you write a single search query.

The neural plugin in Amazon OpenSearch Service makes this dramatically easier. Through the OpenSearch Service console, you set up an ML connector that links your domain to an embedding model hosted on SageMaker or Amazon Bedrock. An ingest pipeline with a text_embedding processor then calls that model automatically every time you index a document, and queries convert to vectors at search time through the same connector. The embedding model still runs on SageMaker or Bedrock, but the integration, the vectorization pipeline, and the query-time conversion all live inside OpenSearch Service, so you are managing one workflow instead of stitching together three. If you want an even lighter path, Automatic Semantic Enrichment (which I wrote about in an earlier post) adds sparse vector enrichment at index time without deploying any model at all.

I set this up recently for a product catalog, and the whole process from zero to working semantic search took about fifteen minutes.

What the Old Playbook Looked Like

Without the neural plugin, implementing semantic search on OpenSearch Service means treating the search engine as a passive vector store. You deploy embedding models separately, build SageMaker processing jobs to convert your content into vectors, write orchestration code to call those endpoints, and manage the batch ingestion pipeline that feeds vectors into your index. When your content updates, both systems need to know. When your embedding model changes, you reprocess everything.

The architecture worked, but the operational tax was real. I spent more time debugging the plumbing between the embedding service and the search index than I spent on search relevance. The integration layer became its own project, and keeping the ML pipeline and the search index in sync consumed engineering cycles that had nothing to do with the actual search experience.

The Neural Plugin Changes the Contract

The neural plugin integrates embedding models directly into the search workflow. A pre-built CloudFormation template (available in the Integrations tab of the Amazon OpenSearch Service console) deploys the embedding model (Amazon Titan Text Embeddings through Amazon Bedrock, or a model on Amazon SageMaker), creates the IAM roles, and configures the ML connector in a single stack. The model runs on SageMaker or Bedrock, and the CloudFormation stack outputs an internal OpenSearch model ID that you use in subsequent steps. OpenSearch Service calls the model automatically at ingest and query time through the connector. With the model deployed, OpenSearch Dashboards provides a visual workflow experience for building the end-to-end AI search flow through the AI Search Flows interface (under OpenSearch Plugins). The same setup is available via the ML Commons API for automation.

When you create an OpenSearch ingest pipeline with a text_embedding processor, the neural plugin automatically calls your embedding model as documents enter the index. You specify which text fields should generate embeddings (product descriptions, document content, whatever drives your search experience), and the pipeline handles vectorization at ingest time. The entire workflow runs inside OpenSearch Service, so you are not maintaining batch jobs, custom code, or a sync layer between two systems.

At query time, the same model converts the user's search text into a vector, and OpenSearch Service runs a k-NN search against the stored embeddings. The entire semantic search workflow (text to vector to search to results) happens inside the search engine. You interact with plain text on both sides: index text documents, search with text queries. The vectors are internal.

What Semantic Search Actually Does to Your Results

I tested this against a product catalog. A keyword search for "what to wear in office" returned random clothing items: t-shirts, boots, undergarments. The query matched on individual words without any sense of the relationship between them. A semantic search for the same query returned dress suits, professional shoes, ties, and business attire. The embedding model had learned from training data that these terms co-occur in professional-clothing contexts, and the vector similarity surfaced results that lexical matching missed entirely.

For "accessory for hike," keyword search returned nothing useful. Semantic search returned backpacks, water bottles, and outdoor gear. The vocabulary mismatch between the query and the catalog vocabulary was total, and semantic search bridged the gap without a single synonym mapping.

The Search Relevance Workbench in OpenSearch Dashboards lets you run keyword and neural queries side-by-side against the same index. When stakeholders ask whether semantic search justifies the investment, the results comparison makes the case directly. The difference between matched-on-keywords and matched-on-co-occurrence-patterns is visible in the first three results. OpenSearch Dashboards also includes a visual builder for these AI-powered search flows, so you can configure and test ingest pipelines, ML connectors, and search pipelines through a graphical interface rather than writing JSON by hand.

Getting Started in 15 Minutes

Before you start, two prerequisites need to be in place. You need model access enabled in Amazon Bedrock for your chosen embedding model, and you need to add the Lambda invoke role as a backend role in your OpenSearch ML Commons configuration. Budget 10 minutes for this setup before you touch CloudFormation. With the prerequisites in place, deploy the CloudFormation template that provisions the embedding model, the ML connector, and the IAM roles. The stack outputs an internal OpenSearch model ID that you use for the rest of the setup. Create an ingest pipeline with the text_embedding processor, create an index that uses the pipeline, and start indexing documents.

The operational burden of adding ML-powered capabilities to search has dropped by an order of magnitude. The neural plugin, the CloudFormation templates, and the managed ML connectors are not doing anything fundamentally new. They are collapsing the integration layer that used to sit between your search engine and your embedding model. The search engine now handles both sides of the conversation.

If you have been putting off semantic search because the ML pipeline seemed too heavy, that calculation has changed. Deploy the CloudFormation stack, create an ingest pipeline, index a few hundred documents, and run the comparison tool. The difference in search quality is measurable in minutes, and the setup takes a fraction of the time the old approach required.

Enterprise RAG Without Per-Token Pricing: DeepSeek R1 on SageMaker with OpenSearch

Jon Handler — Tue, 08 Sep 2026 18:16:28 +0000

Search that actually thinks. Without the per-token invoice.

Open-source reasoning models have closed the gap with proprietary APIs. DeepSeek R1 on Amazon SageMaker delivers GPT-4-class reasoning at managed-service economics: no per-token fees, no rate limits, full control over data residency and model behavior. Pair that with Amazon OpenSearch Service as the vector database, and you have a RAG architecture where retrieval scales under enterprise workloads and generation does not send you a surprise invoice at the end of the month.

I deployed this stack recently and the experience changed how I think about RAG infrastructure. The reasoning quality is real. The cost structure is fundamentally different. And the integration between OpenSearch and SageMaker-hosted models is tighter than I expected.

Retrieval Is Half the Problem

When I started building RAG systems, I assumed retrieval was the whole problem. Get the right documents in front of the model, and the model handles the rest. I spent weeks fine-tuning embeddings, vector similarity metrics, and index configurations. The retrieval was excellent. The answers were still wrong.

Perfectly retrieved context fed into a model that could not reason about what it was reading produced confident, well-formatted nonsense. The model saw the right documents and still could not connect the dots: numerical reasoning failed, multi-hop questions produced hallucinations, and edge cases where the answer was "the context does not say" got fabricated responses instead. RAG is not one problem. It is three: retrieval that is fast and accurate, reasoning that is reliable, and infrastructure that does not require a dedicated team to operate. Solve any two and you still fail.

For a while, the only models that could reason well enough for production RAG were proprietary APIs. GPT-4 and Claude gave good answers but came with per-token pricing, rate limits, and data residency questions that enterprise customers could not always resolve. The open-source alternatives were inexpensive but could not handle the reasoning step reliably. DeepSeek R1 changes that tradeoff.

DeepSeek R1 on SageMaker, OpenSearch as the Vector Store

Unlike black-box APIs, DeepSeek R1 shows its reasoning process. When you ask it a question with retrieved context, it thinks through the problem step by step before answering. For RAG, this is valuable: you can see whether the model is using your retrieved documents or going off the rails. I deployed the 14B parameter distilled variant on SageMaker JumpStart in under 10 minutes. The instance (ml.g5.12xlarge) gives you the GPU memory the model needs, and SageMaker handles the deployment lifecycle.

On the retrieval side, Amazon OpenSearch Service handles storage and search. OpenSearch Service stores your documents and their vector embeddings, runs k-nearest-neighbor search when a query arrives, and returns the top matches. I used a separate, lightweight embedding model (all-MiniLM-L6-v2, also on SageMaker) for vectorization. Keeping the embedding model and the generation model on separate endpoints lets you optimize cost and latency independently: the embedding model runs on a smaller instance, and the generation model scales based on query volume.

At ingest time, an OpenSearch pipeline with a text_embedding processor ties the two sides together. When you index a document, the pipeline automatically calls your embedding model and stores the vector alongside the source text. Your embeddings stay in sync with your data without batch jobs or manual re-embedding. At query time, OpenSearch converts the user's question to a vector, runs a k-NN search, retrieves relevant documents, and passes them to DeepSeek R1 through a retrieval-augmented generation processor in the search pipeline. The model receives the context with instructions to answer based only on what it was given.

From Query to Answer (and the Setup Tax to Get There)

I indexed a small dataset and asked: "what is the population increase of New York?" OpenSearch Service retrieved the relevant documents about New York population data from 2021-2023, sent them to DeepSeek R1 with clear instructions, and returned an actual answer with the specific number and its source, not a ranked list of documents for me to read through.

Beyond the answer itself, the reasoning trace is visible. DeepSeek R1 shows how it arrived at the answer: which documents it considered, which facts it extracted, and how it combined them. When the model gets it wrong, you can see exactly where the reasoning broke down. That debuggability is something you do not get from a proprietary API where the inference is a black box.

Compared to a managed API call, the setup takes more work. Your OpenSearch Service cluster needs IAM permissions to invoke your SageMaker endpoints. You need ML connectors that define how OpenSearch Service talks to your models. You need the ingestion pipeline configured with the right embedding model and processor chain. The first time through, it feels like you are connecting a dozen different services. I used an AI coding assistant to generate the IAM role creation, trust relationships, and role mappings, which cut what used to be a week of manual configuration down to about 20 minutes.

After the first deployment, every subsequent project reuses the same patterns. The IAM configuration is the same for any SageMaker-backed model. The ingest pipeline structure is the same whether you are embedding product descriptions or legal documents. The search pipeline with the RAG processor follows the same template. You solve the infrastructure puzzle once and reuse it across projects.

Per-Instance, Not Per-Token

With a proprietary API, you pay per token. Every query incurs a generation cost that scales linearly with volume. For applications with unpredictable or high query volumes (customer support, internal knowledge bases, research tools), the per-token model makes cost forecasting difficult and cost spikes real. With DeepSeek R1 on SageMaker, you pay for instance hours. The model runs on your infrastructure, and the cost is predictable regardless of query volume. For high-volume applications, the unit economics shift in your favor quickly.

On the vector database side, OpenSearch Service cost scales with data volume and cluster size, not with query count. Together, the stack gives you a RAG architecture where the bill is a function of infrastructure, not usage. For enterprise applications where query volume is the whole point, that distinction matters.

If you are evaluating RAG architectures, start small: a few hundred documents, a handful of test queries, and the DeepSeek R1 distilled model on SageMaker. See whether the reasoning quality meets your bar. See whether the infrastructure complexity is manageable for your team. The entire setup (SageMaker endpoint, OpenSearch domain, ingest pipeline, search pipeline with RAG processor) can run as a proof of concept in an afternoon.

The question for most organizations is not whether RAG is useful. It is whether you can build a RAG system where the reasoning is reliable, the retrieval scales, and the cost does not grow linearly with every question your users ask. Open-source reasoning models on managed infrastructure, paired with a vector database built for enterprise workloads, give you a path to all three.

OpenSearch Service GPU Acceleration and Auto-Optimization: vector search with ease

Jon Handler — Thu, 03 Sep 2026 19:07:15 +0000

Without GPU acceleration, a billion-vector reindex on general-purpose CPUs takes more than 24 hours. With GPU acceleration on Amazon OpenSearch Service, that drops to under an hour. That changes what is possible. You can swap embedding models and fully reindex without scheduling a maintenance window. You can experiment with different vector dimensions and quantization strategies and see results the same afternoon. You can keep your search index current as your data changes daily instead of treating a reindex as a quarterly infrastructure event. The speed unlocks iteration, and iteration is where search quality actually improves.

Before GPU acceleration and auto-optimization arrived, getting to production was the hard part. Vector search at scale requires choosing between graph-based and bucket-based algorithms, picking a quantization method (product, binary, scalar, ranging from 2x to 64x compression), deciding between in-memory and disk-based storage, and tuning dozens of hyperparameters. Each choice cascades into three more decisions. Until recently, figuring out the right combination for your data meant weeks of manual experimentation, where each iteration took 6-12 hours and cost hundreds of dollars in compute.

Instead of running that cycle yourself, auto-optimization in OpenSearch Service runs it for you. Instead of manually testing configurations, you tell the system what recall and latency you need, and the system runs hyperparameter optimization against your actual vector data on a serverless fleet. Within an hour, you get ranked recommendations with detailed performance metrics: memory footprint, expected latency, recall rates, and cost. You are choosing between business tradeoffs, not debugging algorithm parameters.

The Configuration Problem (And Why You Should Not Be Solving It)

I walk customers through k-NN algorithm tradeoffs regularly. HNSW ef_construction values, FAISS IVF cluster counts, binary quantization with rescoring, product quantization with different subvector configurations. The conversations were useful, but they always ended the same way: the customer spent weeks running experiments and still was not confident the result was optimal for their data.

Rather than asking you to become an algorithm expert, auto-optimization in Amazon OpenSearch Service takes the problem off your plate. You provide a sample of your vector data and specify what you actually care about: recall targets and latency thresholds. Behind the scenes, a serverless fleet runs hyperparameter optimization jobs against your actual data, testing algorithms, compression techniques, and storage modes. Within an hour, you get ranked recommendations. Each recommendation shows the memory footprint, expected latency, recall rate, and cost implications. You are choosing between business tradeoffs, not debugging hyperparameters.

Because each recommendation shows its tradeoffs explicitly, you are making business decisions, not algorithmic guesses. Recommendation one might use binary quantization with in-memory storage at a given cost. Recommendation three might use product quantization with slightly better recall but 2x the memory footprint. You can see why each option scores the way it does. The system is not a black box. It is a specialist that shows its work.

The Build Time Problem (And Why GPUs Change the Math)

Even with the right configuration, building a billion-vector index on general-purpose CPUs takes days. Graph construction is massively parallel work, and CPUs handle it sequentially. GPUs are built for exactly this kind of computation, but you do not want to pay for a GPU fleet sitting idle between index builds.

One setting changes the math entirely: enable GPU acceleration in OpenSearch Service. When you enable it, OpenSearch Service dynamically attaches a single-tenant serverless GPU fleet to your cluster during compute-intensive indexing operations. The GPU workers use NVIDIA cuVS to parallelize vector operations, building indexes up to 10x faster than CPU-based approaches. When the build completes, the GPU resources scale down. You pay for GPU time actually used, not for idle capacity.

Under the hood, the architectural separation is what makes this practical. Your data nodes focus on serving search queries. When indexing throughput exceeds a tunable threshold, graph builds offload to an ephemeral GPU fleet. The threshold gives you control: lightweight incremental updates stay on the data nodes, while heavy graph construction moves to GPUs. Less resource contention on the search path, more predictable latencies.

Across different datasets and vector dimensions, the benchmarks confirm the gains. Billion-scale indexes that previously took more than 24 hours now complete in under an hour. The cost runs about a quarter of CPU-based indexing, because you are using GPU resources for minutes instead of hours and you are not over-provisioning your main cluster to handle indexing spikes. With graph construction moved off the data nodes, both indexing throughput and search latency improve: CPU utilization and P95 search latencies drop by up to 50% as client load increases.

Pipe the Two Together

Separately, auto-optimization and GPU acceleration each solve a real problem. Together, they compound. Auto-optimization tells you what to build. GPU acceleration builds it fast. The combined workflow: upload your vectors to S3, specify your recall and latency requirements, let auto-optimization recommend a configuration, select one, and let GPU acceleration build the index. Raw vectors to a production-ready billion-vector index in under an hour, with no algorithm expertise required.

With auto-optimization and GPU acceleration, you can focus on what matters to you: search quality. When a full rebuild takes 45 minutes instead of 12 hours, you can test different embedding models against your actual query patterns. You can A/B test quantization strategies. You can rebuild weekly as your content changes instead of treating reindexing as a quarterly infrastructure event. The speed turns vector search from a static deployment into something you can iterate on.

CPU-based indexing runs for hours on general-purpose hardware that bills by the hour. GPU acceleration changes the equation: specialized hardware for parallel computation, billed by the minute, attached only when needed. Faster builds at lower cost because you are using the right tool for each job.

Enable It Today

Both auto-optimization and GPU acceleration are available now on Amazon OpenSearch Service, for provisioned domains and serverless collections. Enable GPU acceleration, run auto-optimization against your data, and deploy a production-ready vector index in an afternoon instead of a quarter.

With configuration automated and builds running in under an hour, the barrier to production vector search has dropped from "hire a k-NN expert" to "specify your requirements and deploy." If you have been putting off vector search because the operational overhead felt too heavy, that calculus has changed.

Build a Search App in 200 Lines. Let OpenSearch Serverless Handle the Other 10,000.

Jon Handler — Tue, 01 Sep 2026 15:03:14 +0000

I spent three weeks helping a customer tune their cluster settings. Memory allocation, shard distribution, node sizing. The kind of deep infrastructure work that has nothing to do with the actual search experience users need. Three weeks in, and nobody had touched search relevance yet.

Search infrastructure has a hidden tax. I have seen this pattern dozens of times: someone starts with an easy goal (help users find things) and ends up becoming a cluster operator. Monitoring node health at 2 AM, building capacity planning spreadsheets, explaining to the product manager why another sprint is needed just to handle autoscaling. The goal was a search experience. The result was a second job.

The operational burden does not decrease as you scale. More data means more shards to balance. More users means more capacity to plan. More features means more configuration to maintain. The infrastructure grows faster than the search logic.

Managed Does Not Mean Hands-Off

OpenSearch Service domains give you full control over your search infrastructure. You choose instance types, node counts, shard strategies, and plugin configurations. For workloads that need that level of tuning (large-scale analytics, custom ranking pipelines, specific hardware profiles), managed domains are the right choice. The tradeoff is that you own the capacity decisions and the operational overhead that comes with them.

If what you want is a search-backed application and the infrastructure is a means to that end, serverless is a different path. You skip the capacity planning, the instance sizing, and the scaling configuration. The question is not which is better. The question is whether your project needs the control or the simplicity.

The deeper issue: I keep encountering the assumption that sophisticated search requires sophisticated infrastructure. That if you are not wrestling with configuration files and deployment pipelines, you must be sacrificing capability. The assumption keeps people stuck in operational quicksand, convinced that the pain is necessary for the outcome.

The Architecture That Gets Out of Your Way

If you want to build a search-backed application, the infrastructure is three components: an Amazon OpenSearch Serverless collection for the search engine, an AWS Lambda function for query logic, and Amazon API Gateway for the public interface. Start with the API contract and let the infrastructure disappear.

I built a POC for a customer recently using a few thousand of their actual search documents. I used an AI coding tool to build out all of the front-end logic, including agent-driven search, and a search back end supporting hybrid search on OpenSearch Serverless. The whole process took a couple of hours from concept to finished implementation. No servers to size. No clusters to monitor. No capacity planning spreadsheets. Serverless all the way through.

I deployed the search back end into an OpenSearch Serverless collection. A collection is a logical grouping of indexes: I pointed the data at the collection, and the service handled indexing, scaling, and availability automatically. When we load-tested with simulated traffic, compute scaled up to match. When the test ended and traffic dropped to zero, compute dropped to zero and the customer paid for stored data only. Collection groups give you the knobs to control cost (set capacity limits, group collections by workload profile) without ever choosing instance types or node counts. The operational decisions that normally take weeks of planning just do not exist.

Where the Intelligence Lives

The Lambda function is where query logic lives. A Python script authenticates to OpenSearch Serverless using IAM credentials and processes search queries. The key technique is the multi-match query, which searches the same text across multiple fields simultaneously. When someone searches for "Harry Potter," the function looks in titles, plot summaries, and actor names at once, then ranks results by relevance. Matches across multiple fields rank higher than single-field matches. Exact matches outrank partial ones.

I have overcomplicated multi-field search myself, and I have watched others do the same. The instinct is to build separate queries for each field, then merge and rank results in application code. The result is slower, more complex, and produces worse rankings. OpenSearch is already optimized for multi-field relevance ranking. Let the search engine do what the search engine is built to do.

API Gateway sits in front as the public interface. API Gateway handles throttling, validation, and request routing. You set rate limits (1,000 requests per second with a burst allowance of 500) and the gateway enforces the limits automatically. No rate-limiting logic in your Lambda function. No DDoS protection to build from scratch. The gateway handles the HTTP layer so your search logic can focus on being search logic.

The data access layer uses IAM policies instead of a separate authentication system. Your Lambda function gets permission to query the collection through standard AWS IAM roles. No API keys to rotate, no credentials to leak, no separate authentication database to maintain. The same identity and access management system you already use for everything else in AWS.

The entire stack is deployable through CloudFormation or Terraform, which means the application is reproducible from day one. Define the collection, the Lambda function, the API Gateway, the IAM roles, and the VPC endpoint in a single template. Spin up the whole application in one deploy, tear it down just as fast, replicate it across regions or accounts without clicking a single console button. For network isolation, OpenSearch Serverless supports VPC endpoints: your Lambda function reaches the collection without traffic leaving the AWS network.

Wire It Up and Walk Away

Wire the components together and the request flow is almost boring. User types a query. JavaScript sends the query to API Gateway. The gateway validates and routes to Lambda. Lambda queries OpenSearch Serverless. Results flow back through the same chain. The entire round trip happens in milliseconds, and you wrote maybe 200 lines of actual code.

The shift to a serverless application architecture changes what you spend your time on. That customer I mentioned at the start eventually rebuilt as a serverless application. Two weeks later, the customer was working on search relevance tuning and user experience improvements. The work that actually matters.

The future of search-backed applications is that the search infrastructure becomes invisible. Not weaker, but invisible. You define what you want to search and how you want results ranked. The underlying compute and storage arrange themselves automatically.

Adding semantic search, vector embeddings, or ML-powered ranking should not require a complete infrastructure redesign. With a serverless application architecture, those capabilities become configuration changes and code updates, not migration projects. OpenSearch Serverless already supports vector search alongside keyword search in the same collection.

One thing I would recommend doing this week: look at your current search implementation and calculate how much time goes to infrastructure versus search quality. If the ratio is anything other than heavily skewed toward quality, you are solving the wrong problems. The best search experiences I have seen come from people who spend their time on user intent and relevance, not from people who have mastered Kubernetes deployments.

The infrastructure should be boring. The application should be where you invest your creativity. Wire the three components together and walk away from the cluster. OpenSearch Serverless handles the rest.

Your AI Agent Just Read the man Pages. OpenSearch Agent Skills Turn Your IDE Into a Search Engineer.

Jon Handler — Thu, 27 Aug 2026 14:52:48 +0000

You are probably already using AI to write code. Claude Code, Cursor, Kiro, GitHub Copilot, or some combination. You prompt, you get functions, you move fast. But then you need to build a search application and the workflow fractures. You leave your IDE to read OpenSearch documentation. You switch to the AWS console to provision a domain. You open a terminal to configure indices and pipelines. You write client code to connect everything. Each step requires a different tool, a different mental model, and a different tab.

Your AI agent can generate the code for each piece. What your agent cannot do (until now) is make the architectural decisions, execute the configuration, and verify that the whole system works together. The agent autocompletes functions but has no idea how to actually stand up a search application end to end.

OpenSearch Agent Skills change that. Agent Skills are executable workflows, packaged as SKILL.md files, that give your AI coding agent the domain expertise to build, configure, and deploy OpenSearch applications from natural language. Not documentation. Not code snippets. Actual capabilities your agent can run against your infrastructure.

What Ships Today

The OpenSearch project ships skills covering the workflows where context-switching hurts most: search application development, log and trace investigation, document processing, and cloud deployment to Amazon OpenSearch Service and Serverless.

The Search skill (opensearch-launchpad) builds complete search applications from a single prompt. Tell your agent "build a hybrid search app for my product catalog" and the skill handles index creation, ingest pipeline configuration, ML model registration with Amazon Bedrock, retrieval strategy selection (BM25, semantic, hybrid, or agentic), and a working search UI. The skill supports four retrieval strategies and knows when to recommend each one. Ask for a search bar and it configures a flow agent. Ask for a chatbot and it sets up a conversational agent with memory. You stay in your IDE the entire time.

The Logs skill (log-analytics and trace-analytics) brings the same pattern to observability. Instead of manually writing PPL queries to hunt for error patterns or navigating trace waterfalls to find the slow span, you describe the symptom: "why is my service returning 500s?" The skill queries your log data, identifies error patterns and volume anomalies, investigates distributed traces to find failing spans and service dependencies, and correlates logs to traces by traceId. Root cause analysis that used to require expertise in PPL syntax and trace data structures now requires a sentence.

The Cloud skills (aws-setup, aiven-setup) handle deployment. Once your search application works locally in Docker, the cloud skill deploys the same configuration to Amazon OpenSearch Service or Serverless. No manual console work, no translating local settings into production infrastructure. The Ingest skills cover two paths: document-processing handles local document ingestion via Docling (parse PDFs, HTML, office documents into indexable chunks), while managed-ingestion-service configures Amazon OpenSearch Ingestion pipelines with Automatic Semantic Enrichment for production scale. Separately, the OpenSearch Migration Assistant handles Solr-to-OpenSearch migrations as its own packaged tool, though Agent Skills may absorb that workflow in a future release.

What "Build a Search App" Actually Looks Like

Here is the search skill in action. You open Claude Code (or Cursor, or Kiro) and type: "Build a hybrid search application for my product documentation." The agent activates the opensearch-launchpad skill and moves through five phases.

First, data strategy. The agent asks what data you want to search and offers to work with your files, pull from a URL, or use a built-in sample dataset for prototyping. Second, retrieval strategy. Since you said hybrid, the agent knows to configure both BM25 keyword matching and semantic vector search with a neural pipeline. Third, architecture planning. The agent presents the full plan: index name, field mappings, embedding model from Amazon Bedrock, ingest pipeline configuration, search pipeline with hybrid scoring, and a React frontend. You review and approve.

Fourth, execution. The agent spins up a local OpenSearch cluster in Docker, creates the index, registers the Bedrock model, builds the ingest and search pipelines, indexes your data, and launches the UI. You have a working hybrid search application running locally. Fifth, deployment. The agent offers to deploy the exact same configuration to Amazon OpenSearch Service. Not a re-implementation. The same domain, same index, same pipelines, same model connector. Local-to-cloud with no translation layer.

The whole sequence takes minutes. The agent made the architectural decisions (which retrieval strategy, which embedding model, how to wire the pipeline), executed the configuration, verified the results, and deployed. You never left your IDE. You never opened the AWS console. You never read a documentation page about ingest processor syntax.

How Skills Work (The man Pages Analogy)

Think of Agent Skills as pre-read man pages with executable authority. Your AI agent already knows how to generate code. What the agent lacks is the domain-specific judgment about what to configure, in what order, with what settings. Agent Skills supply that judgment as structured SKILL.md files: step-by-step workflows, reference documentation, and executable scripts bundled together.

Skills load on demand. You can install the full collection without bloating your agent's context window. When you say "build a hybrid search app," the agent activates only the matching skill, follows its workflow, and calls the right OpenSearch APIs. When you say "investigate my 500 errors," a different skill activates. Each skill is small enough to fit in a tight context window but complete enough to handle real end-to-end workflows.

The format is agent-agnostic. Agent Skills were developed by Anthropic as a lightweight, open specification. Any agent that supports the Agent Skills protocol can use them: Claude Code, Cursor, Kiro, VS Code, GitHub Copilot, Codex. You install once and every compatible agent in your workflow gains the same capabilities.

Getting Started

Installation is one command. Run npx skills add opensearch-project/opensearch-agent-skills and your agent gains access to the full skill library. Or install specific skills: npx skills add opensearch-project/opensearch-agent-skills@opensearch-launchpad --full-depth for search, @log-analytics for log investigation, @trace-analytics for distributed traces, @aws-setup for cloud deployment. Prerequisites are Python 3.11+, uv, and Docker running locally. AWS credentials are optional (needed only for deploying to OpenSearch Service).

Once installed, express your intent. "I want to build a semantic search app with OpenSearch." The agent reads the skill instructions, runs the scripts, and handles the rest. No MCP server required. No additional tooling. The skills are files your agent reads and executes.

This Is Open Source. Build With Us.

The skills repository lives at github.com/opensearch-project/opensearch-agent-skills. It is open source, Apache 2.0 licensed, and accepting contributions. The launch skills cover search, observability, ingest, and cloud deployment. The roadmap includes dashboards, security configuration, migration workflows, and more. But the most interesting skills will come from people solving real problems in production.

If you have built a workflow that reliably configures OpenSearch for a specific use case, that workflow is a candidate for a skill. Package the steps, the reference docs, and the scripts into a SKILL.md file and submit a PR. Every skill you contribute means every developer using an AI coding agent gains that expertise automatically. You solve a problem once. The entire community benefits.