DEV Community: Jon Handler

OpenSearch Unifies the Retrieval Layer for Structured, Time Series, and Full Text Data

Jon Handler — Thu, 24 Sep 2026 20:07:35 +0000

Text, time series, and structured data in one engine. One query interface. One place for agents to look.

For a decade, search quality meant one thing: getting the right link into the top slot. Ten blue links. Click-through rate. Precision at rank one. The entire search industry organized itself around a single user staring at a results page, deciding which link to click. We got very good at that problem.

Agents changed the question. An agent does not click links. An agent synthesizes. It pulls a fact from a research document, a number from a time-series index, a constraint from a structured database, and weaves them into a response. The agent does not care whether the data came from a full-text index, a vector embedding, or an aggregation pipeline. It cares that the data is accurate, that the retrieval was fast, and that it did not have to call four different backends with four different query languages to assemble one answer.

That shift, from person-as-consumer to agent-as-consumer, changes what a search engine needs to be. It is no longer enough to do one type of retrieval well. The engine needs to unify text retrieval (structured and unstructured), time-series analytics, and hybrid search behind a single query interface. Amazon OpenSearch Service does all three.

The financial analyst's agent

Consider a financial services company building an agent to help its research analysts. The agent needs to answer questions like "What is the market sentiment on Acme Corp following their Q3 earnings, and how has the stock performed since the product launch in June?" That question touches four data types: news articles (unstructured text), internal research reports (semi-structured documents), transaction records (structured data), and stock price history (time series). Each data type lives in a different system today. The agent has to call each one, reconcile the results, and hope the latency budget survives.

OpenSearch Service consolidates those retrievals. The text data, the structured records, the time-series metrics, and the vector embeddings all live in the same engine. The agent can choose the right retrieval strategy per question: a pure lexical query when the analyst asks for a specific company by name, a semantic query when the question is conceptual, a hybrid query that combines both when the question has precise and fuzzy elements, or a multi-clause query that pulls text and time-series data together in a single call. The engine supports all of these through one query interface. The integration complexity that used to live in the agent's orchestration layer moves into the search engine, where it belongs.

Text retrieval: lexical, semantic, and hybrid together

Search engines have been refining text retrieval for more than thirty years. Inverted indices, BM25 scoring, boolean query algebra, stemming, synonyms, phrase matching, reranking: the toolbox is deep and battle-tested. OpenSearch inherits all of that. Lexical search through BM25 gives you precise term matching, boolean queries, function_score for custom relevance logic, and integration with reranking models. When the analyst searches for "Acme Corp revenue guidance," the lexical path finds documents that contain those exact terms. Lexical search remains the foundation: it handles exact matching, structured field filters, and queries where the user's words map directly to the vocabulary in the index.

Semantic search through vector embeddings finds results where the surrounding text contexts are similar, even when the exact words differ. If a user searches for "laptop" and your index contains "notebook computer," lexical search returns nothing because the terms do not overlap. Semantic search surfaces the match because "laptop" and "notebook computer" appear in overlapping lexical contexts across the training corpus. The match is based on distributional patterns in how words co-occur, not on shared terms. When the analyst searches for "market outlook," semantic search surfaces documents about "forward guidance" and "earnings projections" for the same reason. OpenSearch supports both exact and approximate nearest-neighbor search, with quantization methods and tiered storage that let you optimize vector costs as the collection grows. Auto-optimization tunes the model and indexing parameters so you do not have to.

Hybrid search combines both paths in a single query with per-clause weighting. This is the default for production retrieval today, and for good reason: the lexical leg catches exact terminology while the semantic leg catches contextual relevance. For agents, the per-clause weighting is especially useful. The agent can emphasize the lexical clause when the user asks for a specific company name and emphasize the semantic clause when the user asks a conceptual question. One query, adjustable emphasis.

Semantic search does not paper over weak lexical search. Builders today often hope that adding a vector path will compensate for a poorly tuned BM25 configuration, but hybrid search amplifies both signals. If the lexical side is noisy, the combined results are noisy. In our testing, tuning the lexical side of hybrid search produced significantly better results overall than tuning the semantic side alone. Both legs have to be good for hybrid to deliver.

Back to the financial analyst: the agent searches for information about Acme Corp's innovation strategy. The lexical clause matches documents mentioning "Acme Corp" by name. The semantic clause surfaces research reports about the company's R&D investments and patent filings, even when those reports never use the word "innovation." The combined result set gives the agent the full picture without requiring two separate queries.

Time series retrieval: aggregation and analysis

Time-series data gives agents the ability to aggregate, trend, and correlate. Stock prices, server metrics, transaction volumes, sensor readings: these are all sequences of values over time, and agents need to query them the same way they query text. OpenSearch Service provides an optimized engine for time-series data with fast ingestion, sub-second aggregations, and tiered storage for data that regulatory requirements demand you keep for years.

Agent observability is a natural fit here. The financial company wants to control its AI costs and understand how its agents perform. OpenSearch's observability stack ingests OpenTelemetry traces and metrics from the agent infrastructure. The company builds KPI dashboards around agent interactions: latency per tool call, token consumption per query, success and failure rates across agent-to-agent handoffs. The same engine that stores the analyst's research documents also stores the operational data about how the agent system itself is performing.

Hybrid queries can bring time-series data into the same result set as text retrieval. The analyst wants to see Acme Corp's stock performance over the last six months and correlate price movements with product launch dates. The time-series clause returns the aggregated price data. The text clause returns the news articles and press releases around each launch. The agent receives both in a single response and synthesizes the correlation without orchestrating separate backend calls.

Why OpenSearch fits

The information agents need spans every data type in the stack: unstructured text, structured records, vector embeddings, and time-series metrics. Most retrieval architectures treat these as separate concerns, each with a dedicated system. OpenSearch fits naturally because it handles all of them natively. Full-text search, vector search, hybrid search, sub-second aggregations over time-series data, and structured field queries all run inside the same engine, against the same indices, through the same query API.

That native breadth is what makes OpenSearch a natural retrieval layer for agents. The agent does not need to learn four query languages or stitch together results from four response formats. The agent issues one query with clauses that target different data types, and the engine handles the internal fan-out, score normalization, and result fusion. The agent gets back one ranked list and spends its compute on reasoning, not plumbing.

The value compounds as the data grows. Adding a new data type to the retrieval layer means adding an index and a query clause, not integrating a new backend. The financial company that started with research documents and stock prices can add regulatory filings, earnings call transcripts, and agent telemetry data to the same engine without changing the agent's retrieval architecture.

The engine backs that breadth with production-grade performance. Amazon OpenSearch Service delivers single-digit-millisecond query latencies, sustains thousands to tens of thousands of queries per second, and scales vector storage into the hundreds of billions of embeddings. Time-series ingestion runs at hundreds of thousands of events per second with sub-second aggregation response times. These are not theoretical limits. They are the operating parameters of production workloads running on the service today. When an agent needs to retrieve across text, vectors, and time series in one call, the latency budget has to hold for all three data types. OpenSearch Service holds it.

Where this is heading

The shift from people to agents as search consumers is still early, but the retrieval patterns are taking shape. Agents need unified retrieval across data types, not a patchwork of specialized backends stitched together with orchestration code. They need a single query interface that handles text, vectors, time series, and structured data with per-clause weighting so the agent can tune the emphasis on each data type per query.

OpenSearch Service delivers that unified interface at production scale. Lexical search, semantic search, hybrid search with reciprocal rank fusion, time-series analytics, observability, and tiered storage all run inside the same engine with single-digit-millisecond latencies and thousands of queries per second. The retrieval layer does not need to be assembled from parts. It is one service, one query language, one place for your agents to look.

Hybrid Search Explained: Combining Lexical and Semantic Search in OpenSearch

Jon Handler — Tue, 22 Sep 2026 16:23:51 +0000

Your keyword search is precise. Your vector search is contextual. Use both.

Hybrid search in Amazon OpenSearch Service combines BM25 keyword matching with vector similarity in a single query. You get the precision of lexical search (exact terms, filters, structured fields) and the contextual reach of semantic search (vocabulary mismatch handled, related terms surfaced) without choosing between them. As of OpenSearch 2.11, score normalization and combination run natively inside the search pipeline, so you do not need to build the merge logic yourself.

I ran an experiment recently that made the case for hybrid search more clearly than any architecture diagram could.

One Query, Three Approaches

I indexed a product catalog and searched for "women's shoes." With pure lexical search, the results included two women's shoes (matched on both words) and two men's shoes (matched on "shoes" alone). Technically correct: the query terms appeared in all four documents. But the user's intent was clearly gendered, and lexical search had no way to capture that.

With pure semantic search, all four results were boots. The vector space placed boots close to "shoes" based on co-occurrence patterns in the training corpus, which is accurate as far as distributional similarity goes. But the results lost variety entirely. If the user wanted sandals, flats, or sneakers, semantic search had narrowed the options to a single style.

With hybrid search, the top result was a boat shoe (strong lexical match on the query terms) and the remaining results included boots and other styles (semantic similarity expanding the candidate set). The user got what they searched for at the top, plus contextually related options below. Precision where it matters, discovery where it helps.

Score Normalization Is the Hard Part (and OpenSearch Handles It)

Combining lexical and semantic scores is not as easy as adding them together. BM25 returns scores from 1 to infinity. FAISS vector similarity returns scores between 0 and 1. Without normalization, the BM25 scores dominate every time, and the semantic signal disappears.

OpenSearch 2.11 introduced a score normalization processor in search pipelines that handles this at the coordinator level, globally across all shards. You configure the normalization technique (min-max or L2), the combination method (arithmetic mean, geometric mean, or harmonic mean), and the weight distribution between lexical and semantic scores. The OpenSearch team's benchmarks on public datasets show min-max normalization with arithmetic mean delivers strong results across a range of datasets, but the right settings depend on your data and your users.

I found equal weighting (0.5 lexical, 0.5 semantic) provided the best balance of relevance and discovery in my tests. Higher semantic weights pushed results toward the pure semantic outcome, which is useful when user intent is ambiguous but loses precision when the user knows exactly what they want. Lower semantic weights approach pure lexical behavior. The weight is a dial, and the right setting is a product decision, not a technical one.

Neither Lexical Nor Semantic Is Enough Alone

Lexical search is fast, interpretable, and works without any ML model. It handles exact matches, filters on structured fields, and generalizes across domains without fine-tuning. For queries where the user types the exact term that exists in your data, lexical search is already optimal. The failure mode is vocabulary mismatch: the user says "beach shoes" and your catalog says "water-resistant footwear."

Semantic search via vector embeddings handles vocabulary mismatch by correlating terms that co-occur in similar contexts across the training corpus. But vector search comes with costs: you need an embedding model (either hosted or via a service like Amazon Bedrock), the vectors consume memory, and the retrieval can over-cluster results around a narrow region of the vector space, as the boots example showed. Pure semantic search also loses the ability to do exact filtering and structured field matching that lexical search handles natively.

Hybrid search gives you both. Exact matches rank high because BM25 rewards them. Semantically related results fill in below because vector similarity surfaces them. Filters on structured fields (price range, category, availability) work through the lexical path. The combined result set is richer than either approach alone.

Implementing Hybrid Search in OpenSearch Service

The implementation runs through search pipelines. You create a pipeline with a normalization processor that specifies the normalization technique, combination method, and weights. At query time, you send a hybrid query that includes both a BM25 text match and a k-NN vector query. OpenSearch Service runs both queries in parallel, normalizes the scores using your configured method, combines them with your specified weights, and returns a single ranked result set.

The key configuration choices are the normalization technique and the weight distribution. For normalization, min-max rescales both score sets to a 0-1 range. L2 normalizes by the Euclidean magnitude of the score vector. In practice, min-max with arithmetic mean is a strong default. For weights, start at 0.5/0.5 and adjust based on how your users search: if most queries are specific product names, lean lexical; if most queries are natural language descriptions, lean semantic.

On the semantic side, you need an embedding model. OpenSearch Service integrates with Amazon Bedrock for embedding generation, or you can host your own model on SageMaker. The ingest pipeline with a text_embedding processor converts documents to vectors at index time. At query time, the same model converts the user's query to a vector for the k-NN search leg of the hybrid query.

Where Hybrid Search Fits

E-commerce search is the obvious case: users switch between exact product names and natural language descriptions within the same session. Internal knowledge bases are another strong fit, where users search for concepts they cannot name precisely. RAG retrieval pipelines benefit from hybrid search because the lexical path catches exact terminology (error codes, API names, product IDs) while the semantic path catches conceptual relevance.

If you are running pure lexical search today, adding the semantic path is where the lift comes from. If you have already invested in semantic search, adding the lexical path back in restores the precision you may have lost. Either way, hybrid search is the baseline for production search experiences now, not an advanced feature.

Start by running your current query set through a hybrid pipeline alongside your existing implementation. Measure not just relevance metrics but user behavior: click-through rates, time to conversion, search refinement patterns. The difference is usually visible within days.

Semantic Search Setup in Amazon OpenSearch Service: From Zero to Vector Search in 15 Minutes

Jon Handler — Thu, 10 Sep 2026 19:10:44 +0000

Tight integration brings the embedding model to the search engine. Vectors happen automagically.

If you have been considering semantic search, you already know the list of unknowns. Which embedding model do you choose? How do you host it? How do you get your documents converted to vectors and keep those vectors in sync as your data changes? How do you handle vectorization at query time without adding latency? And once you have vectors, how do you wire everything together so that your search index, your embedding model, and your ingest pipeline all talk to each other without a custom integration layer in between?

Those questions can take weeks of work to resolve, and for a lot of organizations they represent a huge hill to climb. Selecting and deploying an embedding model on SageMaker. Writing batch processing jobs to convert content into vectors. Building orchestration code to keep the ML pipeline and the search index in sync. Managing two separate systems that both need to know when your content or your model changes. The cognitive overhead alone can be a barrier, before you write a single search query.

The neural plugin in Amazon OpenSearch Service makes this dramatically easier. Through the OpenSearch Service console, you set up an ML connector that links your domain to an embedding model hosted on SageMaker or Amazon Bedrock. An ingest pipeline with a text_embedding processor then calls that model automatically every time you index a document, and queries convert to vectors at search time through the same connector. The embedding model still runs on SageMaker or Bedrock, but the integration, the vectorization pipeline, and the query-time conversion all live inside OpenSearch Service, so you are managing one workflow instead of stitching together three. If you want an even lighter path, Automatic Semantic Enrichment (which I wrote about in an earlier post) adds sparse vector enrichment at index time without deploying any model at all.

I set this up recently for a product catalog, and the whole process from zero to working semantic search took about fifteen minutes.

What the Old Playbook Looked Like

Without the neural plugin, implementing semantic search on OpenSearch Service means treating the search engine as a passive vector store. You deploy embedding models separately, build SageMaker processing jobs to convert your content into vectors, write orchestration code to call those endpoints, and manage the batch ingestion pipeline that feeds vectors into your index. When your content updates, both systems need to know. When your embedding model changes, you reprocess everything.

The architecture worked, but the operational tax was real. I spent more time debugging the plumbing between the embedding service and the search index than I spent on search relevance. The integration layer became its own project, and keeping the ML pipeline and the search index in sync consumed engineering cycles that had nothing to do with the actual search experience.

The Neural Plugin Changes the Contract

The neural plugin integrates embedding models directly into the search workflow. A pre-built CloudFormation template (available in the Integrations tab of the Amazon OpenSearch Service console) deploys the embedding model (Amazon Titan Text Embeddings through Amazon Bedrock, or a model on Amazon SageMaker), creates the IAM roles, and configures the ML connector in a single stack. The model runs on SageMaker or Bedrock, and the CloudFormation stack outputs an internal OpenSearch model ID that you use in subsequent steps. OpenSearch Service calls the model automatically at ingest and query time through the connector. With the model deployed, OpenSearch Dashboards provides a visual workflow experience for building the end-to-end AI search flow through the AI Search Flows interface (under OpenSearch Plugins). The same setup is available via the ML Commons API for automation.

When you create an OpenSearch ingest pipeline with a text_embedding processor, the neural plugin automatically calls your embedding model as documents enter the index. You specify which text fields should generate embeddings (product descriptions, document content, whatever drives your search experience), and the pipeline handles vectorization at ingest time. The entire workflow runs inside OpenSearch Service, so you are not maintaining batch jobs, custom code, or a sync layer between two systems.

At query time, the same model converts the user's search text into a vector, and OpenSearch Service runs a k-NN search against the stored embeddings. The entire semantic search workflow (text to vector to search to results) happens inside the search engine. You interact with plain text on both sides: index text documents, search with text queries. The vectors are internal.

What Semantic Search Actually Does to Your Results

I tested this against a product catalog. A keyword search for "what to wear in office" returned random clothing items: t-shirts, boots, undergarments. The query matched on individual words without any sense of the relationship between them. A semantic search for the same query returned dress suits, professional shoes, ties, and business attire. The embedding model had learned from training data that these terms co-occur in professional-clothing contexts, and the vector similarity surfaced results that lexical matching missed entirely.

For "accessory for hike," keyword search returned nothing useful. Semantic search returned backpacks, water bottles, and outdoor gear. The vocabulary mismatch between the query and the catalog vocabulary was total, and semantic search bridged the gap without a single synonym mapping.

The Search Relevance Workbench in OpenSearch Dashboards lets you run keyword and neural queries side-by-side against the same index. When stakeholders ask whether semantic search justifies the investment, the results comparison makes the case directly. The difference between matched-on-keywords and matched-on-co-occurrence-patterns is visible in the first three results. OpenSearch Dashboards also includes a visual builder for these AI-powered search flows, so you can configure and test ingest pipelines, ML connectors, and search pipelines through a graphical interface rather than writing JSON by hand.

Getting Started in 15 Minutes

Before you start, two prerequisites need to be in place. You need model access enabled in Amazon Bedrock for your chosen embedding model, and you need to add the Lambda invoke role as a backend role in your OpenSearch ML Commons configuration. Budget 10 minutes for this setup before you touch CloudFormation. With the prerequisites in place, deploy the CloudFormation template that provisions the embedding model, the ML connector, and the IAM roles. The stack outputs an internal OpenSearch model ID that you use for the rest of the setup. Create an ingest pipeline with the text_embedding processor, create an index that uses the pipeline, and start indexing documents.

The operational burden of adding ML-powered capabilities to search has dropped by an order of magnitude. The neural plugin, the CloudFormation templates, and the managed ML connectors are not doing anything fundamentally new. They are collapsing the integration layer that used to sit between your search engine and your embedding model. The search engine now handles both sides of the conversation.

If you have been putting off semantic search because the ML pipeline seemed too heavy, that calculation has changed. Deploy the CloudFormation stack, create an ingest pipeline, index a few hundred documents, and run the comparison tool. The difference in search quality is measurable in minutes, and the setup takes a fraction of the time the old approach required.

Enterprise RAG Without Per-Token Pricing: DeepSeek R1 on SageMaker with OpenSearch

Jon Handler — Tue, 08 Sep 2026 18:16:28 +0000

Search that actually thinks. Without the per-token invoice.

Open-source reasoning models have closed the gap with proprietary APIs. DeepSeek R1 on Amazon SageMaker delivers GPT-4-class reasoning at managed-service economics: no per-token fees, no rate limits, full control over data residency and model behavior. Pair that with Amazon OpenSearch Service as the vector database, and you have a RAG architecture where retrieval scales under enterprise workloads and generation does not send you a surprise invoice at the end of the month.

I deployed this stack recently and the experience changed how I think about RAG infrastructure. The reasoning quality is real. The cost structure is fundamentally different. And the integration between OpenSearch and SageMaker-hosted models is tighter than I expected.

Retrieval Is Half the Problem

When I started building RAG systems, I assumed retrieval was the whole problem. Get the right documents in front of the model, and the model handles the rest. I spent weeks fine-tuning embeddings, vector similarity metrics, and index configurations. The retrieval was excellent. The answers were still wrong.

Perfectly retrieved context fed into a model that could not reason about what it was reading produced confident, well-formatted nonsense. The model saw the right documents and still could not connect the dots: numerical reasoning failed, multi-hop questions produced hallucinations, and edge cases where the answer was "the context does not say" got fabricated responses instead. RAG is not one problem. It is three: retrieval that is fast and accurate, reasoning that is reliable, and infrastructure that does not require a dedicated team to operate. Solve any two and you still fail.

For a while, the only models that could reason well enough for production RAG were proprietary APIs. GPT-4 and Claude gave good answers but came with per-token pricing, rate limits, and data residency questions that enterprise customers could not always resolve. The open-source alternatives were inexpensive but could not handle the reasoning step reliably. DeepSeek R1 changes that tradeoff.

DeepSeek R1 on SageMaker, OpenSearch as the Vector Store

Unlike black-box APIs, DeepSeek R1 shows its reasoning process. When you ask it a question with retrieved context, it thinks through the problem step by step before answering. For RAG, this is valuable: you can see whether the model is using your retrieved documents or going off the rails. I deployed the 14B parameter distilled variant on SageMaker JumpStart in under 10 minutes. The instance (ml.g5.12xlarge) gives you the GPU memory the model needs, and SageMaker handles the deployment lifecycle.

On the retrieval side, Amazon OpenSearch Service handles storage and search. OpenSearch Service stores your documents and their vector embeddings, runs k-nearest-neighbor search when a query arrives, and returns the top matches. I used a separate, lightweight embedding model (all-MiniLM-L6-v2, also on SageMaker) for vectorization. Keeping the embedding model and the generation model on separate endpoints lets you optimize cost and latency independently: the embedding model runs on a smaller instance, and the generation model scales based on query volume.

At ingest time, an OpenSearch pipeline with a text_embedding processor ties the two sides together. When you index a document, the pipeline automatically calls your embedding model and stores the vector alongside the source text. Your embeddings stay in sync with your data without batch jobs or manual re-embedding. At query time, OpenSearch converts the user's question to a vector, runs a k-NN search, retrieves relevant documents, and passes them to DeepSeek R1 through a retrieval-augmented generation processor in the search pipeline. The model receives the context with instructions to answer based only on what it was given.

From Query to Answer (and the Setup Tax to Get There)

I indexed a small dataset and asked: "what is the population increase of New York?" OpenSearch Service retrieved the relevant documents about New York population data from 2021-2023, sent them to DeepSeek R1 with clear instructions, and returned an actual answer with the specific number and its source, not a ranked list of documents for me to read through.

Beyond the answer itself, the reasoning trace is visible. DeepSeek R1 shows how it arrived at the answer: which documents it considered, which facts it extracted, and how it combined them. When the model gets it wrong, you can see exactly where the reasoning broke down. That debuggability is something you do not get from a proprietary API where the inference is a black box.

Compared to a managed API call, the setup takes more work. Your OpenSearch Service cluster needs IAM permissions to invoke your SageMaker endpoints. You need ML connectors that define how OpenSearch Service talks to your models. You need the ingestion pipeline configured with the right embedding model and processor chain. The first time through, it feels like you are connecting a dozen different services. I used an AI coding assistant to generate the IAM role creation, trust relationships, and role mappings, which cut what used to be a week of manual configuration down to about 20 minutes.

After the first deployment, every subsequent project reuses the same patterns. The IAM configuration is the same for any SageMaker-backed model. The ingest pipeline structure is the same whether you are embedding product descriptions or legal documents. The search pipeline with the RAG processor follows the same template. You solve the infrastructure puzzle once and reuse it across projects.

Per-Instance, Not Per-Token

With a proprietary API, you pay per token. Every query incurs a generation cost that scales linearly with volume. For applications with unpredictable or high query volumes (customer support, internal knowledge bases, research tools), the per-token model makes cost forecasting difficult and cost spikes real. With DeepSeek R1 on SageMaker, you pay for instance hours. The model runs on your infrastructure, and the cost is predictable regardless of query volume. For high-volume applications, the unit economics shift in your favor quickly.

On the vector database side, OpenSearch Service cost scales with data volume and cluster size, not with query count. Together, the stack gives you a RAG architecture where the bill is a function of infrastructure, not usage. For enterprise applications where query volume is the whole point, that distinction matters.

If you are evaluating RAG architectures, start small: a few hundred documents, a handful of test queries, and the DeepSeek R1 distilled model on SageMaker. See whether the reasoning quality meets your bar. See whether the infrastructure complexity is manageable for your team. The entire setup (SageMaker endpoint, OpenSearch domain, ingest pipeline, search pipeline with RAG processor) can run as a proof of concept in an afternoon.

The question for most organizations is not whether RAG is useful. It is whether you can build a RAG system where the reasoning is reliable, the retrieval scales, and the cost does not grow linearly with every question your users ask. Open-source reasoning models on managed infrastructure, paired with a vector database built for enterprise workloads, give you a path to all three.

OpenSearch Service GPU Acceleration and Auto-Optimization: vector search with ease

Jon Handler — Thu, 03 Sep 2026 19:07:15 +0000

Without GPU acceleration, a billion-vector reindex on general-purpose CPUs takes more than 24 hours. With GPU acceleration on Amazon OpenSearch Service, that drops to under an hour. That changes what is possible. You can swap embedding models and fully reindex without scheduling a maintenance window. You can experiment with different vector dimensions and quantization strategies and see results the same afternoon. You can keep your search index current as your data changes daily instead of treating a reindex as a quarterly infrastructure event. The speed unlocks iteration, and iteration is where search quality actually improves.

Before GPU acceleration and auto-optimization arrived, getting to production was the hard part. Vector search at scale requires choosing between graph-based and bucket-based algorithms, picking a quantization method (product, binary, scalar, ranging from 2x to 64x compression), deciding between in-memory and disk-based storage, and tuning dozens of hyperparameters. Each choice cascades into three more decisions. Until recently, figuring out the right combination for your data meant weeks of manual experimentation, where each iteration took 6-12 hours and cost hundreds of dollars in compute.

Instead of running that cycle yourself, auto-optimization in OpenSearch Service runs it for you. Instead of manually testing configurations, you tell the system what recall and latency you need, and the system runs hyperparameter optimization against your actual vector data on a serverless fleet. Within an hour, you get ranked recommendations with detailed performance metrics: memory footprint, expected latency, recall rates, and cost. You are choosing between business tradeoffs, not debugging algorithm parameters.

The Configuration Problem (And Why You Should Not Be Solving It)

I walk customers through k-NN algorithm tradeoffs regularly. HNSW ef_construction values, FAISS IVF cluster counts, binary quantization with rescoring, product quantization with different subvector configurations. The conversations were useful, but they always ended the same way: the customer spent weeks running experiments and still was not confident the result was optimal for their data.

Rather than asking you to become an algorithm expert, auto-optimization in Amazon OpenSearch Service takes the problem off your plate. You provide a sample of your vector data and specify what you actually care about: recall targets and latency thresholds. Behind the scenes, a serverless fleet runs hyperparameter optimization jobs against your actual data, testing algorithms, compression techniques, and storage modes. Within an hour, you get ranked recommendations. Each recommendation shows the memory footprint, expected latency, recall rate, and cost implications. You are choosing between business tradeoffs, not debugging hyperparameters.

Because each recommendation shows its tradeoffs explicitly, you are making business decisions, not algorithmic guesses. Recommendation one might use binary quantization with in-memory storage at a given cost. Recommendation three might use product quantization with slightly better recall but 2x the memory footprint. You can see why each option scores the way it does. The system is not a black box. It is a specialist that shows its work.

The Build Time Problem (And Why GPUs Change the Math)

Even with the right configuration, building a billion-vector index on general-purpose CPUs takes days. Graph construction is massively parallel work, and CPUs handle it sequentially. GPUs are built for exactly this kind of computation, but you do not want to pay for a GPU fleet sitting idle between index builds.

One setting changes the math entirely: enable GPU acceleration in OpenSearch Service. When you enable it, OpenSearch Service dynamically attaches a single-tenant serverless GPU fleet to your cluster during compute-intensive indexing operations. The GPU workers use NVIDIA cuVS to parallelize vector operations, building indexes up to 10x faster than CPU-based approaches. When the build completes, the GPU resources scale down. You pay for GPU time actually used, not for idle capacity.

Under the hood, the architectural separation is what makes this practical. Your data nodes focus on serving search queries. When indexing throughput exceeds a tunable threshold, graph builds offload to an ephemeral GPU fleet. The threshold gives you control: lightweight incremental updates stay on the data nodes, while heavy graph construction moves to GPUs. Less resource contention on the search path, more predictable latencies.

Across different datasets and vector dimensions, the benchmarks confirm the gains. Billion-scale indexes that previously took more than 24 hours now complete in under an hour. The cost runs about a quarter of CPU-based indexing, because you are using GPU resources for minutes instead of hours and you are not over-provisioning your main cluster to handle indexing spikes. With graph construction moved off the data nodes, both indexing throughput and search latency improve: CPU utilization and P95 search latencies drop by up to 50% as client load increases.

Pipe the Two Together

Separately, auto-optimization and GPU acceleration each solve a real problem. Together, they compound. Auto-optimization tells you what to build. GPU acceleration builds it fast. The combined workflow: upload your vectors to S3, specify your recall and latency requirements, let auto-optimization recommend a configuration, select one, and let GPU acceleration build the index. Raw vectors to a production-ready billion-vector index in under an hour, with no algorithm expertise required.

With auto-optimization and GPU acceleration, you can focus on what matters to you: search quality. When a full rebuild takes 45 minutes instead of 12 hours, you can test different embedding models against your actual query patterns. You can A/B test quantization strategies. You can rebuild weekly as your content changes instead of treating reindexing as a quarterly infrastructure event. The speed turns vector search from a static deployment into something you can iterate on.

CPU-based indexing runs for hours on general-purpose hardware that bills by the hour. GPU acceleration changes the equation: specialized hardware for parallel computation, billed by the minute, attached only when needed. Faster builds at lower cost because you are using the right tool for each job.

Enable It Today

Both auto-optimization and GPU acceleration are available now on Amazon OpenSearch Service, for provisioned domains and serverless collections. Enable GPU acceleration, run auto-optimization against your data, and deploy a production-ready vector index in an afternoon instead of a quarter.

With configuration automated and builds running in under an hour, the barrier to production vector search has dropped from "hire a k-NN expert" to "specify your requirements and deploy." If you have been putting off vector search because the operational overhead felt too heavy, that calculus has changed.

Build a Search App in 200 Lines. Let OpenSearch Serverless Handle the Other 10,000.

Jon Handler — Tue, 01 Sep 2026 15:03:14 +0000

I spent three weeks helping a customer tune their cluster settings. Memory allocation, shard distribution, node sizing. The kind of deep infrastructure work that has nothing to do with the actual search experience users need. Three weeks in, and nobody had touched search relevance yet.

Search infrastructure has a hidden tax. I have seen this pattern dozens of times: someone starts with an easy goal (help users find things) and ends up becoming a cluster operator. Monitoring node health at 2 AM, building capacity planning spreadsheets, explaining to the product manager why another sprint is needed just to handle autoscaling. The goal was a search experience. The result was a second job.

The operational burden does not decrease as you scale. More data means more shards to balance. More users means more capacity to plan. More features means more configuration to maintain. The infrastructure grows faster than the search logic.

Managed Does Not Mean Hands-Off

OpenSearch Service domains give you full control over your search infrastructure. You choose instance types, node counts, shard strategies, and plugin configurations. For workloads that need that level of tuning (large-scale analytics, custom ranking pipelines, specific hardware profiles), managed domains are the right choice. The tradeoff is that you own the capacity decisions and the operational overhead that comes with them.

If what you want is a search-backed application and the infrastructure is a means to that end, serverless is a different path. You skip the capacity planning, the instance sizing, and the scaling configuration. The question is not which is better. The question is whether your project needs the control or the simplicity.

The deeper issue: I keep encountering the assumption that sophisticated search requires sophisticated infrastructure. That if you are not wrestling with configuration files and deployment pipelines, you must be sacrificing capability. The assumption keeps people stuck in operational quicksand, convinced that the pain is necessary for the outcome.

The Architecture That Gets Out of Your Way

If you want to build a search-backed application, the infrastructure is three components: an Amazon OpenSearch Serverless collection for the search engine, an AWS Lambda function for query logic, and Amazon API Gateway for the public interface. Start with the API contract and let the infrastructure disappear.

I built a POC for a customer recently using a few thousand of their actual search documents. I used an AI coding tool to build out all of the front-end logic, including agent-driven search, and a search back end supporting hybrid search on OpenSearch Serverless. The whole process took a couple of hours from concept to finished implementation. No servers to size. No clusters to monitor. No capacity planning spreadsheets. Serverless all the way through.

I deployed the search back end into an OpenSearch Serverless collection. A collection is a logical grouping of indexes: I pointed the data at the collection, and the service handled indexing, scaling, and availability automatically. When we load-tested with simulated traffic, compute scaled up to match. When the test ended and traffic dropped to zero, compute dropped to zero and the customer paid for stored data only. Collection groups give you the knobs to control cost (set capacity limits, group collections by workload profile) without ever choosing instance types or node counts. The operational decisions that normally take weeks of planning just do not exist.

Where the Intelligence Lives

The Lambda function is where query logic lives. A Python script authenticates to OpenSearch Serverless using IAM credentials and processes search queries. The key technique is the multi-match query, which searches the same text across multiple fields simultaneously. When someone searches for "Harry Potter," the function looks in titles, plot summaries, and actor names at once, then ranks results by relevance. Matches across multiple fields rank higher than single-field matches. Exact matches outrank partial ones.

I have overcomplicated multi-field search myself, and I have watched others do the same. The instinct is to build separate queries for each field, then merge and rank results in application code. The result is slower, more complex, and produces worse rankings. OpenSearch is already optimized for multi-field relevance ranking. Let the search engine do what the search engine is built to do.

API Gateway sits in front as the public interface. API Gateway handles throttling, validation, and request routing. You set rate limits (1,000 requests per second with a burst allowance of 500) and the gateway enforces the limits automatically. No rate-limiting logic in your Lambda function. No DDoS protection to build from scratch. The gateway handles the HTTP layer so your search logic can focus on being search logic.

The data access layer uses IAM policies instead of a separate authentication system. Your Lambda function gets permission to query the collection through standard AWS IAM roles. No API keys to rotate, no credentials to leak, no separate authentication database to maintain. The same identity and access management system you already use for everything else in AWS.

The entire stack is deployable through CloudFormation or Terraform, which means the application is reproducible from day one. Define the collection, the Lambda function, the API Gateway, the IAM roles, and the VPC endpoint in a single template. Spin up the whole application in one deploy, tear it down just as fast, replicate it across regions or accounts without clicking a single console button. For network isolation, OpenSearch Serverless supports VPC endpoints: your Lambda function reaches the collection without traffic leaving the AWS network.

Wire It Up and Walk Away

Wire the components together and the request flow is almost boring. User types a query. JavaScript sends the query to API Gateway. The gateway validates and routes to Lambda. Lambda queries OpenSearch Serverless. Results flow back through the same chain. The entire round trip happens in milliseconds, and you wrote maybe 200 lines of actual code.

The shift to a serverless application architecture changes what you spend your time on. That customer I mentioned at the start eventually rebuilt as a serverless application. Two weeks later, the customer was working on search relevance tuning and user experience improvements. The work that actually matters.

The future of search-backed applications is that the search infrastructure becomes invisible. Not weaker, but invisible. You define what you want to search and how you want results ranked. The underlying compute and storage arrange themselves automatically.

Adding semantic search, vector embeddings, or ML-powered ranking should not require a complete infrastructure redesign. With a serverless application architecture, those capabilities become configuration changes and code updates, not migration projects. OpenSearch Serverless already supports vector search alongside keyword search in the same collection.

One thing I would recommend doing this week: look at your current search implementation and calculate how much time goes to infrastructure versus search quality. If the ratio is anything other than heavily skewed toward quality, you are solving the wrong problems. The best search experiences I have seen come from people who spend their time on user intent and relevance, not from people who have mastered Kubernetes deployments.

The infrastructure should be boring. The application should be where you invest your creativity. Wire the three components together and walk away from the cluster. OpenSearch Serverless handles the rest.

Your AI Agent Just Read the man Pages. OpenSearch Agent Skills Turn Your IDE Into a Search Engineer.

Jon Handler — Thu, 27 Aug 2026 14:52:48 +0000

You are probably already using AI to write code. Claude Code, Cursor, Kiro, GitHub Copilot, or some combination. You prompt, you get functions, you move fast. But then you need to build a search application and the workflow fractures. You leave your IDE to read OpenSearch documentation. You switch to the AWS console to provision a domain. You open a terminal to configure indices and pipelines. You write client code to connect everything. Each step requires a different tool, a different mental model, and a different tab.

Your AI agent can generate the code for each piece. What your agent cannot do (until now) is make the architectural decisions, execute the configuration, and verify that the whole system works together. The agent autocompletes functions but has no idea how to actually stand up a search application end to end.

OpenSearch Agent Skills change that. Agent Skills are executable workflows, packaged as SKILL.md files, that give your AI coding agent the domain expertise to build, configure, and deploy OpenSearch applications from natural language. Not documentation. Not code snippets. Actual capabilities your agent can run against your infrastructure.

What Ships Today

The OpenSearch project ships skills covering the workflows where context-switching hurts most: search application development, log and trace investigation, document processing, and cloud deployment to Amazon OpenSearch Service and Serverless.

The Search skill (opensearch-launchpad) builds complete search applications from a single prompt. Tell your agent "build a hybrid search app for my product catalog" and the skill handles index creation, ingest pipeline configuration, ML model registration with Amazon Bedrock, retrieval strategy selection (BM25, semantic, hybrid, or agentic), and a working search UI. The skill supports four retrieval strategies and knows when to recommend each one. Ask for a search bar and it configures a flow agent. Ask for a chatbot and it sets up a conversational agent with memory. You stay in your IDE the entire time.

The Logs skill (log-analytics and trace-analytics) brings the same pattern to observability. Instead of manually writing PPL queries to hunt for error patterns or navigating trace waterfalls to find the slow span, you describe the symptom: "why is my service returning 500s?" The skill queries your log data, identifies error patterns and volume anomalies, investigates distributed traces to find failing spans and service dependencies, and correlates logs to traces by traceId. Root cause analysis that used to require expertise in PPL syntax and trace data structures now requires a sentence.

The Cloud skills (aws-setup, aiven-setup) handle deployment. Once your search application works locally in Docker, the cloud skill deploys the same configuration to Amazon OpenSearch Service or Serverless. No manual console work, no translating local settings into production infrastructure. The Ingest skills cover two paths: document-processing handles local document ingestion via Docling (parse PDFs, HTML, office documents into indexable chunks), while managed-ingestion-service configures Amazon OpenSearch Ingestion pipelines with Automatic Semantic Enrichment for production scale. Separately, the OpenSearch Migration Assistant handles Solr-to-OpenSearch migrations as its own packaged tool, though Agent Skills may absorb that workflow in a future release.

What "Build a Search App" Actually Looks Like

Here is the search skill in action. You open Claude Code (or Cursor, or Kiro) and type: "Build a hybrid search application for my product documentation." The agent activates the opensearch-launchpad skill and moves through five phases.

First, data strategy. The agent asks what data you want to search and offers to work with your files, pull from a URL, or use a built-in sample dataset for prototyping. Second, retrieval strategy. Since you said hybrid, the agent knows to configure both BM25 keyword matching and semantic vector search with a neural pipeline. Third, architecture planning. The agent presents the full plan: index name, field mappings, embedding model from Amazon Bedrock, ingest pipeline configuration, search pipeline with hybrid scoring, and a React frontend. You review and approve.

Fourth, execution. The agent spins up a local OpenSearch cluster in Docker, creates the index, registers the Bedrock model, builds the ingest and search pipelines, indexes your data, and launches the UI. You have a working hybrid search application running locally. Fifth, deployment. The agent offers to deploy the exact same configuration to Amazon OpenSearch Service. Not a re-implementation. The same domain, same index, same pipelines, same model connector. Local-to-cloud with no translation layer.

The whole sequence takes minutes. The agent made the architectural decisions (which retrieval strategy, which embedding model, how to wire the pipeline), executed the configuration, verified the results, and deployed. You never left your IDE. You never opened the AWS console. You never read a documentation page about ingest processor syntax.

How Skills Work (The man Pages Analogy)

Think of Agent Skills as pre-read man pages with executable authority. Your AI agent already knows how to generate code. What the agent lacks is the domain-specific judgment about what to configure, in what order, with what settings. Agent Skills supply that judgment as structured SKILL.md files: step-by-step workflows, reference documentation, and executable scripts bundled together.

Skills load on demand. You can install the full collection without bloating your agent's context window. When you say "build a hybrid search app," the agent activates only the matching skill, follows its workflow, and calls the right OpenSearch APIs. When you say "investigate my 500 errors," a different skill activates. Each skill is small enough to fit in a tight context window but complete enough to handle real end-to-end workflows.

The format is agent-agnostic. Agent Skills were developed by Anthropic as a lightweight, open specification. Any agent that supports the Agent Skills protocol can use them: Claude Code, Cursor, Kiro, VS Code, GitHub Copilot, Codex. You install once and every compatible agent in your workflow gains the same capabilities.

Getting Started

Installation is one command. Run npx skills add opensearch-project/opensearch-agent-skills and your agent gains access to the full skill library. Or install specific skills: npx skills add opensearch-project/opensearch-agent-skills@opensearch-launchpad --full-depth for search, @log-analytics for log investigation, @trace-analytics for distributed traces, @aws-setup for cloud deployment. Prerequisites are Python 3.11+, uv, and Docker running locally. AWS credentials are optional (needed only for deploying to OpenSearch Service).

Once installed, express your intent. "I want to build a semantic search app with OpenSearch." The agent reads the skill instructions, runs the scripts, and handles the rest. No MCP server required. No additional tooling. The skills are files your agent reads and executes.

This Is Open Source. Build With Us.

The skills repository lives at github.com/opensearch-project/opensearch-agent-skills. It is open source, Apache 2.0 licensed, and accepting contributions. The launch skills cover search, observability, ingest, and cloud deployment. The roadmap includes dashboards, security configuration, migration workflows, and more. But the most interesting skills will come from people solving real problems in production.

If you have built a workflow that reliably configures OpenSearch for a specific use case, that workflow is a candidate for a skill. Package the steps, the reference docs, and the scripts into a SKILL.md file and submit a PR. Every skill you contribute means every developer using an AI coding agent gains that expertise automatically. You solve a problem once. The entire community benefits.

The $11,000/Month diff: You're Shipping 600 DPI When 150 Gets the Job Done

Jon Handler — Tue, 25 Aug 2026 15:07:50 +0000

You loaded 125 million product embeddings into your vector search cluster. The math looked fine on paper: 1536 dimensions, HNSW indexing, three availability zones. Then the first bill arrived. Eighteen thousand dollars a month. Your entire text search infrastructure costs six thousand. The vector layer alone is triple that.

The product manager asked a reasonable question: why does a column of numbers cost more than the rest of the application combined? The answer is precision nobody needs.

Think of vector storage like shipping a library across the country. You could photograph every page at 600 DPI, lossless TIFF, archival quality. Or you could photograph them at 150 DPI JPEG, more than enough for anyone who wants to read the words. The first option fills a shipping container. The second fits in a suitcase. Vector search, by default, ships the container.

Why Vectors Blow Up Your Budget

A single 1536-dimension embedding at full precision: 12,288 bytes. Each dimension is a 4-byte float. Multiply by 125 million: 1.5 terabytes of vector storage before you think about the search graph.

HNSW builds multi-layer graph structures that need RAM for fast queries. The formula: (1.1 × 4 × dimensions) + (8 × edges × vector_count). With defaults and 125 million vectors: 862 GB of RAM.

Traditional text search is disk-bound. Vector search is memory-bound. The graph lives in RAM or queries take seconds. You cannot solve this by adding nodes. More nodes = more shards = the graph splits into smaller neighborhoods = degraded recall. The scaling pattern that works for logs does not work for vectors.

The Precision You're Paying For

Embedding models default to 1536 dimensions at 32-bit float. You can choose lower-dimensional outputs and reduced precision, but mostly you take the default because models benchmark best at full width.

The question nobody asks at architecture time: how much of that precision matters for retrieval?

Retrieval is not final ranking. You are narrowing 125 million candidates to a few hundred, then rescoring with a cross-encoder or business logic. The initial pass needs the right neighborhood, not perfect ordering within that neighborhood. Binary precision (1 bit per dimension) maintains 90%+ recall for that job.

The Cost-Latency-Accuracy Dial

Amazon OpenSearch Service gives you several points on this dial:

Full precision in RAM (FP32): 862 GB RAM, single-digit ms latency, $18,000/month.

FAISS FP16 in RAM: ~430 GB, single-digit ms, negligible accuracy loss. ~$10,000/month.

INT8 or binary vectors in RAM: quarter or 32x reduction. Latency stays fast. Accuracy depends on your data — for retrieval into a reranker, most workloads absorb it.

On_disk + binary quantization: graph on SSD, 44 GB RAM, ~200 ms p50, 95% accuracy. $7,000/month. Best for RAG pipelines, agent context assembly, batch reranking.

Dimensionality: 768-dim instead of 1536 halves the footprint at every level. Stacks with all of the above.

Operational Tuning

Shard sizing: vector workloads want 50-75 GB per shard (not the 10-30 GB text search guidance). HNSW graphs work better with denser neighborhoods. Fewer, bigger shards = better recall.

Off-heap memory: OpenSearch caps heap at 32 GB. Remaining memory splits between filesystem cache and k-NN vector cache. The circuit breaker knn.memory.circuit_breaker.limit (default 50%) controls how much off-heap goes to vectors. Raise it for vector-heavy workloads:

# Give 75% of off-heap to the k-NN graph cache
knn.memory.circuit_breaker.limit: 75%

On a 128 GB node: 32 GB heap + 96 GB off-heap. At 75%, vectors get 72 GB — plenty for a quantized graph with headroom.

Run diff On the Deployment

Configuration	RAM	Latency	Monthly Cost
Full precision (FP32) in RAM	862 GB	<10 ms	$18,000
FAISS FP16 in RAM	~430 GB	<10 ms	~$10,000
On_disk + binary quantization	44 GB	~200 ms	$7,000
768-dim + FP16 in RAM	~215 GB	<10 ms	~$6,000

$11,000/month between the extremes. $132,000/year. But you do not have to choose an extreme. Pick the point on the dial that matches your latency budget.

The Actual Lesson

If your CloudWatch memory utilization sits above 60% with full-precision vectors, you have room to move down the dial. Test FP16 on a shadow index. Measure the accuracy delta on your real queries.

Back to the product manager's question: why does a column of numbers cost more than the rest of the application? Because each number is stored at 32x the precision retrieval needs, and the graph lives entirely in RAM. Fix the precision, and the column of numbers costs less than the text catalog it serves.

Your Search Engine Is Running grep on a Dictionary. OpenSearch Automatic Semantic Enrichment Fixes That.

Jon Handler — Fri, 21 Aug 2026 17:05:20 +0000

Traditional search is a glorified Ctrl+F. If your beach sandals are labeled "water-resistant footwear" in your product database, nobody searching for "beach shoes" will find them. "Water," "resistant," and "footwear" do not match "beach" or "shoes," so the search engine returns nothing. Users who cannot find what they want within two or three searches leave the site entirely. They do not email support. They do not try different keywords. They buy from the competitor whose search finds products where related but missing terms also describe those products.

The familiar fix is synonyms. Teams build elaborate synonym dictionaries: "sneakers = shoes = footwear = kicks = trainers." They maintain spreadsheets with thousands of mappings. Every time a product fails to surface, someone adds another row. One retail team had a dedicated person whose entire job was updating the synonym list. Language is not a lookup table, though. "Kicks" means shoes in one sentence and something else entirely in another. "Trunk" is luggage, part of a car, or an elephant's nose depending on surrounding words. No manual mapping captures the contextual, distributional nature of how words actually cluster in usage.

The other common approach is full semantic search from scratch: spin up ML infrastructure, train or fine-tune embedding models, manage vector databases, build pipelines that transform text into dense numerical representations. Converting text into vectors that correlate terms by co-occurrence works well. But the infrastructure overhead is real. You manage model serving, handle inference latency, version models, monitor drift, and keep everything running at scale. Teams that go this route often discover that their inference costs at production scale are astronomical because the model runs on every single search query. They solved the relevance problem and created a performance and cost problem in its place.

Automatic Semantic Enrichment in Amazon OpenSearch Serverless keeps the relevance without the per-query cost. At index time, the system generates sparse embeddings from your content, capturing which terms co-occur in similar contexts across millions of documents, and stores those embeddings alongside the original text. No ML infrastructure, no model management, no changes to your application code.

Why Sparse Vectors

Automatic Semantic Enrichment supports both sparse and dense embeddings. Sparse is the path worth starting with. A sparse vector is a weighted list of correlated terms: the model identifies which words co-occur with your content across large corpora and stores those associations as rank features in the inverted index. No approximate nearest neighbor graph. No vector database. No new query syntax. You search with text, and the inverted index does what inverted indices have always done, except now the vocabulary is expanded beyond what the author originally wrote.

The model runs once, at ingest time. When a document enters the index, OpenSearch Serverless passes the text through a sparse encoder trained on distributional co-occurrence patterns. The encoder outputs a weighted term list: "beach sandals" produces rank features for "waterproof," "summer footwear," "flip-flops," and other terms that appear in overlapping lexical contexts, even though none of those words exist in the original text. Those features are stored alongside the document. At search time, a lightweight tokenization process matches the query against both the original text and the enriched terms. No full model inference on the hot path.

If your relevance requirements are more demanding, Automatic Semantic Enrichment also supports dense embeddings with bi-encoding: full vector representations for both documents and queries, matched by approximate nearest neighbor search. Dense vectors capture finer-grained distributional relationships and generally rank better on hard queries. But dense requires a vector index, ANN infrastructure, and model inference at query time. Sparse gives you most of the relevance lift for a fraction of the operational weight. Start sparse. Move to dense when you have the query volume and relevance data to justify the added infrastructure.

The cost structure reinforces the choice. With sparse enrichment, you pay for model inference once per document at ingest. Search queries hit the inverted index without touching the model. No per-query inference cost, no GPU fleet at search time, no latency spike when traffic surges. Your search cost scales with query volume against a standard inverted index, not with model inference. For most workloads, this is the difference between a search feature you can afford to run at scale and one that lives permanently in staging because production costs are unpredictable.

The grep Moment

Here is where the grep analogy breaks. Traditional lexical search is grep "scarlet sneakers" catalog.txt. No match, no result, end of story. Automatic semantic enrichment is more like having a pre-built thesaurus of co-occurrence patterns baked into the index itself. The query does not need to match the exact string because the index already knows which terms cluster together based on how they appear across millions of documents.

Back to the beach sandals. A customer searches for "shoes for the beach." Your product catalog says "water-resistant footwear." Lexical search returns nothing. With Automatic Semantic Enrichment, the sparse encoder already identified at ingest time that "beach," "sandals," "water-resistant," "footwear," and "shoes" all co-occur in overlapping lexical contexts across the training corpus. Those correlations are stored as rank features in the index. The query "shoes for the beach" matches against those enriched terms and surfaces the product without a single exact word match in the original listing.

Setup Is Embarrassingly Easy

You create an index, designate which fields should have automatic semantic enrichment, choose a language model (English or multilingual), and OpenSearch Serverless creates the ingest and search pipelines automatically. You index documents as plain text. You search with plain text queries. The semantic enrichment happens invisibly during ingestion. AWS benchmarks show up to 20% better search accuracy compared to pure lexical matching.

The approach augments lexical search rather than replacing it. You keep traditional text fields alongside semantic fields. Exact matching works when you need precision. Semantic correlation works when the user's vocabulary does not match your catalog's vocabulary. Both live in the same index.

Where This Is Heading

The default expectation for search is shifting. Users trained on Google, voice assistants, and ChatGPT do not think in terms of keyword matching anymore. They expect search to correlate vocabulary, not match strings. The companies that ship semantic correlation first have a structural advantage: not because their products are better, but because customers can actually find those products.

If you are running e-commerce search, documentation search, internal knowledge bases, or any retrieval system where vocabulary mismatch causes failed queries, automatic semantic enrichment removes the friction without requiring ML expertise, model management, or changes to your application architecture. Pick one high-value use case where search is failing users. Enable enrichment. Measure the difference in search success rates. The improvement is measurable within days, and the setup takes minutes rather than sprints.

What If Your Search Infrastructure Only Existed When Someone Searched?

Jon Handler — Wed, 19 Aug 2026 16:30:01 +0000

Create a collection. Send data. Query. That is the entire setup for Amazon OpenSearch Serverless. No cluster sizing spreadsheet. No capacity planning meeting. No argument about how many nodes you need for peak versus average. You create, you use, you pay for what you used. When you're not using it, you pay for storage and nothing else.

That last part is new, and it is the part worth getting excited about.

The joy of just using it

OpenSearch Serverless scales compute to zero when there is no activity. Not "scales to a minimum baseline." Zero. Your dev/test collection that nobody touches on weekends costs you storage and nothing else until Monday morning. Your QA environment that runs a test suite for 20 minutes a day pays for 20 minutes of compute. Your production search that handles a thousand queries per second at noon and twelve queries per second at 3 AM pays for a thousand at noon and twelve at 3 AM.

Indexing and search compute scale independently. This is the part that deserves a pause. If you have steady-state ingest running (a CDC pipeline feeding documents from your database, say), the search side still scales down when nobody is querying. You do not pay search compute costs to keep your ingest running. Two workloads, two scaling curves, one collection. The economics of running both together just changed.

Autoscaling responds in seconds. When a marketing campaign drives a traffic spike at noon, compute scales up immediately to match demand. Your users see the same latency at 10x traffic as they saw at 1x. When the campaign ends and traffic subsides overnight, compute scales back down just as fast. By morning, if nobody is querying, the collection is back at zero compute cost. This cycle happens automatically, every day, without alarms, without manual intervention, and without you ever provisioning for a peak that may or may not arrive. The infrastructure mirrors your actual workload in real time, and the bill reflects what actually happened rather than what you feared might happen.

What this looks like in practice

In a recent demo, our team built a hybrid search application (semantic vector search combined with keyword search) against a dataset of 5,000 movies. The entire setup happened through OpenSearch Agent Skills: pre-built capabilities that let an AI agent (Kiro, Claude Code, or any MCP-compatible tool) provision and configure OpenSearch Serverless from natural language.

The agent created a serverless collection, set up IAM roles, configured an Amazon Bedrock connector for Titan text embeddings (1,024-dimension vectors), created ingestion and search pipelines, indexed the movie data, and built a React frontend. All from a single instruction: "Build a search application using the movies data in my local folder."

The search worked as expected. A query for "racing" returned semantically relevant movies, not just exact title matches. Hybrid search handled both concept-based and precise queries. But the infrastructure behavior is what stood out.

After the demo ended and traffic stopped, compute dropped to zero OCUs (OpenSearch Compute Units) within minutes. The collection existed. The data remained indexed. The cost was storage only. When we ran queries again, compute spun up in seconds and the collection responded at full speed.

For teams evaluating hybrid search, the Bedrock integration and automated pipeline setup remove most of the friction. You can prototype a semantic search system in an afternoon and pay nothing for compute between prototype sessions.

Where this matters most

Agentic AI workloads are the clearest fit. Agents are bursty and unpredictable by nature: an agent might fire 50 search queries in 10 seconds during a research task, then go silent for an hour. Provisioning for peak agent traffic wastes money. Provisioning for average risks latency during bursts. Scale-to-zero with seconds-fast autoscaling handles both without you thinking about it.

The same economics apply anywhere traffic is uneven or intermittent. Dev/test environments sit idle 90% of the time, burning compute that nobody is using. QA workloads run a test suite for a few minutes each day, then sit dark until tomorrow. Staging environments spin up for weekly deploys and do nothing in between. Internal tools see a flurry of activity during business hours and silence overnight. Even production search has genuine off-peak hours where query volume drops to a fraction of daytime traffic. In every one of these cases, you are paying for compute capacity that has nothing to do. OpenSearch Serverless makes that idle compute disappear from your bill.

The independent scaling of indexing and search unlocks a pattern that was previously expensive: continuous ingest (keeping your search index fresh from a database CDC stream) without paying for search compute during low-traffic hours. Your data stays current. Your search costs reflect actual query volume, not ingest activity.

Compared to provisioning OpenSearch Service domains for peak capacity, OpenSearch Serverless can deliver up to 60% lower cost for workloads with variable traffic patterns. The savings come from not paying for compute that has nothing to do.

The `chmod +x` moment

OpenSearch Serverless makes search executable without infrastructure work. You create a collection, point your data pipeline at it, and start querying. Scale happens underneath you. Cost tracks what you actually use. You do not monitor OCU utilization graphs. You do not wake up to adjust capacity for a Monday morning traffic pattern that differs from Sunday. The infrastructure disappears into the background where it belongs, and you spend your time on the search experience your users see rather than the plumbing they never should.

If you are currently sizing clusters, managing OCU minimums from the previous generation, or over-provisioning because you cannot predict traffic, the math just changed. Check your CloudWatch utilization graphs. If average utilization is below 50%, you are paying for compute that OpenSearch Serverless would scale away automatically.

Documentation: Amazon OpenSearch Serverless (https://aws.amazon.com/opensearch-service/features/serverless/)

Read Your AI Agent's Mind. OpenSearch Observability for LLM Traces.

Jon Handler — Mon, 17 Aug 2026 16:45:13 +0000

You shipped your AI agent. It answers questions, calls tools, retrieves documents, and mostly does the right thing. Mostly. When it does the wrong thing, you have no idea why. The prompt looked fine. The retrieval seemed relevant. Somewhere between the user's question and the final answer, something went sideways, and you are staring at a log that says "200 OK" while your customer stares at nonsense.

APM dashboards show you HTTP latency and error rates. They have no idea what your agent is thinking. An agent is not a request-response cycle. An agent reasons, branches, calls tools, re-evaluates, and sometimes loops. Tracing that execution requires a different kind of observability, one that understands LLM calls, token budgets, tool invocations, and multi-step reasoning flows. OpenTelemetry now provides semantic conventions for exactly this, and Amazon OpenSearch Service gives you the full stack to collect, store, query, and visualize those traces in one place.

OpenTelemetry learned to trace agents

OpenTelemetry (OTel) is the open standard for distributed tracing, metrics, and logs. If you run microservices, you probably already emit OTel spans for HTTP calls and database queries. The recent addition of generative AI semantic conventions extends that vocabulary to cover LLM-specific operations. Every model call, every tool invocation, every reasoning step gets a standardized span with attributes like model name, token counts (input and output), temperature, tool names, and agent step identifiers.

A few lines of setup and every agent execution emits a structured trace. A GenAI SDK (Python or TypeScript) provides decorators and auto-instrumentation for popular frameworks: Strands Agents, LangGraph, CrewAI, the OpenAI Agents SDK. It hooks into providers like OpenAI, Anthropic, Amazon Bedrock, and LangChain without requiring you to rewrite application code. You add a few lines of setup, and every agent execution emits a structured trace that captures the full decision tree.

What makes this different from logging model calls yourself? Structure and correlation. Each span carries a trace ID that links it to the parent request. A single user question might trigger three LLM calls, two tool invocations, and a retrieval step. OTel stitches them into a single trace tree with timing, token cost, and dependency relationships intact. Without this correlation, you are left grepping through scattered logs trying to reconstruct what happened.

What agent traces tell you

Structured traces make previously impossible questions answerable without manual debugging.

Where is the agent spending tokens? Token usage maps directly to cost. A trace shows you that 80% of your tokens went to a single retrieval-augmented call that stuffed too much context. Or that the agent retried a tool call three times because the first two responses were malformed. You cannot optimize what you cannot measure, and token-level attribution across a multi-step flow is exactly what agent traces provide.

Why did the agent choose that tool? Agent frameworks make routing decisions: which tool to call, in what order, with what parameters. The trace captures each decision as a span. When the agent calls a calculator instead of a database lookup, the trace shows you the reasoning step and input that led there. That visibility turns "the agent hallucinated" from a mystery into a debuggable event.

How long does each step take? Latency in agentic applications compounds. If each LLM call takes 2 seconds and the agent chains four of them, your user waits 8 seconds before seeing a response. The trace timeline exposes these serial dependencies, letting you identify which calls can be parallelized or cached. The difference between a 2-second and a 10-second agent response is often a single unnecessary sequential step.

Is the agent looping? Agents with autonomous tool use can fall into retry loops or circular reasoning patterns. A trace that shows 15 spans where you expected 4 is an immediate signal. OpenSearch Dashboards renders these as hierarchical trees and directed acyclic graphs (DAGs), making pathological patterns visually obvious before they drain your token budget.

OpenSearch Service is the easy button

Any OTLP-compatible backend can store these traces. OpenSearch Service stands out because it provides the full pipeline as a managed, unified stack: collection, processing, storage, querying, visualization, alerting, and anomaly detection. No stitching together four separate tools.

From your application to a dashboard, the pipeline has four stages. Your application emits OTel spans through the GenAI SDK. An OpenTelemetry Collector normalizes them using the generative AI semantic conventions. Amazon OpenSearch Ingestion (Data Prepper) routes the OTLP data into your OpenSearch Service domain, creating service maps, correlating traces with logs, and computing RED metrics (Rate, Errors, Duration) automatically. OpenSearch Dashboards then renders agent traces as explorable trees, DAGs, and timelines with token-cost overlays.

Open the Agent Traces panel in OpenSearch Dashboards and you see something APM views never show you. It differs from the standard APM view (which monitors HTTP services) in that it understands model calls, token budgets, and reasoning flows as first-class concepts. You see a hierarchical breakdown of each agent execution: which model was called, what it cost, which tools fired, how long each step took, and whether the agent looped or branched. This is not a general-purpose tracing UI adapted for AI. It is a dedicated interface built on OTel's generative AI conventions.

When token cost spikes or an agent loops unexpectedly, OpenSearch Service fires alerts on trace data (fire a Slack notification when token cost per request exceeds a threshold), anomaly detection on agent behavior (flag when a normally 3-step agent suddenly takes 12 steps), and log-to-trace correlation (click from a trace span directly to the error log that explains the failure). These capabilities exist in one domain, under one security model, queried with one language.

Now you know

Your agent is no longer a black box. OTel's generative AI conventions give you structured, correlated traces across every LLM call, tool invocation, and reasoning step. OpenSearch Service stores those traces, visualizes them as explorable execution graphs, alerts on anomalies, and correlates them with your existing logs and metrics. The operational model for AI agents just caught up with the operational model you already have for microservices.

Think of it as tail -f for your agent's brain. Except instead of scrolling text, you get a full execution graph with cost attribution, latency breakdowns, and anomaly detection built in. The next time your agent does something inexplicable, you will not be guessing. You will be looking at the trace.

To get started, check the OpenSearch Observability platform page and the agent traces documentation. The OpenSearch Observability Stack provides a Docker Compose setup with a preconfigured OTel Collector, Data Prepper, and example applications so you can see agent traces working end to end in minutes.

1TB Migrated in 33 Minutes. The Agent Didn't Even Need Coffee.

Jon Handler — Fri, 14 Aug 2026 16:10:16 +0000

In a recent demo, our team migrated one terabyte of production data from Elasticsearch 7.10 running on Amazon OpenSearch Service to OpenSearch 3.x running on OpenSearch Service in 33 minutes. The migration was orchestrated entirely by an AI agent. No runbook. No manual configuration files. No engineer babysitting terminal windows at 2 AM. The agent made infrastructure decisions, monitored progress, handled errors, and completed the migration while the team watched from the sidelines.

The toolchain was Kiro connected to Migration Assistant for Amazon OpenSearch Service running on EKS. Migration Assistant is open source and supports migrations from Elasticsearch, self-managed OpenSearch, and Apache Solr (versions 6.x through 9.x) to both Amazon OpenSearch Service domains and Amazon OpenSearch Serverless collections. It handles snapshot-based backfill, live traffic capture and replay for validation, and metadata translation. The AI-assisted experience lets you drive the entire workflow from Kiro, Claude Code, or any MCP-compatible tool: the agent plans the migration, deploys infrastructure, executes the data movement, monitors progress, and validates results. In this demo, the result challenges assumptions about what still requires human hands during a database migration.

Why migrations still eat engineering weeks

Database migrations have three layers of difficulty. The first is planning: figuring out which indices to migrate, what version incompatibilities exist, how to handle mapping differences, and what order to execute steps. The second is execution: configuring snapshots, parallelizing backfill, managing network throughput, and coordinating the cutover. The third is monitoring: watching progress, detecting errors, calculating ETAs, and deciding whether a failure is recoverable or fatal.

Most teams solve all three layers with the same tool: a senior engineer. That engineer writes the plan, executes it step by step, monitors progress manually, and makes judgment calls when things go wrong. The work is cognitively demanding but largely repetitive across migrations. The cluster sizes differ. The index counts differ. The judgment calls are the same patterns applied to different numbers.

This is the kind of work where AI agents should excel: well-defined operations with clear success criteria, where the decisions follow from observable state rather than ambiguous business context. The question is whether the tooling is mature enough to let an agent make those decisions reliably. Based on what we observed, the answer is yes.

What the agent actually did

The demo started with Kiro connected to a Migration Assistant deployment running on EKS. The Migration Assistant was bootstrapped in a fresh VPC via CloudFormation. The source was an Amazon OpenSearch Service domain running Elasticsearch 7.10 with 1 TB of primary data (2 TB including replicas). The target was an Amazon OpenSearch Service domain running OpenSearch 3.x.

Kiro scanned the AWS account, discovered the source and target clusters, and presented options. When told to migrate as fast as possible with no other workloads running, the agent adjusted its approach accordingly. It set snapshot throttling high (800 MB/s) because there was no competing I/O to protect. It calculated optimal parallelization at 480 concurrent reindex pods, matching the shard count. It configured EKS auto mode to provision EC2 instances on demand so the infrastructure scaled to match the workload.

None of these decisions came from a template. The agent queried the source cluster for real-time state and made context-aware choices. If other workloads had been running, it would have throttled the snapshot to avoid disk contention. The throttling decision came from understanding the tradeoff between migration speed and source cluster impact.

The timeline

The 33-minute migration broke down as follows. Snapshot took 5 minutes at full throttle. Metadata migration (index mappings, aliases, templates) took 1 minute. Backfill across 480 parallel pods took 12 minutes. The remainder was setup, validation, and the agent confirming success. End to end, from first prompt to validated completion, 33 minutes for a terabyte.

Monitoring and error handling

During the snapshot phase, Kiro queried the source cluster every 30 seconds for progress metrics. During backfill, it recalculated ETAs every minute. This monitoring behavior was not preconfigured in the Argo workflow. The agent added it because that is what a human operator would want to know. The feedback loop from these AI-assisted runs is now informing what gets built into the Migration Assistant itself.

The agent caught errors and decided what to do with them. The agent distinguished between recoverable errors (retry) and fatal errors (stop and surface for human investigation). The team could have disconnected entirely and the migration would have completed successfully. That level of autonomy is the point. The agent was not generating scripts for a human to run. It was running infrastructure.

AI handles choreography. Humans handle strategy.

Watch this demo and a clear division of labor emerges. The humans decided which clusters to migrate, what performance tradeoffs to accept, and whether to prioritize speed over source cluster stability. The agent handled everything else: the configuration, the monitoring loops, the progress checks, the validation queries, and the error recovery.

The point is not fewer engineers. The point is engineers spending time on decisions, not choreography. A 1 TB migration that previously required 4-6 hours of active senior engineering time now requires a few prompts and strategic oversight. The operational choreography disappears. The judgment about what to migrate and why stays with the human.

Fifty cents. That is what the AI orchestration for the entire migration consumed in compute. Compare that to the fully loaded cost of a senior engineer spending half a day on the same work. The economics push toward using the agent for execution and reserving human time for decisions that require business context.

The tooling is open source

Migration Assistant for Amazon OpenSearch Service is open source at opensearch-project/opensearch-migrations and supports Elasticsearch, Solr, and self-managed OpenSearch as sources. The s/runbook/agent/ substitution is available today. Point Kiro, Claude Code, or any MCP-compatible tool at the Migration Assistant. The agent handles the infrastructure decisions, the progress monitoring, and the error recovery. The senior engineer who used to babysit the migration can now review the plan, approve it, and check back when the agent reports completion.

Medium tags: opensearch, elasticsearch, migration, ai-agents, devops