DEV Community: Jon Handler

Build a Search App in 200 Lines. Let OpenSearch Serverless Handle the Other 10,000.

Jon Handler — Tue, 01 Sep 2026 15:03:14 +0000

I spent three weeks helping a customer tune their cluster settings. Memory allocation, shard distribution, node sizing. The kind of deep infrastructure work that has nothing to do with the actual search experience users need. Three weeks in, and nobody had touched search relevance yet.

Search infrastructure has a hidden tax. I have seen this pattern dozens of times: someone starts with an easy goal (help users find things) and ends up becoming a cluster operator. Monitoring node health at 2 AM, building capacity planning spreadsheets, explaining to the product manager why another sprint is needed just to handle autoscaling. The goal was a search experience. The result was a second job.

The operational burden does not decrease as you scale. More data means more shards to balance. More users means more capacity to plan. More features means more configuration to maintain. The infrastructure grows faster than the search logic.

Managed Does Not Mean Hands-Off

OpenSearch Service domains give you full control over your search infrastructure. You choose instance types, node counts, shard strategies, and plugin configurations. For workloads that need that level of tuning (large-scale analytics, custom ranking pipelines, specific hardware profiles), managed domains are the right choice. The tradeoff is that you own the capacity decisions and the operational overhead that comes with them.

If what you want is a search-backed application and the infrastructure is a means to that end, serverless is a different path. You skip the capacity planning, the instance sizing, and the scaling configuration. The question is not which is better. The question is whether your project needs the control or the simplicity.

The deeper issue: I keep encountering the assumption that sophisticated search requires sophisticated infrastructure. That if you are not wrestling with configuration files and deployment pipelines, you must be sacrificing capability. The assumption keeps people stuck in operational quicksand, convinced that the pain is necessary for the outcome.

The Architecture That Gets Out of Your Way

If you want to build a search-backed application, the infrastructure is three components: an Amazon OpenSearch Serverless collection for the search engine, an AWS Lambda function for query logic, and Amazon API Gateway for the public interface. Start with the API contract and let the infrastructure disappear.

I built a POC for a customer recently using a few thousand of their actual search documents. I used an AI coding tool to build out all of the front-end logic, including agent-driven search, and a search back end supporting hybrid search on OpenSearch Serverless. The whole process took a couple of hours from concept to finished implementation. No servers to size. No clusters to monitor. No capacity planning spreadsheets. Serverless all the way through.

I deployed the search back end into an OpenSearch Serverless collection. A collection is a logical grouping of indexes: I pointed the data at the collection, and the service handled indexing, scaling, and availability automatically. When we load-tested with simulated traffic, compute scaled up to match. When the test ended and traffic dropped to zero, compute dropped to zero and the customer paid for stored data only. Collection groups give you the knobs to control cost (set capacity limits, group collections by workload profile) without ever choosing instance types or node counts. The operational decisions that normally take weeks of planning just do not exist.

Where the Intelligence Lives

The Lambda function is where query logic lives. A Python script authenticates to OpenSearch Serverless using IAM credentials and processes search queries. The key technique is the multi-match query, which searches the same text across multiple fields simultaneously. When someone searches for "Harry Potter," the function looks in titles, plot summaries, and actor names at once, then ranks results by relevance. Matches across multiple fields rank higher than single-field matches. Exact matches outrank partial ones.

I have overcomplicated multi-field search myself, and I have watched others do the same. The instinct is to build separate queries for each field, then merge and rank results in application code. The result is slower, more complex, and produces worse rankings. OpenSearch is already optimized for multi-field relevance ranking. Let the search engine do what the search engine is built to do.

API Gateway sits in front as the public interface. API Gateway handles throttling, validation, and request routing. You set rate limits (1,000 requests per second with a burst allowance of 500) and the gateway enforces the limits automatically. No rate-limiting logic in your Lambda function. No DDoS protection to build from scratch. The gateway handles the HTTP layer so your search logic can focus on being search logic.

The data access layer uses IAM policies instead of a separate authentication system. Your Lambda function gets permission to query the collection through standard AWS IAM roles. No API keys to rotate, no credentials to leak, no separate authentication database to maintain. The same identity and access management system you already use for everything else in AWS.

The entire stack is deployable through CloudFormation or Terraform, which means the application is reproducible from day one. Define the collection, the Lambda function, the API Gateway, the IAM roles, and the VPC endpoint in a single template. Spin up the whole application in one deploy, tear it down just as fast, replicate it across regions or accounts without clicking a single console button. For network isolation, OpenSearch Serverless supports VPC endpoints: your Lambda function reaches the collection without traffic leaving the AWS network.

Wire It Up and Walk Away

Wire the components together and the request flow is almost boring. User types a query. JavaScript sends the query to API Gateway. The gateway validates and routes to Lambda. Lambda queries OpenSearch Serverless. Results flow back through the same chain. The entire round trip happens in milliseconds, and you wrote maybe 200 lines of actual code.

The shift to a serverless application architecture changes what you spend your time on. That customer I mentioned at the start eventually rebuilt as a serverless application. Two weeks later, the customer was working on search relevance tuning and user experience improvements. The work that actually matters.

The future of search-backed applications is that the search infrastructure becomes invisible. Not weaker, but invisible. You define what you want to search and how you want results ranked. The underlying compute and storage arrange themselves automatically.

Adding semantic search, vector embeddings, or ML-powered ranking should not require a complete infrastructure redesign. With a serverless application architecture, those capabilities become configuration changes and code updates, not migration projects. OpenSearch Serverless already supports vector search alongside keyword search in the same collection.

One thing I would recommend doing this week: look at your current search implementation and calculate how much time goes to infrastructure versus search quality. If the ratio is anything other than heavily skewed toward quality, you are solving the wrong problems. The best search experiences I have seen come from people who spend their time on user intent and relevance, not from people who have mastered Kubernetes deployments.

The infrastructure should be boring. The application should be where you invest your creativity. Wire the three components together and walk away from the cluster. OpenSearch Serverless handles the rest.

Your AI Agent Just Read the man Pages. OpenSearch Agent Skills Turn Your IDE Into a Search Engineer.

Jon Handler — Thu, 27 Aug 2026 14:52:48 +0000

You are probably already using AI to write code. Claude Code, Cursor, Kiro, GitHub Copilot, or some combination. You prompt, you get functions, you move fast. But then you need to build a search application and the workflow fractures. You leave your IDE to read OpenSearch documentation. You switch to the AWS console to provision a domain. You open a terminal to configure indices and pipelines. You write client code to connect everything. Each step requires a different tool, a different mental model, and a different tab.

Your AI agent can generate the code for each piece. What your agent cannot do (until now) is make the architectural decisions, execute the configuration, and verify that the whole system works together. The agent autocompletes functions but has no idea how to actually stand up a search application end to end.

OpenSearch Agent Skills change that. Agent Skills are executable workflows, packaged as SKILL.md files, that give your AI coding agent the domain expertise to build, configure, and deploy OpenSearch applications from natural language. Not documentation. Not code snippets. Actual capabilities your agent can run against your infrastructure.

What Ships Today

The OpenSearch project ships skills covering the workflows where context-switching hurts most: search application development, log and trace investigation, document processing, and cloud deployment to Amazon OpenSearch Service and Serverless.

The Search skill (opensearch-launchpad) builds complete search applications from a single prompt. Tell your agent "build a hybrid search app for my product catalog" and the skill handles index creation, ingest pipeline configuration, ML model registration with Amazon Bedrock, retrieval strategy selection (BM25, semantic, hybrid, or agentic), and a working search UI. The skill supports four retrieval strategies and knows when to recommend each one. Ask for a search bar and it configures a flow agent. Ask for a chatbot and it sets up a conversational agent with memory. You stay in your IDE the entire time.

The Logs skill (log-analytics and trace-analytics) brings the same pattern to observability. Instead of manually writing PPL queries to hunt for error patterns or navigating trace waterfalls to find the slow span, you describe the symptom: "why is my service returning 500s?" The skill queries your log data, identifies error patterns and volume anomalies, investigates distributed traces to find failing spans and service dependencies, and correlates logs to traces by traceId. Root cause analysis that used to require expertise in PPL syntax and trace data structures now requires a sentence.

The Cloud skills (aws-setup, aiven-setup) handle deployment. Once your search application works locally in Docker, the cloud skill deploys the same configuration to Amazon OpenSearch Service or Serverless. No manual console work, no translating local settings into production infrastructure. The Ingest skills cover two paths: document-processing handles local document ingestion via Docling (parse PDFs, HTML, office documents into indexable chunks), while managed-ingestion-service configures Amazon OpenSearch Ingestion pipelines with Automatic Semantic Enrichment for production scale. Separately, the OpenSearch Migration Assistant handles Solr-to-OpenSearch migrations as its own packaged tool, though Agent Skills may absorb that workflow in a future release.

What "Build a Search App" Actually Looks Like

Here is the search skill in action. You open Claude Code (or Cursor, or Kiro) and type: "Build a hybrid search application for my product documentation." The agent activates the opensearch-launchpad skill and moves through five phases.

First, data strategy. The agent asks what data you want to search and offers to work with your files, pull from a URL, or use a built-in sample dataset for prototyping. Second, retrieval strategy. Since you said hybrid, the agent knows to configure both BM25 keyword matching and semantic vector search with a neural pipeline. Third, architecture planning. The agent presents the full plan: index name, field mappings, embedding model from Amazon Bedrock, ingest pipeline configuration, search pipeline with hybrid scoring, and a React frontend. You review and approve.

Fourth, execution. The agent spins up a local OpenSearch cluster in Docker, creates the index, registers the Bedrock model, builds the ingest and search pipelines, indexes your data, and launches the UI. You have a working hybrid search application running locally. Fifth, deployment. The agent offers to deploy the exact same configuration to Amazon OpenSearch Service. Not a re-implementation. The same domain, same index, same pipelines, same model connector. Local-to-cloud with no translation layer.

The whole sequence takes minutes. The agent made the architectural decisions (which retrieval strategy, which embedding model, how to wire the pipeline), executed the configuration, verified the results, and deployed. You never left your IDE. You never opened the AWS console. You never read a documentation page about ingest processor syntax.

How Skills Work (The man Pages Analogy)

Think of Agent Skills as pre-read man pages with executable authority. Your AI agent already knows how to generate code. What the agent lacks is the domain-specific judgment about what to configure, in what order, with what settings. Agent Skills supply that judgment as structured SKILL.md files: step-by-step workflows, reference documentation, and executable scripts bundled together.

Skills load on demand. You can install the full collection without bloating your agent's context window. When you say "build a hybrid search app," the agent activates only the matching skill, follows its workflow, and calls the right OpenSearch APIs. When you say "investigate my 500 errors," a different skill activates. Each skill is small enough to fit in a tight context window but complete enough to handle real end-to-end workflows.

The format is agent-agnostic. Agent Skills were developed by Anthropic as a lightweight, open specification. Any agent that supports the Agent Skills protocol can use them: Claude Code, Cursor, Kiro, VS Code, GitHub Copilot, Codex. You install once and every compatible agent in your workflow gains the same capabilities.

Getting Started

Installation is one command. Run npx skills add opensearch-project/opensearch-agent-skills and your agent gains access to the full skill library. Or install specific skills: npx skills add opensearch-project/opensearch-agent-skills@opensearch-launchpad --full-depth for search, @log-analytics for log investigation, @trace-analytics for distributed traces, @aws-setup for cloud deployment. Prerequisites are Python 3.11+, uv, and Docker running locally. AWS credentials are optional (needed only for deploying to OpenSearch Service).

Once installed, express your intent. "I want to build a semantic search app with OpenSearch." The agent reads the skill instructions, runs the scripts, and handles the rest. No MCP server required. No additional tooling. The skills are files your agent reads and executes.

This Is Open Source. Build With Us.

The skills repository lives at github.com/opensearch-project/opensearch-agent-skills. It is open source, Apache 2.0 licensed, and accepting contributions. The launch skills cover search, observability, ingest, and cloud deployment. The roadmap includes dashboards, security configuration, migration workflows, and more. But the most interesting skills will come from people solving real problems in production.

If you have built a workflow that reliably configures OpenSearch for a specific use case, that workflow is a candidate for a skill. Package the steps, the reference docs, and the scripts into a SKILL.md file and submit a PR. Every skill you contribute means every developer using an AI coding agent gains that expertise automatically. You solve a problem once. The entire community benefits.

The $11,000/Month diff: You're Shipping 600 DPI When 150 Gets the Job Done

Jon Handler — Tue, 25 Aug 2026 15:07:50 +0000

You loaded 125 million product embeddings into your vector search cluster. The math looked fine on paper: 1536 dimensions, HNSW indexing, three availability zones. Then the first bill arrived. Eighteen thousand dollars a month. Your entire text search infrastructure costs six thousand. The vector layer alone is triple that.

The product manager asked a reasonable question: why does a column of numbers cost more than the rest of the application combined? The answer is precision nobody needs.

Think of vector storage like shipping a library across the country. You could photograph every page at 600 DPI, lossless TIFF, archival quality. Or you could photograph them at 150 DPI JPEG, more than enough for anyone who wants to read the words. The first option fills a shipping container. The second fits in a suitcase. Vector search, by default, ships the container.

Why Vectors Blow Up Your Budget

A single 1536-dimension embedding at full precision: 12,288 bytes. Each dimension is a 4-byte float. Multiply by 125 million: 1.5 terabytes of vector storage before you think about the search graph.

HNSW builds multi-layer graph structures that need RAM for fast queries. The formula: (1.1 × 4 × dimensions) + (8 × edges × vector_count). With defaults and 125 million vectors: 862 GB of RAM.

Traditional text search is disk-bound. Vector search is memory-bound. The graph lives in RAM or queries take seconds. You cannot solve this by adding nodes. More nodes = more shards = the graph splits into smaller neighborhoods = degraded recall. The scaling pattern that works for logs does not work for vectors.

The Precision You're Paying For

Embedding models default to 1536 dimensions at 32-bit float. You can choose lower-dimensional outputs and reduced precision, but mostly you take the default because models benchmark best at full width.

The question nobody asks at architecture time: how much of that precision matters for retrieval?

Retrieval is not final ranking. You are narrowing 125 million candidates to a few hundred, then rescoring with a cross-encoder or business logic. The initial pass needs the right neighborhood, not perfect ordering within that neighborhood. Binary precision (1 bit per dimension) maintains 90%+ recall for that job.

The Cost-Latency-Accuracy Dial

Amazon OpenSearch Service gives you several points on this dial:

Full precision in RAM (FP32): 862 GB RAM, single-digit ms latency, $18,000/month.

FAISS FP16 in RAM: ~430 GB, single-digit ms, negligible accuracy loss. ~$10,000/month.

INT8 or binary vectors in RAM: quarter or 32x reduction. Latency stays fast. Accuracy depends on your data — for retrieval into a reranker, most workloads absorb it.

On_disk + binary quantization: graph on SSD, 44 GB RAM, ~200 ms p50, 95% accuracy. $7,000/month. Best for RAG pipelines, agent context assembly, batch reranking.

Dimensionality: 768-dim instead of 1536 halves the footprint at every level. Stacks with all of the above.

Operational Tuning

Shard sizing: vector workloads want 50-75 GB per shard (not the 10-30 GB text search guidance). HNSW graphs work better with denser neighborhoods. Fewer, bigger shards = better recall.

Off-heap memory: OpenSearch caps heap at 32 GB. Remaining memory splits between filesystem cache and k-NN vector cache. The circuit breaker knn.memory.circuit_breaker.limit (default 50%) controls how much off-heap goes to vectors. Raise it for vector-heavy workloads:

# Give 75% of off-heap to the k-NN graph cache
knn.memory.circuit_breaker.limit: 75%

On a 128 GB node: 32 GB heap + 96 GB off-heap. At 75%, vectors get 72 GB — plenty for a quantized graph with headroom.

Run diff On the Deployment

Configuration	RAM	Latency	Monthly Cost
Full precision (FP32) in RAM	862 GB	<10 ms	$18,000
FAISS FP16 in RAM	~430 GB	<10 ms	~$10,000
On_disk + binary quantization	44 GB	~200 ms	$7,000
768-dim + FP16 in RAM	~215 GB	<10 ms	~$6,000

$11,000/month between the extremes. $132,000/year. But you do not have to choose an extreme. Pick the point on the dial that matches your latency budget.

The Actual Lesson

If your CloudWatch memory utilization sits above 60% with full-precision vectors, you have room to move down the dial. Test FP16 on a shadow index. Measure the accuracy delta on your real queries.

Back to the product manager's question: why does a column of numbers cost more than the rest of the application? Because each number is stored at 32x the precision retrieval needs, and the graph lives entirely in RAM. Fix the precision, and the column of numbers costs less than the text catalog it serves.

Your Search Engine Is Running grep on a Dictionary. OpenSearch Automatic Semantic Enrichment Fixes That.

Jon Handler — Fri, 21 Aug 2026 17:05:20 +0000

Traditional search is a glorified Ctrl+F. If your beach sandals are labeled "water-resistant footwear" in your product database, nobody searching for "beach shoes" will find them. "Water," "resistant," and "footwear" do not match "beach" or "shoes," so the search engine returns nothing. Users who cannot find what they want within two or three searches leave the site entirely. They do not email support. They do not try different keywords. They buy from the competitor whose search finds products where related but missing terms also describe those products.

The familiar fix is synonyms. Teams build elaborate synonym dictionaries: "sneakers = shoes = footwear = kicks = trainers." They maintain spreadsheets with thousands of mappings. Every time a product fails to surface, someone adds another row. One retail team had a dedicated person whose entire job was updating the synonym list. Language is not a lookup table, though. "Kicks" means shoes in one sentence and something else entirely in another. "Trunk" is luggage, part of a car, or an elephant's nose depending on surrounding words. No manual mapping captures the contextual, distributional nature of how words actually cluster in usage.

The other common approach is full semantic search from scratch: spin up ML infrastructure, train or fine-tune embedding models, manage vector databases, build pipelines that transform text into dense numerical representations. Converting text into vectors that correlate terms by co-occurrence works well. But the infrastructure overhead is real. You manage model serving, handle inference latency, version models, monitor drift, and keep everything running at scale. Teams that go this route often discover that their inference costs at production scale are astronomical because the model runs on every single search query. They solved the relevance problem and created a performance and cost problem in its place.

Automatic Semantic Enrichment in Amazon OpenSearch Serverless keeps the relevance without the per-query cost. At index time, the system generates sparse embeddings from your content, capturing which terms co-occur in similar contexts across millions of documents, and stores those embeddings alongside the original text. No ML infrastructure, no model management, no changes to your application code.

Why Sparse Vectors

Automatic Semantic Enrichment supports both sparse and dense embeddings. Sparse is the path worth starting with. A sparse vector is a weighted list of correlated terms: the model identifies which words co-occur with your content across large corpora and stores those associations as rank features in the inverted index. No approximate nearest neighbor graph. No vector database. No new query syntax. You search with text, and the inverted index does what inverted indices have always done, except now the vocabulary is expanded beyond what the author originally wrote.

The model runs once, at ingest time. When a document enters the index, OpenSearch Serverless passes the text through a sparse encoder trained on distributional co-occurrence patterns. The encoder outputs a weighted term list: "beach sandals" produces rank features for "waterproof," "summer footwear," "flip-flops," and other terms that appear in overlapping lexical contexts, even though none of those words exist in the original text. Those features are stored alongside the document. At search time, a lightweight tokenization process matches the query against both the original text and the enriched terms. No full model inference on the hot path.

If your relevance requirements are more demanding, Automatic Semantic Enrichment also supports dense embeddings with bi-encoding: full vector representations for both documents and queries, matched by approximate nearest neighbor search. Dense vectors capture finer-grained distributional relationships and generally rank better on hard queries. But dense requires a vector index, ANN infrastructure, and model inference at query time. Sparse gives you most of the relevance lift for a fraction of the operational weight. Start sparse. Move to dense when you have the query volume and relevance data to justify the added infrastructure.

The cost structure reinforces the choice. With sparse enrichment, you pay for model inference once per document at ingest. Search queries hit the inverted index without touching the model. No per-query inference cost, no GPU fleet at search time, no latency spike when traffic surges. Your search cost scales with query volume against a standard inverted index, not with model inference. For most workloads, this is the difference between a search feature you can afford to run at scale and one that lives permanently in staging because production costs are unpredictable.

The grep Moment

Here is where the grep analogy breaks. Traditional lexical search is grep "scarlet sneakers" catalog.txt. No match, no result, end of story. Automatic semantic enrichment is more like having a pre-built thesaurus of co-occurrence patterns baked into the index itself. The query does not need to match the exact string because the index already knows which terms cluster together based on how they appear across millions of documents.

Back to the beach sandals. A customer searches for "shoes for the beach." Your product catalog says "water-resistant footwear." Lexical search returns nothing. With Automatic Semantic Enrichment, the sparse encoder already identified at ingest time that "beach," "sandals," "water-resistant," "footwear," and "shoes" all co-occur in overlapping lexical contexts across the training corpus. Those correlations are stored as rank features in the index. The query "shoes for the beach" matches against those enriched terms and surfaces the product without a single exact word match in the original listing.

Setup Is Embarrassingly Easy

You create an index, designate which fields should have automatic semantic enrichment, choose a language model (English or multilingual), and OpenSearch Serverless creates the ingest and search pipelines automatically. You index documents as plain text. You search with plain text queries. The semantic enrichment happens invisibly during ingestion. AWS benchmarks show up to 20% better search accuracy compared to pure lexical matching.

The approach augments lexical search rather than replacing it. You keep traditional text fields alongside semantic fields. Exact matching works when you need precision. Semantic correlation works when the user's vocabulary does not match your catalog's vocabulary. Both live in the same index.

Where This Is Heading

The default expectation for search is shifting. Users trained on Google, voice assistants, and ChatGPT do not think in terms of keyword matching anymore. They expect search to correlate vocabulary, not match strings. The companies that ship semantic correlation first have a structural advantage: not because their products are better, but because customers can actually find those products.

If you are running e-commerce search, documentation search, internal knowledge bases, or any retrieval system where vocabulary mismatch causes failed queries, automatic semantic enrichment removes the friction without requiring ML expertise, model management, or changes to your application architecture. Pick one high-value use case where search is failing users. Enable enrichment. Measure the difference in search success rates. The improvement is measurable within days, and the setup takes minutes rather than sprints.

What If Your Search Infrastructure Only Existed When Someone Searched?

Jon Handler — Wed, 19 Aug 2026 16:30:01 +0000

Create a collection. Send data. Query. That is the entire setup for Amazon OpenSearch Serverless. No cluster sizing spreadsheet. No capacity planning meeting. No argument about how many nodes you need for peak versus average. You create, you use, you pay for what you used. When you're not using it, you pay for storage and nothing else.

That last part is new, and it is the part worth getting excited about.

The joy of just using it

OpenSearch Serverless scales compute to zero when there is no activity. Not "scales to a minimum baseline." Zero. Your dev/test collection that nobody touches on weekends costs you storage and nothing else until Monday morning. Your QA environment that runs a test suite for 20 minutes a day pays for 20 minutes of compute. Your production search that handles a thousand queries per second at noon and twelve queries per second at 3 AM pays for a thousand at noon and twelve at 3 AM.

Indexing and search compute scale independently. This is the part that deserves a pause. If you have steady-state ingest running (a CDC pipeline feeding documents from your database, say), the search side still scales down when nobody is querying. You do not pay search compute costs to keep your ingest running. Two workloads, two scaling curves, one collection. The economics of running both together just changed.

Autoscaling responds in seconds. When a marketing campaign drives a traffic spike at noon, compute scales up immediately to match demand. Your users see the same latency at 10x traffic as they saw at 1x. When the campaign ends and traffic subsides overnight, compute scales back down just as fast. By morning, if nobody is querying, the collection is back at zero compute cost. This cycle happens automatically, every day, without alarms, without manual intervention, and without you ever provisioning for a peak that may or may not arrive. The infrastructure mirrors your actual workload in real time, and the bill reflects what actually happened rather than what you feared might happen.

What this looks like in practice

In a recent demo, our team built a hybrid search application (semantic vector search combined with keyword search) against a dataset of 5,000 movies. The entire setup happened through OpenSearch Agent Skills: pre-built capabilities that let an AI agent (Kiro, Claude Code, or any MCP-compatible tool) provision and configure OpenSearch Serverless from natural language.

The agent created a serverless collection, set up IAM roles, configured an Amazon Bedrock connector for Titan text embeddings (1,024-dimension vectors), created ingestion and search pipelines, indexed the movie data, and built a React frontend. All from a single instruction: "Build a search application using the movies data in my local folder."

The search worked as expected. A query for "racing" returned semantically relevant movies, not just exact title matches. Hybrid search handled both concept-based and precise queries. But the infrastructure behavior is what stood out.

After the demo ended and traffic stopped, compute dropped to zero OCUs (OpenSearch Compute Units) within minutes. The collection existed. The data remained indexed. The cost was storage only. When we ran queries again, compute spun up in seconds and the collection responded at full speed.

For teams evaluating hybrid search, the Bedrock integration and automated pipeline setup remove most of the friction. You can prototype a semantic search system in an afternoon and pay nothing for compute between prototype sessions.

Where this matters most

Agentic AI workloads are the clearest fit. Agents are bursty and unpredictable by nature: an agent might fire 50 search queries in 10 seconds during a research task, then go silent for an hour. Provisioning for peak agent traffic wastes money. Provisioning for average risks latency during bursts. Scale-to-zero with seconds-fast autoscaling handles both without you thinking about it.

The same economics apply anywhere traffic is uneven or intermittent. Dev/test environments sit idle 90% of the time, burning compute that nobody is using. QA workloads run a test suite for a few minutes each day, then sit dark until tomorrow. Staging environments spin up for weekly deploys and do nothing in between. Internal tools see a flurry of activity during business hours and silence overnight. Even production search has genuine off-peak hours where query volume drops to a fraction of daytime traffic. In every one of these cases, you are paying for compute capacity that has nothing to do. OpenSearch Serverless makes that idle compute disappear from your bill.

The independent scaling of indexing and search unlocks a pattern that was previously expensive: continuous ingest (keeping your search index fresh from a database CDC stream) without paying for search compute during low-traffic hours. Your data stays current. Your search costs reflect actual query volume, not ingest activity.

Compared to provisioning OpenSearch Service domains for peak capacity, OpenSearch Serverless can deliver up to 60% lower cost for workloads with variable traffic patterns. The savings come from not paying for compute that has nothing to do.

The `chmod +x` moment

OpenSearch Serverless makes search executable without infrastructure work. You create a collection, point your data pipeline at it, and start querying. Scale happens underneath you. Cost tracks what you actually use. You do not monitor OCU utilization graphs. You do not wake up to adjust capacity for a Monday morning traffic pattern that differs from Sunday. The infrastructure disappears into the background where it belongs, and you spend your time on the search experience your users see rather than the plumbing they never should.

If you are currently sizing clusters, managing OCU minimums from the previous generation, or over-provisioning because you cannot predict traffic, the math just changed. Check your CloudWatch utilization graphs. If average utilization is below 50%, you are paying for compute that OpenSearch Serverless would scale away automatically.

Documentation: Amazon OpenSearch Serverless (https://aws.amazon.com/opensearch-service/features/serverless/)

Read Your AI Agent's Mind. OpenSearch Observability for LLM Traces.

Jon Handler — Mon, 17 Aug 2026 16:45:13 +0000

You shipped your AI agent. It answers questions, calls tools, retrieves documents, and mostly does the right thing. Mostly. When it does the wrong thing, you have no idea why. The prompt looked fine. The retrieval seemed relevant. Somewhere between the user's question and the final answer, something went sideways, and you are staring at a log that says "200 OK" while your customer stares at nonsense.

APM dashboards show you HTTP latency and error rates. They have no idea what your agent is thinking. An agent is not a request-response cycle. An agent reasons, branches, calls tools, re-evaluates, and sometimes loops. Tracing that execution requires a different kind of observability, one that understands LLM calls, token budgets, tool invocations, and multi-step reasoning flows. OpenTelemetry now provides semantic conventions for exactly this, and Amazon OpenSearch Service gives you the full stack to collect, store, query, and visualize those traces in one place.

OpenTelemetry learned to trace agents

OpenTelemetry (OTel) is the open standard for distributed tracing, metrics, and logs. If you run microservices, you probably already emit OTel spans for HTTP calls and database queries. The recent addition of generative AI semantic conventions extends that vocabulary to cover LLM-specific operations. Every model call, every tool invocation, every reasoning step gets a standardized span with attributes like model name, token counts (input and output), temperature, tool names, and agent step identifiers.

A few lines of setup and every agent execution emits a structured trace. A GenAI SDK (Python or TypeScript) provides decorators and auto-instrumentation for popular frameworks: Strands Agents, LangGraph, CrewAI, the OpenAI Agents SDK. It hooks into providers like OpenAI, Anthropic, Amazon Bedrock, and LangChain without requiring you to rewrite application code. You add a few lines of setup, and every agent execution emits a structured trace that captures the full decision tree.

What makes this different from logging model calls yourself? Structure and correlation. Each span carries a trace ID that links it to the parent request. A single user question might trigger three LLM calls, two tool invocations, and a retrieval step. OTel stitches them into a single trace tree with timing, token cost, and dependency relationships intact. Without this correlation, you are left grepping through scattered logs trying to reconstruct what happened.

What agent traces tell you

Structured traces make previously impossible questions answerable without manual debugging.

Where is the agent spending tokens? Token usage maps directly to cost. A trace shows you that 80% of your tokens went to a single retrieval-augmented call that stuffed too much context. Or that the agent retried a tool call three times because the first two responses were malformed. You cannot optimize what you cannot measure, and token-level attribution across a multi-step flow is exactly what agent traces provide.

Why did the agent choose that tool? Agent frameworks make routing decisions: which tool to call, in what order, with what parameters. The trace captures each decision as a span. When the agent calls a calculator instead of a database lookup, the trace shows you the reasoning step and input that led there. That visibility turns "the agent hallucinated" from a mystery into a debuggable event.

How long does each step take? Latency in agentic applications compounds. If each LLM call takes 2 seconds and the agent chains four of them, your user waits 8 seconds before seeing a response. The trace timeline exposes these serial dependencies, letting you identify which calls can be parallelized or cached. The difference between a 2-second and a 10-second agent response is often a single unnecessary sequential step.

Is the agent looping? Agents with autonomous tool use can fall into retry loops or circular reasoning patterns. A trace that shows 15 spans where you expected 4 is an immediate signal. OpenSearch Dashboards renders these as hierarchical trees and directed acyclic graphs (DAGs), making pathological patterns visually obvious before they drain your token budget.

OpenSearch Service is the easy button

Any OTLP-compatible backend can store these traces. OpenSearch Service stands out because it provides the full pipeline as a managed, unified stack: collection, processing, storage, querying, visualization, alerting, and anomaly detection. No stitching together four separate tools.

From your application to a dashboard, the pipeline has four stages. Your application emits OTel spans through the GenAI SDK. An OpenTelemetry Collector normalizes them using the generative AI semantic conventions. Amazon OpenSearch Ingestion (Data Prepper) routes the OTLP data into your OpenSearch Service domain, creating service maps, correlating traces with logs, and computing RED metrics (Rate, Errors, Duration) automatically. OpenSearch Dashboards then renders agent traces as explorable trees, DAGs, and timelines with token-cost overlays.

Open the Agent Traces panel in OpenSearch Dashboards and you see something APM views never show you. It differs from the standard APM view (which monitors HTTP services) in that it understands model calls, token budgets, and reasoning flows as first-class concepts. You see a hierarchical breakdown of each agent execution: which model was called, what it cost, which tools fired, how long each step took, and whether the agent looped or branched. This is not a general-purpose tracing UI adapted for AI. It is a dedicated interface built on OTel's generative AI conventions.

When token cost spikes or an agent loops unexpectedly, OpenSearch Service fires alerts on trace data (fire a Slack notification when token cost per request exceeds a threshold), anomaly detection on agent behavior (flag when a normally 3-step agent suddenly takes 12 steps), and log-to-trace correlation (click from a trace span directly to the error log that explains the failure). These capabilities exist in one domain, under one security model, queried with one language.

Now you know

Your agent is no longer a black box. OTel's generative AI conventions give you structured, correlated traces across every LLM call, tool invocation, and reasoning step. OpenSearch Service stores those traces, visualizes them as explorable execution graphs, alerts on anomalies, and correlates them with your existing logs and metrics. The operational model for AI agents just caught up with the operational model you already have for microservices.

Think of it as tail -f for your agent's brain. Except instead of scrolling text, you get a full execution graph with cost attribution, latency breakdowns, and anomaly detection built in. The next time your agent does something inexplicable, you will not be guessing. You will be looking at the trace.

To get started, check the OpenSearch Observability platform page and the agent traces documentation. The OpenSearch Observability Stack provides a Docker Compose setup with a preconfigured OTel Collector, Data Prepper, and example applications so you can see agent traces working end to end in minutes.

1TB Migrated in 33 Minutes. The Agent Didn't Even Need Coffee.

Jon Handler — Fri, 14 Aug 2026 16:10:16 +0000

In a recent demo, our team migrated one terabyte of production data from Elasticsearch 7.10 running on Amazon OpenSearch Service to OpenSearch 3.x running on OpenSearch Service in 33 minutes. The migration was orchestrated entirely by an AI agent. No runbook. No manual configuration files. No engineer babysitting terminal windows at 2 AM. The agent made infrastructure decisions, monitored progress, handled errors, and completed the migration while the team watched from the sidelines.

The toolchain was Kiro connected to Migration Assistant for Amazon OpenSearch Service running on EKS. Migration Assistant is open source and supports migrations from Elasticsearch, self-managed OpenSearch, and Apache Solr (versions 6.x through 9.x) to both Amazon OpenSearch Service domains and Amazon OpenSearch Serverless collections. It handles snapshot-based backfill, live traffic capture and replay for validation, and metadata translation. The AI-assisted experience lets you drive the entire workflow from Kiro, Claude Code, or any MCP-compatible tool: the agent plans the migration, deploys infrastructure, executes the data movement, monitors progress, and validates results. In this demo, the result challenges assumptions about what still requires human hands during a database migration.

Why migrations still eat engineering weeks

Database migrations have three layers of difficulty. The first is planning: figuring out which indices to migrate, what version incompatibilities exist, how to handle mapping differences, and what order to execute steps. The second is execution: configuring snapshots, parallelizing backfill, managing network throughput, and coordinating the cutover. The third is monitoring: watching progress, detecting errors, calculating ETAs, and deciding whether a failure is recoverable or fatal.

Most teams solve all three layers with the same tool: a senior engineer. That engineer writes the plan, executes it step by step, monitors progress manually, and makes judgment calls when things go wrong. The work is cognitively demanding but largely repetitive across migrations. The cluster sizes differ. The index counts differ. The judgment calls are the same patterns applied to different numbers.

This is the kind of work where AI agents should excel: well-defined operations with clear success criteria, where the decisions follow from observable state rather than ambiguous business context. The question is whether the tooling is mature enough to let an agent make those decisions reliably. Based on what we observed, the answer is yes.

What the agent actually did

The demo started with Kiro connected to a Migration Assistant deployment running on EKS. The Migration Assistant was bootstrapped in a fresh VPC via CloudFormation. The source was an Amazon OpenSearch Service domain running Elasticsearch 7.10 with 1 TB of primary data (2 TB including replicas). The target was an Amazon OpenSearch Service domain running OpenSearch 3.x.

Kiro scanned the AWS account, discovered the source and target clusters, and presented options. When told to migrate as fast as possible with no other workloads running, the agent adjusted its approach accordingly. It set snapshot throttling high (800 MB/s) because there was no competing I/O to protect. It calculated optimal parallelization at 480 concurrent reindex pods, matching the shard count. It configured EKS auto mode to provision EC2 instances on demand so the infrastructure scaled to match the workload.

None of these decisions came from a template. The agent queried the source cluster for real-time state and made context-aware choices. If other workloads had been running, it would have throttled the snapshot to avoid disk contention. The throttling decision came from understanding the tradeoff between migration speed and source cluster impact.

The timeline

The 33-minute migration broke down as follows. Snapshot took 5 minutes at full throttle. Metadata migration (index mappings, aliases, templates) took 1 minute. Backfill across 480 parallel pods took 12 minutes. The remainder was setup, validation, and the agent confirming success. End to end, from first prompt to validated completion, 33 minutes for a terabyte.

Monitoring and error handling

During the snapshot phase, Kiro queried the source cluster every 30 seconds for progress metrics. During backfill, it recalculated ETAs every minute. This monitoring behavior was not preconfigured in the Argo workflow. The agent added it because that is what a human operator would want to know. The feedback loop from these AI-assisted runs is now informing what gets built into the Migration Assistant itself.

The agent caught errors and decided what to do with them. The agent distinguished between recoverable errors (retry) and fatal errors (stop and surface for human investigation). The team could have disconnected entirely and the migration would have completed successfully. That level of autonomy is the point. The agent was not generating scripts for a human to run. It was running infrastructure.

AI handles choreography. Humans handle strategy.

Watch this demo and a clear division of labor emerges. The humans decided which clusters to migrate, what performance tradeoffs to accept, and whether to prioritize speed over source cluster stability. The agent handled everything else: the configuration, the monitoring loops, the progress checks, the validation queries, and the error recovery.

The point is not fewer engineers. The point is engineers spending time on decisions, not choreography. A 1 TB migration that previously required 4-6 hours of active senior engineering time now requires a few prompts and strategic oversight. The operational choreography disappears. The judgment about what to migrate and why stays with the human.

Fifty cents. That is what the AI orchestration for the entire migration consumed in compute. Compare that to the fully loaded cost of a senior engineer spending half a day on the same work. The economics push toward using the agent for execution and reserving human time for decisions that require business context.

The tooling is open source

Migration Assistant for Amazon OpenSearch Service is open source at opensearch-project/opensearch-migrations and supports Elasticsearch, Solr, and self-managed OpenSearch as sources. The s/runbook/agent/ substitution is available today. Point Kiro, Claude Code, or any MCP-compatible tool at the Migration Assistant. The agent handles the infrastructure decisions, the progress monitoring, and the error recovery. The senior engineer who used to babysit the migration can now review the plan, approve it, and check back when the agent reports completion.

Medium tags: opensearch, elasticsearch, migration, ai-agents, devops

Forget Total Recall. Give Your AI Agent Selective Memory with OpenSearch.

Jon Handler — Tue, 11 Aug 2026 22:44:22 +0000

Imagine handing an LLM the full text of Anna Karenina and asking what Levin thinks about farming. The model has the information somewhere in its window. Good luck getting a focused answer. The number of tokens you can pass to an LLM in a single call can grow to 200K and beyond, but the problem is not capacity. The problem is that LLMs diverge when there is too much information. Relevance degrades with volume. Past a threshold, adding more context makes answers worse, not better. This is why "just make the window bigger" is not a memory strategy.

AI agents need memory that is selective, persistent, and searchable. Not a transcript. Not a sliding window. A system that extracts what matters, stores it durably, and retrieves only the right pieces when the agent needs them. This is a search problem, and Amazon OpenSearch Service now provides purpose-built APIs for exactly this: agentic memory that gives your agents persistent, semantically searchable recall across conversations.

The context window is not memory

LLMs process text in a fixed-size window. GPT-4o gives you 128K tokens. Claude gives you 200K. That sounds like a lot until you try to use it as a memory system. A single week of customer interactions for one user might run 50K tokens. A month blows past any context limit. And even if the window were infinite, retrieval latency and cost scale linearly with token count. You are paying for every token you stuff in there, and most of it is irrelevant to the current question.

The naive solution—append the last N messages—fails on both ends. If N is small, you lose information that matters. If N is large, you lose relevance. The agent drowns in a sea of context where the signal-to-noise ratio degrades with every turn. The customer said they prefer conservative investments six months ago, once. That single statement buried under 200 subsequent messages is invisible to a sliding-window approach.

What you actually want is not a transcript. You want a system that extracts the important pieces, stores them persistently, embeds them for semantic retrieval, and hands back only the facts that matter for the current moment. You want the agent to know that this customer prefers conservative investments without needing to re-read six months of transcripts. That is a search problem, not a context-window problem.

And search is what OpenSearch Service does.

OpenSearch Service agentic memory

OpenSearch Service provides a purpose-built agentic memory system through its ml-commons plugin. The system organizes memory into containers—one per agent or use case—and supports four distinct memory types that work together.

Sessions track conversation metadata: who participated, when the conversation started, what state the interaction reached. Think of sessions as the filing system. Each session is a distinct interaction context, and the memory system tags all stored information with session identifiers so retrieval can scope to the right timeframe.

Working memory holds the active conversation data and agent state during an ongoing interaction. Raw messages, execution traces, current task progress, and temporary data all live here. This is the short-term scratchpad your agent writes to during a session and reads from when the LLM needs immediate context.

Long-term memory is where the real value accumulates. When inference mode is enabled (infer: true), OpenSearch Service passes the conversation through an LLM that extracts key facts, preferences, and insights. Those extracted pieces are embedded as vectors and stored persistently. Six months later, when the same customer calls back, a semantic search against long-term memory surfaces "prefers conservative investments" without loading a single old transcript.

History maintains an audit trail of every memory operation—adds, updates, and deletes—across the container. This gives you the ability to trace how an agent's knowledge evolved and debug unexpected behavior.

Memory processing strategies

Raw storage is only part of the system. OpenSearch Service offers three processing strategies that automatically organize memories as they arrive. The SEMANTIC strategy groups related memories by meaning—conversations about retirement planning cluster together even if the words used differ from session to session. USER_PREFERENCE extracts explicit preferences ("I prefer email," "don't call before 10am") into dedicated long-term entries. SUMMARY creates condensed versions of sessions so your agent can skim a six-month relationship in a few paragraphs rather than re-reading every message.

These strategies run server-side. Your agent sends a conversation specifying infer: true, and OpenSearch Service handles the extraction, embedding, and organization. Your agent code stays focused on the conversation. You can swap embedding models or change strategies without touching agent logic.

Namespaces and isolation

Memory containers support namespaces that partition information by user, session, or agent instance. A contact center serving 100K customers uses one memory container with customer_id as the namespace. Every search is scoped to the current customer—logical isolation without the overhead of 100K separate indexes. The retrieval API combines semantic search with namespace filtering in a single call, so the agent gets back only memories that belong to the right customer and match the current query.

The retrieval loop

Here is what happens during a live conversation. The agent receives a customer message. Before responding, the agent searches long-term memory with the customer's namespace and the current query ("What did we discuss about retirement?"). OpenSearch Service returns the most semantically relevant memories. The agent includes those memories in its prompt alongside the current message. The LLM generates a response informed by real history, not a sliding window of raw text. After the session ends, the full conversation is stored back into the container with inference enabled, updating long-term memory for next time.

Memory makes the difference

The difference between an agent that remembers and one that does not is the difference between a service that builds relationships and one that processes transactions. A customer who has to re-explain their situation on every interaction feels unheard. An agent that picks up where the last conversation left off feels like a partner. OpenSearch Service provides the infrastructure that makes this possible: persistent, searchable, semantically organized memory that your agent can query in milliseconds.

The APIs are available today in OpenSearch 3.3 and later. If you are building agents that talk to the same people more than once—and that is most agents worth building—give your agent a memory that outlasts the context window. The agentic memory documentation covers the full API surface, and the contact center tutorial walks through a complete implementation.

Stop Making Your Database Pretend It Can Search

Jon Handler — Thu, 06 Aug 2026 22:47:10 +0000

Databases are optimized for durable storage: transactions, constraints, consistency, recovery. Amazon OpenSearch Service is optimized for a different job: language-aware search (matching, synonyms, typo tolerance, relevance ranking), deep vector capabilities for hybrid search that combines lexical matching with semantic similarity, and sub-second analytics at scale. Both store data reliably. But each excels at what the other merely tolerates. If your application needs both great transactional writes and great search, you run both. Amazon OpenSearch Ingestion and its zero-ETL integrations now keep them in sync automatically.

The hard part has always been the sync. Your product catalog lives in PostgreSQL. Your session data lives in DynamoDB. Your search experience lives in OpenSearch Service. A customer updates their address, a product goes out of stock, a price changes. That change needs to appear in search results within seconds. Continuously. Without a fragile pipeline that breaks when someone alters a column. OpenSearch Ingestion eliminates that pipeline entirely for Amazon Aurora (Aurora), Amazon RDS (RDS), and Amazon DynamoDB (DynamoDB), with native change data capture and near-real-time synchronization out of the box.

This post walks through why the sync problem is genuinely hard, how OpenSearch Ingestion solves it, and what to expect when you wire it up.

Why Keeping Two Systems in Sync Is Genuinely Hard

The most common first attempt is dual write: the application pushes every change to both the database and OpenSearch Service directly. This works at small scale and feels clean because there is no separate sync system. It breaks once traffic grows. At scale, you are driving thousands of simultaneous connections to OpenSearch Service from your application tier, each with its own overhead. Under load, one write succeeds and the other fails, leaving the two systems out of sync. Retries help, but now your application code is handling distributed transaction semantics that belong in infrastructure, not business logic. Dual write also cannot handle schema migrations, bulk backfills, or replaying historical changes.

The next step teams try is polling. Query the database on an interval, diff against what OpenSearch Service has, push the deltas. This decouples the sync from the write path, which is an improvement. But polling intervals are a tradeoff with no stable answer: too frequent wastes resources, too infrequent means stale results. And the rate of change in a production database is bursty. A product launch triggers thousands of updates in minutes, then nothing for an hour. No fixed interval handles both.

The next level up is change data capture (CDC). MySQL has binary logs. PostgreSQL has logical replication. Read the transaction log, extract the changes, push them to OpenSearch Service. This sounds clean until you start implementing it. You need to parse log formats that differ between database versions, handle schema changes without breaking the pipeline, manage replication slot offsets, deal with connection failures and retries, and ensure exactly-once delivery. What started as "just read the logs" becomes a distributed systems problem with its own failure modes.

Then there is the initial load problem. CDC handles ongoing changes, but what about your existing million rows? You cannot run a massive SELECT against production without locking tables or degrading performance. So you end up with two completely different sync mechanisms: one for the historical backfill, one for the stream. Each has its own failure modes, its own monitoring, its own on-call rotation.

Teams build this infrastructure. It works, mostly. Then someone adds a column and the pipeline breaks. Or a schema migration changes a type and the index mapping rejects the new documents. Or the sync develops a subtle bug that silently drops updates for days before anyone notices. The real cost is not building the pipeline. The real cost is maintaining it indefinitely while the schema underneath keeps evolving.

Amazon OpenSearch Ingestion Now Connects Directly to Your Database

Amazon OpenSearch Ingestion now has native integrations with Amazon Aurora (Aurora), Amazon RDS (RDS), and Amazon DynamoDB (DynamoDB). For Aurora and RDS, it supports MySQL (version 8+) and PostgreSQL (version 16+). For DynamoDB, it uses DynamoDB Streams with point-in-time recovery for initial snapshots. All three provide automatic change data capture and near-real-time synchronization. No Lambda functions, no Kafka clusters, no Glue jobs, no custom code.

The architecture solves both the initial load and the ongoing stream in one pipeline. When you create a pipeline, OpenSearch Ingestion starts with a full snapshot export to S3. This handles the backfill: your database is not locked, the export happens in the background, and OpenSearch Service ingests from S3 at whatever rate makes sense. No production impact.

Once the snapshot is loaded, the pipeline switches to streaming mode. For MySQL, it taps into binary logs (row format, full image). For PostgreSQL, it uses logical replication. Every insert, update, and delete in your database appears in OpenSearch Service within seconds. The pipeline handles offset management, connection recovery, and delivery guarantees. You do not write retry logic. You do not manage replication slots manually.

OpenSearch Ingestion pipelines are configuration-driven. You specify schema mappings, define data mutations (rename fields, drop columns, enrich documents in flight), and control delivery behavior. The pipeline buffers requests for up to 72 hours during downstream outages, retries failed deliveries automatically, and routes undeliverable documents to a dead-letter queue. Default schema mapping is automatic (product IDs become keyword fields, timestamps are typed correctly, text columns are analyzed for full-text search), but you have full control to override any of it through the pipeline configuration.

Our team tested this with a PostgreSQL database containing a product catalog (detailed in Integrating Amazon OpenSearch Ingestion with Amazon RDS and Amazon Aurora). Setup took about twenty minutes: enable logical replication, create a Secrets Manager entry for credentials, define the pipeline in the OpenSearch console. The initial sync happened in the background. After inserting a new record into PostgreSQL, it was searchable in OpenSearch Service within seconds.

That three-second latency is the part that matters. Not the setup. Not the configuration. The fact that changes flow continuously without anyone thinking about it. The pipeline is not a batch job you schedule. It is a live connection that stays current as your database evolves.

Constraints Worth Knowing

The database and your OpenSearch Service domain must be in the same AWS account and region. You can sync one database per pipeline. Multi-AZ database clusters are not supported yet. These are planning considerations, not dealbreakers for most search and analytics use cases. If you run a multi-region architecture, you will need one pipeline per region.

Setup is prerequisite-focused rather than code-focused. For MySQL, enable binary logging with row format and full image. For PostgreSQL, enable logical replication (available in version 16+). For DynamoDB, enable DynamoDB Streams (new and old images) and point-in-time recovery (PITR) for the initial snapshot. Store your database credentials in AWS Secrets Manager. Then define your pipeline using the visual builder in the OpenSearch console or a YAML configuration. You specify which tables to sync and where in OpenSearch Service the data should land. The pipeline handles everything else.

What This Changes

This integration does not solve every data synchronization problem. It solves one specific, common, and expensive one: making your database data searchable and analyzable in real time without building custom infrastructure. If you have a product catalog in RDS and need full-text search with relevance ranking, this is now a configuration task. If you have transactional data in Aurora and need real-time analytics dashboards, you no longer need a team maintaining Kafka and custom consumers. If you have session or user-profile data in DynamoDB and need it searchable alongside your relational data, the same pipeline model applies.

The s/ETL/configuration/ substitution here is real. Teams that were spending months on sync infrastructure can now spend that time on search relevance, query tuning, and the features their users actually see. The plumbing disappears. The search quality work can begin.

If you are currently running custom sync infrastructure between RDS, Aurora, or DynamoDB and OpenSearch Service, or if you have been postponing search capabilities because the integration cost was not worth the effort, the calculation just changed. Enable replication or streams on your database, point OpenSearch Ingestion at it, and your data flows. Continuously. In seconds. Without you maintaining anything in between.

Your Search Backend Speaks MCP Now

Jon Handler — Tue, 04 Aug 2026 01:54:30 +0000

Every MCP-compatible agent in the world needs the same three things from a search backend: discover what data is available, run queries against it, and get structured results back. Claude, Amazon Q, Cursor, Kiro, Strands Agents, and a growing list of open-source frameworks all speak MCP natively now. The protocol side is settled. Your search infrastructure can now speak it back.

Amazon OpenSearch Service now does. Your domain exposes a native MCP endpoint, and agents connect directly. No custom connectors, no middleware, no per-agent integration code. This post walks through what that looks like in practice: what the endpoint exposes, how to secure it, and why the M×N integration problem disappears when your data source speaks the same protocol your agents already understand.

MCP in 60 Seconds

The Model Context Protocol (MCP) is a standard JSON-RPC interface that lets AI agents discover and call tools on external systems. An MCP server advertises what it can do (search an index, check cluster health, run an aggregation), and any MCP-compatible agent can call those tools without custom integration code. One protocol replaces all the bespoke connectors.

The problem MCP solves is combinatorial. If you have M agents connecting to N data sources, custom integrations mean M×N connectors to build and maintain. Three agents talking to five OpenSearch Service domains means fifteen connectors. Each one handles authentication, query formatting, and response parsing in its own way. Add a sixth domain or a fourth agent, and the cycle starts over. MCP collapses that to M+N: each agent speaks one protocol, each data source exposes one server, and any agent can talk to any server without additional code.

Think of it like USB. Before USB, every peripheral needed its own cable and driver. After USB, you plug in and it works. MCP is that standardization applied to the AI integration layer. Your agents are the peripherals. Your data sources are the computer. MCP is the port.

What the MCP Server Exposes

Your OpenSearch Service domain exposes the MCP endpoint at /_plugins/_ml/mcp as part of the ML Commons plugin. Agents connect directly. The endpoint advertises three types of components. Resources provide data context from your indexes. Prompts are reusable instruction templates for recurring analyses. Tools are executable functions: searching indexes, checking cluster health, analyzing performance metrics, running aggregations.

Here is what a tool call looks like. A Python agent connecting with fastmcp:

from fastmcp import Client

async with Client("https://your-domain.us-east-1.es.amazonaws.com/_plugins/_ml/mcp") as client:
    # discover what the domain exposes
    tools = await client.list_tools()
    print([t.name for t in tools])  # ['SearchIndexTool', 'ListIndexTool', 'ClusterHealthTool', ...]

    # call a tool by name
    result = await client.call_tool("SearchIndexTool", {
        "index": "products",
        "query": {"match": {"category": "electronics"}}
    })

The agent discovers available tools, calls them by name, and gets structured results back. No custom SDK, no REST client boilerplate. Any MCP-compatible agent (Amazon Q CLI, Claude, Cursor, Strands Agents) connects the same way.

The Setup (Where It Gets Fun)

For the built-in endpoint, there is nothing to configure on the server side. Your domain already exposes the MCP endpoint. What matters is getting authentication right: IAM roles and backend role mapping determine what each agent can see. Once those are in place, connecting a new agent is a configuration change. I built a domain, enabled MCP, registered tools, and used the SearchIndexTool to query my data on OpenSearch Service. The agent discovered available tools through the protocol's capability negotiation and ran queries without any custom integration code.

Access requires two layers. First, an IAM resource-based policy that lets the agent role reach the domain:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {
      "AWS": "arn:aws:iam::123456789012:role/ai-agent-role"
    },
    "Action": ["es:ESHttpGet", "es:ESHttpPost"],
    "Resource": "arn:aws:es:us-east-1:123456789012:domain/my-domain/*"
  }]
}

Second, with fine-grained access control enabled, map the IAM role to an OpenSearch backend role that has permissions on the ML Commons APIs and the indexes your agent needs to search. In OpenSearch Dashboards, go to Security > Roles, create or choose a role with cluster permissions for ml_full_access (or a narrower custom permission set), add index permissions for the target indexes, then map your agent IAM role ARN to that backend role under Mapped users. Every agent that assumes the same IAM role inherits the same access. One security boundary instead of one per connector.

Scaling Without Rebuilding

The built-in MCP endpoint scales with your OpenSearch Service domain. If your domain handles the query load, the MCP endpoint handles it too. No separate scaling layer to worry about. For teams using the AgentCore-hosted path, AgentCore handles auto-scaling independently.

Adding a new AI agent takes nothing on the server side. The agent connects to your domain’s MCP endpoint and discovers available tools automatically through the protocol’s built-in capability negotiation. This is the M+N property in practice: each new agent is O(1) work, not O(N).

The M+N Payoff

Consider a team with four AI agents connecting to three OpenSearch Service domains. Under the old model, that is twelve custom integrations. With MCP, each agent points at the domain’s endpoint and discovers tools automatically. Adding agent number five is a configuration entry, not a development sprint. Adding a fourth domain means the existing agents can reach it immediately. The complexity stays linear.

The open-source OpenSearch MCP server is part of the OpenSearch project. Community-driven improvements and security updates mean you are not maintaining proprietary integration code. And because the built-in MCP endpoint on your OpenSearch Service domain uses the same protocol, agents that work with one path work with the other—no code changes required.

If your agents already speak MCP, your OpenSearch Service domain is ready to answer. Enable the ML Commons plugin (set plugins.ml_commons.mcp_server_enabled to true), register the tools you want agents to access, configure IAM and backend role mapping, and point your agent at the /_plugins/_ml/mcp endpoint. The second agent costs nothing. The tenth agent costs nothing. The protocol does what protocols are supposed to do: make the next connection free.

"Most Of Your Vectors Are Steerage. Why Are They In First Class?"

Jon Handler — Sat, 01 Aug 2026 00:39:24 +0000

I was on a call last month with a startup CTO who had just gotten their AWS bill. They had built a beautiful RAG application: semantic search, conversational AI, the works. Their vector index was humming along with about 50 million embeddings. Then they hit product-market fit.

Within six weeks, they scaled to 500 million vectors. Their monthly infrastructure costs went from $2,000 to $20,000. The real kicker? When we looked at the access patterns, over 80% of those vectors were queried less than once a week. They were paying hot-storage prices for data that was, by any honest measure, cold.

The standard advice here is "just use a cheaper vector database." The more interesting question is: why are you storing all your vectors at the same temperature in the first place?

The Cost-Recall-Latency Triangle

Vector search forces a three-way tradeoff. You can optimize for cost, recall, and latency, but you only get to pick two. Want high recall and low latency? That costs money (in-memory HNSW graphs with full-precision vectors eating RAM). Want high recall at low cost? Latency goes up. Want cheap and fast? Recall suffers.

Most teams pick a single point on this triangle and apply it uniformly to every vector in their index. That decision made sense when vector databases offered a single storage tier. It makes the same amount of sense as storing your entire filesystem on NVMe SSDs because some files need fast access.

The conventional wisdom says you pick your point on the triangle and live with it. But the conventional wisdom was written before vector storage got interesting. The better approach: tier your vectors the same way you already tier your storage. Different access patterns deserve different economics. The same embedding that costs $0.12/month in RAM might cost $0.004/month on disk and $0.0002/month in object storage. When you have 500 million of them, those decimals matter.

The Hot Tier: In-Memory HNSW and Exact k-NN

For vectors that get hit constantly (your user-facing search, your real-time recommendations, anything in the critical path of a page load), Amazon OpenSearch Service stores HNSW graphs entirely in native memory. On a single r6g.8xlarge node with 113 million vectors at 1,024 dimensions (source), this configuration delivered P90 latency of 25 ms and recall of 0.95 at 300 queries per second. Throughput scales linearly with replica shards.

There is also a case the HNSW literature tends to gloss over: exact k-NN. Exact k-NN compares every candidate by brute force rather than navigating a graph. When your query includes filters (a tenant ID, a category, a date range) that reduce the candidate set below roughly 100,000 vectors, exact k-NN outperforms approximate search. The brute-force scan finishes faster than an HNSW traversal at that scale, uses less RAM (no graph to build or maintain, no ef_construction tuning, no ef_search parameter), and returns perfect recall.

The hot tier is the right home for vectors with high query frequency, latency requirements under 50 ms, or both.

Quantization methods (FP16, binary quantization, product quantization) give you further control over the RAM-versus-recall tradeoff at each tier. That topic deserves its own article.

The Warm Tier: On-Disk Mode (Where Most of Your Vectors Should Live)

Here is the part of this story that deserves more attention than it gets.

OpenSearch Service's on-disk mode keeps a quantized navigation graph in RAM and stores the full-precision vectors on SSD. The default is binary quantization at 32× compression (each float dimension collapses to a single bit), but you can choose 2×, 4×, 8×, or 16× to trade more RAM for higher recall. When a query arrives, OpenSearch walks the compressed graph in memory, identifies candidates, then rescores against the full vectors on disk.

On the same 113M-vector benchmark (source), on-disk mode at 8× compression delivered P90 latency of 96 ms with 0.98 recall. At 32×, latency was 104 ms with 0.94 recall. Compare that to the in-memory tier's 25 ms. You trade 70-80 ms of latency for a memory reduction approaching two thirds: 1 million vectors at 256 dimensions drop from 1.31 GB (FP32) to 0.18 GB for the navigation graph, plus the derived source optimization saves up to two-thirds more by deduplicating vector storage across replicas.

Most workloads belong here. In a RAG pipeline where LLM generation takes 2-3 seconds, 100 ms for vector retrieval is invisible. The only workloads that need 25 ms retrieval are the ones in the critical path of a user-facing page load with tight SLAs.

To enable on-disk mode,

The Cold Tier: S3 Vectors for Archive Scale

For vectors that are accessed rarely, Amazon S3 Vectors provides native vector storage and search at S3 economics. You specify "engine": "s3_vector" in your index mapping on OpenSearch 2.19+ with O-series instances, and OpenSearch manages the rest. OpenSearch offloads vector data to S3, keeps metadata on the cluster for filtering, and routes the k-NN portion of each query to S3 transparently. From the application's perspective, you query the same _search API you use for in-memory and on-disk indexes.

Response times are sub-second (500-800 ms typical), and storage costs drop by up to 70% compared to in-memory indexes. S3 Vectors uses pay-per-query pricing, so you never pay for idle capacity. That model is ideal for large datasets with low to moderate query traffic (up to thousands of queries per day). For workloads with sustained high throughput (hundreds of queries per second), on-disk mode is the better complement because its costs are capacity-based. The two tiers work together. S3 Vectors handles large, infrequently accessed data, and on-disk mode handles data with steady query traffic.

S3 Vectors also works as a standalone vector store outside the OpenSearch integration. For teams using S3 Vectors independently, a one-click export moves data into OpenSearch Serverless, where queries run at sub-200ms latency. This lets teams take advantage of OpenSearch Serverless throughput when access patterns shift and cold data needs to become hot.

The Decision Framework

Where each vector lives should follow from two questions: how often does the vector get queried, and what latency does the caller need?

Vectors that get hit constantly and need sub-30ms responses belong in memory with in-memory HNSW on OpenSearch Service (FP16 recommended). Exact k-NN is worth testing when query filters narrow candidates below 100,000, which is common in multi-tenant applications. Vectors with steady query traffic where 100ms latency is acceptable belong on disk with 32× compression. Think internal search tools, batch RAG pipelines, analytics, product catalogs. On-disk mode is the default tier for most production workloads at scale. Vectors that are large in volume, accessed at low to moderate frequency, and tolerant of 500-800ms latency belong in S3 Vectors with the "engine": "s3_vector" integration: historical archives, compliance retention, seasonal catalogs outside their season. If access patterns change, a one-click export promotes cold data to OpenSearch Serverless.

The beauty of this tiering is that the calling application does not need to know about it. Every tier exposes the same OpenSearch _search API. The application sends a query. OpenSearch routes the query to the right storage layer based on which index you target. Migrations between tiers are operational decisions, not application code changes.

What Happened to the $20,000 Bill

The CTO from my opening story did the access-pattern audit. About 15% of their 500 million vectors were hot: they powered the real-time search and needed in-memory speed. Around 60% were warm: knowledge-base vectors for their RAG pipeline where 100 ms was invisible inside a 3-second LLM call. The remaining 25% were seasonal product embeddings from marketing campaigns that had ended months ago.

They moved the warm vectors to on-disk mode and the seasonal vectors to S3 Vectors. Their monthly bill dropped by roughly two-thirds without changing a single query from the application layer. The application still called the same _search API. The tiering was invisible to the calling code.

If you are running a vector workload today, the one thing worth doing this week is pulling up your access-pattern metrics. Look at which embeddings get queried at high frequency versus which ones are sitting idle. The answer tells you exactly which tier each vector belongs in. The math tends to be obvious once you look, and the savings tend to surprise people who assumed all their vectors needed to be hot.

What does your vector access-pattern distribution look like? I am curious whether others are seeing the same 80/20 split between rarely-touched and actively-queried embeddings.

Fix Your Search, Fix Your RAG Output

Jon Handler — Mon, 27 Jul 2026 20:23:49 +0000

I watched a customer demo their new RAG application last month. They had spent three months building it. The interface was beautiful. The LLM responses were eloquent and well-formatted. And almost every answer was subtly, confidently wrong.

“The model keeps hallucinating,” they told me, frustrated. They had tried three different LLMs. They had adjusted temperature settings. They had rewritten their prompts a dozen times. But here is what nobody had looked at yet: the search layer feeding context into those prompts.

This is the part of RAG that deserves more attention than it gets: most quality problems are retrieval problems. When your generative AI makes things up, it is usually because the context it received was off. Too much, too little, or just plain irrelevant. The LLM is doing exactly what you asked: generating fluent text based on the information you provided. If that information misses the mark, well, garbage in, eloquent garbage out.

The challenge is that retrieval tends to get treated as a solved problem. Chunk the documents, throw everything into a vector database, retrieve the top 10 results, done. That works for demos. In production, though, you need the system to distinguish between a precise question about part number XJ-447 and an abstract question about how plumbing works. That distinction is where things get genuinely interesting.

The Part of Chunking That Trips People Up

Here is something I find fascinating about document chunking: it looks like a technical problem, but it is actually a semantic one.

The conventional wisdom goes something like this: take your documents, split them into 512-token chunks with 50-token overlap, generate embeddings, done. Clean. Systematic. And it often destroys retrieval quality in ways that are genuinely hard to diagnose.

Think about what you are actually doing when you chunk a document. You are deciding what the atomic unit of meaning is for your system. That 512-token window might slice right through the middle of a critical explanation. It might mash together two unrelated concepts that happened to share a page. Your chunks are not just storage units. They are the answers your system will retrieve.

I have seen teams spend weeks optimizing their embedding models while using paragraph-based chunking that splits “The solution to this problem is…” from the actual solution in the next paragraph. Once you see it, the fix is obvious. But it is easy to miss because chunking happens early in the pipeline and its effects show up late, as vague or wrong answers from the LLM.

The approach that works: chunk semantically, not mechanically. If you are working with structured documents, and most enterprise content is structured, respect that structure. A section about pricing belongs together. A troubleshooting procedure should stay intact. A paragraph explaining a concept should not be split because it crossed some arbitrary token threshold.

In my own work, I use a tiered strategy: first split at structural markers (headings, section breaks), then embed sliding sentence windows and split at valleys in cosine similarity between adjacent windows. Those valleys are the points where the topic naturally shifts. This technique (well described visually here) is now built into LangChain and LlamaIndex as their default semantic chunkers. The embedding-based valley detection catches topic boundaries that structural markers miss, without the cost of sending every chunk through an LLM.

Here is the related subtlety: not every chunk needs to be retrieved independently. Sometimes you want to retrieve a full document, like when someone needs that PDF about 1957 nail prices. In those cases, use chunks as subdocuments with a parent-child relationship. The chunks help you find the right document, but you return the whole thing. Other times, especially in RAG scenarios, you want the chunk itself. You are not looking for a document about plumbing. You want the specific paragraph that explains how a P-trap works.

Most teams pick one approach and apply it everywhere, which explains why their system is great at some queries and puzzling at others.

The Precision Problem That Makes Things Interesting

Here is something about vector search that is genuinely counterintuitive once you see it: vectors never return zero results.

That is not a bug, it is math. In vector space, everything has a nearest neighbor. Ask a question about quantum physics to a database of cooking recipes, and it will cheerfully return the five “most relevant” results. They will be completely wrong, but the distance metrics will look reasonable.

This is especially tricky in RAG systems because the LLM will work with whatever you give it. Feed it those irrelevant cooking recipes as context for a quantum physics question, and it will generate a confident-sounding answer that blends both domains into nonsense. The user has no signal that the underlying search missed.

Lexical search does not have this problem. If your keywords do not match, you get nothing back. That absence is actually useful information. When someone asks about part number XJ-447, you want exact matching. You want lexical search. When they ask about “solutions for preventing pipe corrosion in coastal environments,” you want semantic search to correlate terms by the company they keep.

The right approach is not choosing between lexical and semantic search. It is knowing when to use each, and increasingly, using both together. Hybrid search strategies that combine keyword matching with semantic correlation consistently outperform either approach alone.

But you need something to orchestrate that decision. Is this a precise query or an abstract one? Does it contain specific identifiers that demand exact matching? Are there domain-specific terms that semantic search might miss? Most teams try to solve this with increasingly complex query logic. The more interesting move is letting an agent figure it out.

Building Search That Actually Works for RAG

So what does a properly architected RAG search layer look like?

Start with the retrieval target. Before you chunk anything, ask: what am I trying to retrieve? If you are building a document discovery system, your target is documents. Chunk for findability, but return the full document. If you are building a question-answering system, your target is the specific piece of information that answers the question. Chunk for semantic coherence, and return just that chunk.

For RAG specifically, smaller and more semantically coherent beats larger and more comprehensive. You are not trying to give the LLM everything that might be relevant. You are trying to give it exactly what is relevant. A tightly focused paragraph that directly addresses the question will outperform a kitchen-sink approach that includes three pages of tangentially related content.

This is where reranking enters the picture, and where it helps to be precise about what reranking actually does. Reranking is not a magic quality boost you toggle on. It is a precision tool. If your initial retrieval is pulling back mostly irrelevant results, reranking just reorders them. The conventional wisdom says to over-sample: retrieve 50 results, rerank them, take the top 10. Think about what that means. You are deliberately retrieving low-relevance results (because that is what results 11 through 50 usually are) and hoping the reranker will find hidden gems.

Sometimes that works. More often, you are adding latency and cost while marginally improving already-poor results. The better approach: write queries that retrieve high-precision results in the first place, then use reranking to fine-tune the order. If you need to retrieve 50 results to get 10 good ones, the query strategy is where the leverage is.

This is also where agentic frameworks start to make sense. Instead of building the perfect query upfront, let an agent run multiple query strategies, evaluate the results, and decide what to pass forward. Amazon OpenSearch Service has internal agentic capabilities that can orchestrate this: running lexical and semantic searches in parallel, applying different ranking strategies, and using an LLM to judge which results actually answer the question.

That last point is becoming the new standard for search quality evaluation. Traditionally, measuring precision and recall required humans to manually review results. That does not scale. But an LLM can evaluate whether retrieved chunks contain information relevant to the query, and it can do it in real time. This lets you build systems that are self-critical, that recognize when search results are poor and try a different approach rather than confidently generating wrong answers.

What This Means Going Forward

The RAG systems that work in production are not the ones with the fanciest LLMs or the most sophisticated prompts. They are the ones that treat retrieval as a first-class problem worthy of the same engineering attention as the generative layer.

We are moving away from the “embed everything, retrieve top-k, generate answer” pipeline toward systems that reason about queries, orchestrate multiple search strategies, and critically evaluate their own results before generating responses. The search layer is becoming intelligent, not just functional.

If you are building or fixing a RAG system this week, try starting not with the LLM but with a simple question: when I retrieve context for this query, am I getting back the specific information that actually answers it? Pull up your logs, look at what is being retrieved, and check whether a human could answer the question from that context alone.

Because if a human cannot, the LLM will not either. It will just be more eloquent about being wrong. And that is a problem worth the s/prompt engineering/actual engineering/.