Tight integration brings the embedding model to the search engine. Vectors happen automagically.
If you have been considering semantic search, you already know the list of unknowns. Which embedding model do you choose? How do you host it? How do you get your documents converted to vectors and keep those vectors in sync as your data changes? How do you handle vectorization at query time without adding latency? And once you have vectors, how do you wire everything together so that your search index, your embedding model, and your ingest pipeline all talk to each other without a custom integration layer in between?
Those questions can take weeks of work to resolve, and for a lot of organizations they represent a huge hill to climb. Selecting and deploying an embedding model on SageMaker. Writing batch processing jobs to convert content into vectors. Building orchestration code to keep the ML pipeline and the search index in sync. Managing two separate systems that both need to know when your content or your model changes. The cognitive overhead alone can be a barrier, before you write a single search query.
The neural plugin in Amazon OpenSearch Service makes this dramatically easier. Through the OpenSearch Service console, you set up an ML connector that links your domain to an embedding model hosted on SageMaker or Amazon Bedrock. An ingest pipeline with a text_embedding processor then calls that model automatically every time you index a document, and queries convert to vectors at search time through the same connector. The embedding model still runs on SageMaker or Bedrock, but the integration, the vectorization pipeline, and the query-time conversion all live inside OpenSearch Service, so you are managing one workflow instead of stitching together three. If you want an even lighter path, Automatic Semantic Enrichment (which I wrote about in an earlier post) adds sparse vector enrichment at index time without deploying any model at all.
I set this up recently for a product catalog, and the whole process from zero to working semantic search took about fifteen minutes.
What the Old Playbook Looked Like
Without the neural plugin, implementing semantic search on OpenSearch Service means treating the search engine as a passive vector store. You deploy embedding models separately, build SageMaker processing jobs to convert your content into vectors, write orchestration code to call those endpoints, and manage the batch ingestion pipeline that feeds vectors into your index. When your content updates, both systems need to know. When your embedding model changes, you reprocess everything.
The architecture worked, but the operational tax was real. I spent more time debugging the plumbing between the embedding service and the search index than I spent on search relevance. The integration layer became its own project, and keeping the ML pipeline and the search index in sync consumed engineering cycles that had nothing to do with the actual search experience.
The Neural Plugin Changes the Contract
The neural plugin integrates embedding models directly into the search workflow. A pre-built CloudFormation template (available in the Integrations tab of the Amazon OpenSearch Service console) deploys the embedding model (Amazon Titan Text Embeddings through Amazon Bedrock, or a model on Amazon SageMaker), creates the IAM roles, and configures the ML connector in a single stack. The model runs on SageMaker or Bedrock, and the CloudFormation stack outputs an internal OpenSearch model ID that you use in subsequent steps. OpenSearch Service calls the model automatically at ingest and query time through the connector. With the model deployed, OpenSearch Dashboards provides a visual workflow experience for building the end-to-end AI search flow through the AI Search Flows interface (under OpenSearch Plugins). The same setup is available via the ML Commons API for automation.
When you create an OpenSearch ingest pipeline with a text_embedding processor, the neural plugin automatically calls your embedding model as documents enter the index. You specify which text fields should generate embeddings (product descriptions, document content, whatever drives your search experience), and the pipeline handles vectorization at ingest time. The entire workflow runs inside OpenSearch Service, so you are not maintaining batch jobs, custom code, or a sync layer between two systems.
At query time, the same model converts the user's search text into a vector, and OpenSearch Service runs a k-NN search against the stored embeddings. The entire semantic search workflow (text to vector to search to results) happens inside the search engine. You interact with plain text on both sides: index text documents, search with text queries. The vectors are internal.
What Semantic Search Actually Does to Your Results
I tested this against a product catalog. A keyword search for "what to wear in office" returned random clothing items: t-shirts, boots, undergarments. The query matched on individual words without any sense of the relationship between them. A semantic search for the same query returned dress suits, professional shoes, ties, and business attire. The embedding model had learned from training data that these terms co-occur in professional-clothing contexts, and the vector similarity surfaced results that lexical matching missed entirely.
For "accessory for hike," keyword search returned nothing useful. Semantic search returned backpacks, water bottles, and outdoor gear. The vocabulary mismatch between the query and the catalog vocabulary was total, and semantic search bridged the gap without a single synonym mapping.
The Search Relevance Workbench in OpenSearch Dashboards lets you run keyword and neural queries side-by-side against the same index. When stakeholders ask whether semantic search justifies the investment, the results comparison makes the case directly. The difference between matched-on-keywords and matched-on-co-occurrence-patterns is visible in the first three results. OpenSearch Dashboards also includes a visual builder for these AI-powered search flows, so you can configure and test ingest pipelines, ML connectors, and search pipelines through a graphical interface rather than writing JSON by hand.
Getting Started in 15 Minutes
Before you start, two prerequisites need to be in place. You need model access enabled in Amazon Bedrock for your chosen embedding model, and you need to add the Lambda invoke role as a backend role in your OpenSearch ML Commons configuration. Budget 10 minutes for this setup before you touch CloudFormation. With the prerequisites in place, deploy the CloudFormation template that provisions the embedding model, the ML connector, and the IAM roles. The stack outputs an internal OpenSearch model ID that you use for the rest of the setup. Create an ingest pipeline with the text_embedding processor, create an index that uses the pipeline, and start indexing documents.
The operational burden of adding ML-powered capabilities to search has dropped by an order of magnitude. The neural plugin, the CloudFormation templates, and the managed ML connectors are not doing anything fundamentally new. They are collapsing the integration layer that used to sit between your search engine and your embedding model. The search engine now handles both sides of the conversation.
If you have been putting off semantic search because the ML pipeline seemed too heavy, that calculation has changed. Deploy the CloudFormation stack, create an ingest pipeline, index a few hundred documents, and run the comparison tool. The difference in search quality is measurable in minutes, and the setup takes a fraction of the time the old approach required.
Top comments (0)