Social post Disclaimer: This post is submitted as part of
The Elastic Blogathon.
Blog Disclaimer: “This blog post was submitted to the Elastic Blogathon Contest and is eligible to win a prize."
Author Intro
My name is Prosenjeet Saha. I work as a Full Stack and AI Solutions Architect, and a large part of my day-to-day work involves fixing search systems that look good in demos but break down under real operational pressure. This blog is based on patterns I have repeatedly seen while designing and reviewing enterprise search and AI platforms, particularly in environments where scale, observability, and governance are non-negotiable.
Abstract
Over the past few years, I have watched enterprise search move from keyword matching to embeddings and RAG pipelines. What has become clear is that retrieval alone is not enough. In this article, I describe how Elastic’s Agent Builder can be used to build agentic search systems that reason over data iteratively, using Elasticsearch not just for storage, but as a core part of the decision-making loop.
**Why I Started Looking Beyond Traditional Search
The first time I ran into the limitations of traditional enterprise search was during a production incident review. We had hundreds of documents—runbooks, postmortems, and architecture notes—but no reliable way to connect them under time pressure. Engineers were searching, skimming, and guessing.
We later introduced semantic search and, eventually, a RAG-based assistant. It helped, but only to a point. The system still behaved like a one-shot tool: retrieve once, generate once, and stop. When the answer was incomplete, the user had to manually rephrase the question.
That experience pushed me to explore agentic search—systems that can
pause, reconsider, search again, and adapt their strategy. Elastic’s Agent Builder stood out because it allowed this behavior while keeping the system observable and controllable.
*What Elastic’s Agent Builder Actually Solves
*
From a practical engineering perspective, Elastic’s Agent Builder addresses a very specific problem: how do you let an AI agent interact with search infrastructure without turning it into an unpredictable black box?
Instead of letting a language model issue arbitrary queries, Agent Builder enforces structure:
The agent can only use approved tools
Every search call is logged and observable
Memory is persisted in Elasticsearch indices
Retrieval can mix keyword relevance and vector similarity
This matters in real systems. When something goes wrong, you need to know why an agent made a decision—not just what answer it produced.
Architecture: How Agentic Search Fits Together
In systems I have worked on, agentic search usually settles into four layers:
- A user-facing interface (chat, API, or internal tool)
- A reasoning layer responsible for planning
- An orchestration layer that controls tools and memory
- A search and storage layer that remains stable and scalable
Elastic fits naturally into layers three and four. Elasticsearch handles retrieval and memory, while Agent Builder controls how and when those capabilities are used. This separation makes the system easier to reason about and significantly easier to debug.
Demo Scenario: Internal Engineering Knowledge Assistant
Why This Use Case Is Realistic
I am using an internal engineering knowledge assistant as the demo because I have seen this exact problem across multiple organizations. Documentation exists, but it is fragmented. During incidents or design discussions, engineers need synthesized guidance, not raw search results.
Expected Agent Behavior
In this setup, the agent is expected to:
Interpret loosely phrased technical questions
Search across different document types
Detect when results are incomplete
Perform follow-up searches automatically
Produce a structured response that reflects engineering best practices
Hands-On Implementation Details
Elasticsearch Setup
The system runs on Elastic Cloud. This choice is deliberate—managed
infrastructure reduces operational overhead and ensures consistent
performance during peak usage. Index Design
Each document is stored with both raw text and an embedding vector. Hybrid retrieval turned out to be essential in practice, especially when dealing with acronyms, product names, and domain-specific language.
{
"mappings": {
"properties": {
"title": { "type": "text" },
"content": { "type": "text" },
"embedding": {
"type": "dense_vector",
"dims": 1024,
"index": true,
"similarity": "cosine"
}
}
}
}
Agent Configuration
The agent is intentionally constrained. It has one primary search tool and a dedicated memory index. This keeps behavior predictable.
agent = ElasticAgent(
name="EnterpriseKnowledgeAgent",
tools=["elasticsearch_search"],
memory_index="agent_memory",
reasoning_model="gpt-4.1"
)
In testing, overly permissive agents produced impressive demos but failed consistency checks. Constraints mattered more than raw intelligence.
Example Interaction from Testing
Question: “How should we investigate intermittent latency in Kubernetes services?”
Observed Behavior: The agent performed an initial broad search, identified gaps around networking and autoscaling, and then issued follow-up searches focused on prior incidents. Only after correlating multiple sources did it produce a checklist-style response.
This kind of behavior is difficult to achieve with traditional RAG pipelines.
*Why Elastic Works Well for Agentic Systems
*
In my experience, Elastic’s biggest advantage is not any single feature, but the fact that search, storage, and observability live in the same ecosystem. Agent behavior can be inspected, traced, and tuned using tools teams already trust.
This reduces friction when moving from experimentation to production.
*Final Thoughts
*
Agentic search is not a silver bullet. It introduces complexity and requires careful constraints. That said, when the problem demands exploration rather than retrieval, agentic systems provide a clear advantage.
Elastic’s Agent Builder makes this approach feasible in real-world environments by combining control, scalability, and transparency. Based on my experience, this balance is what ultimately determines whether an AI system survives beyond the demo stage.
GitHub Reference Structure
agentic-search-elastic/
├─ ingestion/
├─ agent/
├─ config/
├─ notebooks/
└─ README.md
Github: https://github.com/Prosenjeet33/agentic-search-elastic
Top comments (0)