Foundry IQ Explained: How Does It Relate to Azure AI Search and Traditional RAG?

#microsoftfoundry #foundryiq #azureaisearch #rag

When people first encounter Foundry IQ, many assume that it is simply a new name for Azure AI Search—or traditional RAG wrapped in a new user interface.

But once you start configuring it in a real project, a few questions quickly emerge:

What exactly is the relationship between Foundry IQ and Azure AI Search?
Why does retrieval need an LLM?
Is Foundry IQ genuinely different from traditional RAG, or is it simply new product packaging?

Rather than listing product features, this article focuses on these common points of confusion. The goal is to help you understand which problems Foundry IQ is designed to solve—and which ones it is not.

First, a Naming Clarification: Are the Two Foundry IQ Entry Points the Same?

They provide access to the same underlying Foundry IQ knowledge base capabilities, but through different management experiences.

Foundry IQ (Azure AI Search) in the Azure portal focuses on lower-level resource management, including Search services, indexes, indexers, knowledge sources, knowledge bases, and agentic retrieval.
Knowledge (Foundry IQ) within a Microsoft Foundry project focuses on the agent development experience, allowing developers to create, configure, and connect knowledge bases.

In both cases, the knowledge bases and knowledge sources are hosted in the selected Azure AI Search resource. If the Search endpoint—for example, https://my-search-service.search.windows.net—and the knowledge base name are the same, you are looking at the same underlying knowledge base.

Some product and pricing pages use the name Foundry IQ (Azure AI Search). However, this does not mean that every Azure AI Search technical term has been replaced. At the time of writing, official documentation still describes Azure AI Search as the underlying indexing and retrieval infrastructure for Foundry IQ. Terms such as Index, Indexer, Skillset, Vector Search, and Search API also remain part of the Azure AI Search product vocabulary.

Microsoft Foundry
  └─ Knowledge (Foundry IQ)
       └─ Project Connection
            └─ Azure AI Search Resource
            ├─ Knowledge Base / Knowledge Sources
            ├─ Index / Indexer / Vector Search
            └─ Agentic Retrieval

In one sentence: Azure AI Search performs the search, while Foundry IQ organizes search into a reusable knowledge service for agents.

They are not competing products. They are two layers with different responsibilities in the same capability stack.

Why Does a Knowledge Base Need an LLM?

The Chat Completions model shown in the screenshot is part of the knowledge base retrieval pipeline. It is not the embedding model, and it is not necessarily the agent's primary model.

According to the official documentation available at the time of writing, the LLMs supported for query planning are primarily Azure OpenAI deployments from the GPT-4o, GPT-4.1, and GPT-5 families. This support may change over time.

The model has three main responsibilities:

Query planning: It uses the user's question and conversation history to break a complex request into more focused subqueries.
Query and knowledge-source planning: It determines which sources to query and generates the subqueries. Azure AI Search then performs keyword, vector, or hybrid retrieval, followed by semantic reranking.
Optional answer synthesis: When Output Mode is set to Answer synthesis, the model combines the evidence into an answer with citations. With Extracted data, the retrieved content and its sources are passed to the agent, which generates the final response.

Retrieval reasoning effort controls how extensively the LLM participates in retrieval. It also determines the applicable resource limits:

Mode	Core behavior	Limits from the official FAQ	Recommendation
`Minimal`	Searches directly without LLM-based query planning	Up to 10 knowledge sources; no LLM and no answer synthesis	Lowest cost and latency; supports extracted results only and does not support web knowledge sources
`Low`	Performs one round of query planning and source selection; the default mode	Up to 3 sources and 3 subqueries; 5,000-token answer-synthesis budget	A good starting point for most scenarios
`Medium`	Can revise the query and retrieve once more if the first results are insufficient	Up to 5 sources and 5 subqueries; 10,000-token answer-synthesis budget	Better suited to complex, cross-source tasks where completeness matters

Higher reasoning effort generally improves retrieval coverage, but it also increases latency and token cost. Always check the latest official limits and FAQ for current values.

Remote knowledge sources such as the web require Answer synthesis to return results. As a result, Minimal mode—which does not support answer synthesis—cannot use web sources.

It is also important to distinguish between two model roles: the knowledge base LLM optimizes how evidence is found, while the agent LLM decides how to complete the task. They may use the same model deployment, but they do not have to.

For a Foundry agent that must call other tools and assemble a unified response, Microsoft recommends starting with Extracted data. This gives the agent the raw evidence and citations, allowing it to reason over them without having both the knowledge base and the agent generate an answer. Answer synthesis is more appropriate when retrieval results are presented directly to the end user and do not require further processing by an agent.

Put simply, the LLM here is not primarily responsible for answering the question. Its job is to ask the question correctly and find the right evidence.

Why Does a Blob Knowledge Source Also Require Models?

The short answer is this: models configured on the Blob page transform raw documents into more searchable content during ingestion, while the model configured on the Knowledge Base page interprets questions and orchestrates retrieval at query time.

Both pages include a Chat Completions model, but the models operate at different stages of the RAG pipeline:

Model	When it runs	What it does
Blob Chat Model	During document ingestion, indexing, or refresh	Performs generative content extraction, particularly by describing visual content such as images and charts as searchable text
Blob Embedding Model	During indexing and vector queries	Converts document chunks and user queries into vectors for semantic similarity search
Knowledge Base Chat Model	During each user retrieval request	Interprets the question, creates subqueries, selects knowledge sources, and optionally synthesizes the answer

When you create a Blob knowledge source, the system automatically generates a data source, indexer, skillset, and search index. The component names matter less than the processing logic behind them: documents are parsed and chunked, enriched with searchable text, and then converted into vectors for retrieval.

In the screenshot, Content extraction mode = Standard invokes Azure Content Understanding for more advanced document parsing and chunking. If image descriptions are enabled, the Blob Chat Model converts visual information from images and charts into text. The Embedding Model then converts both content chunks and queries into vectors for subsequent vector retrieval.

How Is This Different from Connecting an Agent Directly to Azure AI Search?

In a traditional RAG architecture, an agent typically connects directly to a search index. You configure the query type, top_k, and filters; Search returns document passages; and the agent's primary model produces the answer. This shorter, more directly controlled pipeline works well for a single index and predictable query patterns.

A Foundry IQ knowledge base is not limited to Foundry Agent Service. Any application, Microsoft Agent Framework implementation, or custom service that supports the Azure AI Search Knowledge Base API can call it. Foundry Agent Service simply provides native integration and a managed experience.

Foundry IQ does not replace RAG. Instead, it moves increasingly complex retrieval orchestration out of individual agents:

Direct use of Azure AI Search	Foundry IQ Knowledge Base
Typically targets one index	Can organize multiple index-based or remote knowledge sources
Usually runs one preconfigured search	Can decompose questions, select sources, run parallel queries, and iterate on retrieval
The agent processes documents and generates the answer	The knowledge base reranks and merges results while preserving citations
Retrieval configuration is distributed across agents	A domain knowledge base can be shared by multiple agents and updated independently
Cost and latency are easier to predict	Complex questions may produce better results, with additional model calls and latency

The central benefit is separation of concerns: Azure AI Search executes retrieval; the knowledge base owns domain knowledge and retrieval strategy; and the agent handles conversation, tool use, and business actions.

In Microsoft's published benchmark, agentic retrieval improved response quality by approximately 36% over traditional single-query RAG under the company's test conditions. This figure is best treated as a directional data point, not as a universal result that every workload will reproduce.

What Foundry IQ changes, therefore, is not merely the user interface. It changes who owns the retrieval strategy within a RAG system.

How Should You Choose?

If you have one mature index, straightforward questions, and strict requirements for control and low latency, continue connecting directly to an Azure AI Search index or tool.
If you need to answer complex questions across multiple sources, understand conversational context, or share the same domain knowledge among multiple agents, consider Foundry IQ.

The simplest way to think about it is: