Build Enterprise RAG on AWS with Managed Knowledge Bases, Smart Parsing, and Agentic Retrieval

Enterprise AI assistants need more than a language model. They need access to the right documents, reliable retrieval, and answers that users can verify.

Imagine asking an internal assistant:

Which team owns our ML platform, what is its annual cloud budget, and does our purchasing policy allow annual prepayment?

The answer may be spread across a team ownership document, a budget file, and an expense policy. A basic retrieval pipeline might find only one of them, leaving the answer incomplete.

Amazon Bedrock Managed Knowledge Base helps simplify enterprise Retrieval-Augmented Generation (RAG) with managed ingestion and retrieval, native data connectors, Smart Parsing, and agentic retrieval.

In this tutorial, we'll explore these capabilities and build a Python demo that retrieves evidence from multiple enterprise documents.

1. What is Amazon Bedrock Managed Knowledge Base?

RAG combines document retrieval with a generative AI model. Instead of relying only on information learned during training, an application retrieves relevant information from configured sources and uses it to generate an answer.

A typical RAG pipeline includes:

• Data source connections
• Document parsing and chunking
• Embeddings and indexing
• Retrieval and ranking
• Answer generation
• Source references and evaluation

Managing each component independently can add operational complexity.

Amazon Bedrock Managed Knowledge Base provides a managed approach to knowledge ingestion and retrieval, helping developers focus on the application rather than building every RAG component themselves.

Official documentation: Amazon Bedrock Managed Knowledge Base

2. Native data connectors

Enterprise knowledge is often distributed across different platforms.

Managed Knowledge Base supports native connectors for supported sources such as:

• Amazon S3
• Microsoft SharePoint
• Atlassian Confluence
• Google Drive
• Microsoft OneDrive
• Web Crawler

These connectors can reduce the amount of custom ingestion code required to bring enterprise content into a knowledge base.

For example, a company could retrieve engineering runbooks from S3 and business policies from a supported collaboration platform, then use that information to answer internal questions.

Connectors still require appropriate permissions, synchronization, and access-control configuration. Always confirm the currently supported sources and prerequisites in the AWS documentation.

Reference: Managed Knowledge Base announcement

3. Smart Parsing for complex documents

Not all enterprise data is plain text.

PDFs, reports, presentations, and technical documents may contain tables, diagrams, screenshots, or complex layouts. Basic text extraction can lose important relationships between these elements.

Smart Parsing helps process supported content so that it can be used more effectively during retrieval.

Potential use cases include:

• Architecture documentation
• Technical reports
• Business policies
• Documents containing tables and diagrams
• Supported scanned or visual content

Parsing quality should still be tested with representative documents. Important figures, table relationships, and diagram details may need verification.

Reference: Amazon Bedrock Managed Knowledge Base announcement

4. Agentic retrieval: Handle multi-part questions

Traditional retrieval typically searches for content relevant to a query. This works well for straightforward questions, but more complex questions may require multiple searches.

Consider:

Compare our platform budget with the purchasing policy. Is annual prepayment permitted, and what approvals are required?

The application needs to retrieve the budget, find the policy, connect the relevant conditions, and generate an answer grounded in the available evidence.

Agentic retrieval uses a foundation model to plan and refine retrieval across configured knowledge base retrievers. It can decompose a complex question into sub-queries, retrieve information iteratively, and assess whether enough evidence is available.

This can help with:

• Enterprise policy assistants
• Technical support
• Internal documentation search
• Cross-document comparisons
• Questions with multiple related parts

Agentic retrieval can improve the retrieval process, but it does not guarantee correct answers. Validate the results and citations before relying on them in production.

Reference: Use agentic retrieval to query a knowledge base

5. Demo architecture

We'll create an enterprise policy assistant using three fictional documents.

Enterprise Documents
       |
       v
Amazon S3
       |
       v
Amazon Bedrock Managed Knowledge Base
       |
       v
Document Parsing and Indexing
       |
       v
Agentic Retrieval
       |
       +--> Team ownership
       +--> Cloud budget
       +--> Expense policy
       |
       v
Generated answer + source references

Amazon Bedrock manages much of the underlying ingestion and retrieval infrastructure. The application still needs permissions, error handling, and security controls.

6. Build the Python demo

Prerequisites

You'll need:

• An AWS account
• Access to Amazon Bedrock in a supported Region
• A Managed Knowledge Base
• An S3 bucket containing the sample documents
• An appropriate IAM identity and permissions
• Python 3 and Boto3

Step 1: Create the project

enterprise-rag-demo/
├── documents/
│   ├── team-ownership.md
│   ├── cloud-budget.md
│   └── expense-policy.md
└── query_enterprise_kb.py

Step 2: Add sample documents

Create documents/team-ownership.md:

# ML Platform Team Ownership

The ML Platform team owns the internal machine learning
platform and its shared deployment infrastructure.

The team supports model deployment, platform monitoring,
and infrastructure reliability.

The team reports to the Engineering organization.

Create documents/cloud-budget.md:

# ML Platform Cloud Budget

The ML Platform team has an illustrative annual cloud
infrastructure budget of $120,000.

The budget covers approved compute, storage, networking,
monitoring, and platform infrastructure expenses.

Purchases must follow the company expense policy and
the organization's approval process.

Create documents/expense-policy.md:

# Cloud Expense and Prepayment Policy

Annual cloud prepayment may be considered when the
purchase meets procurement requirements and has the
required financial approval.

The request must include the business justification,
contract terms, expected savings, and budget confirmation.

Having sufficient budget does not automatically approve
a purchase.

The responsible finance approver must review the request
before a commitment is made.

These are fictional examples. The budget is not an AWS pricing estimate.

Step 3: Upload documents to S3

Create an S3 bucket in a supported Region and upload the three files to a prefix such as:

s3://YOUR-BUCKET-NAME/enterprise-rag-demo/

Replace YOUR-BUCKET-NAME with your bucket name.

Step 4: Create the Managed Knowledge Base

Open the Amazon Bedrock console and:

1. Navigate to Knowledge Bases.
2. Create a Managed Knowledge Base.
3. Configure the name and supported Region.
4. Select Amazon S3 as the source.
5. Configure the bucket or prefix containing your documents.
6. Review the IAM permissions and role configuration.
7. Complete creation and synchronize the data source.
8. Wait for ingestion to finish successfully.

Save the knowledge base ID. We'll use it in the Python application.

See the official setup guide for current requirements and console options.

Step 5: Install Boto3

python -m pip install --upgrade boto3

Configure credentials through an appropriate IAM identity, CLI profile, or execution role.

For a local AWS CLI setup, you can use:

aws configure

Avoid hardcoding AWS access keys in your application.

Step 6: Query the knowledge base

Create query_enterprise_kb.py:

import os
import boto3

REGION = os.getenv("AWS_REGION", "us-east-1")
KNOWLEDGE_BASE_ID = os.environ["KNOWLEDGE_BASE_ID"]

client = boto3.client(
    "bedrock-agent-runtime",
    region_name=REGION
)

question = (
    "Which team owns the ML platform, what is its annual "
    "cloud budget, and does the expense policy allow annual "
    "prepayment? Explain the conditions and identify what "
    "must happen before a commitment is made."
)

response = client.agentic_retrieve_stream(
    messages=[
        {
            "role": "user",
            "content": {
                "text": question
            }
        }
    ],
    retrievers=[
        {
            "description": "Enterprise policies and platform documents",
            "configuration": {
                "knowledgeBase": {
                    "knowledgeBaseId": KNOWLEDGE_BASE_ID
                }
            }
        }
    ],
    agenticRetrieveConfiguration={
        "foundationModelType": "MANAGED",
        "rerankingModelType": "MANAGED"
    },
    generateResponse=True
)

for event in response["stream"]:
    if "responseEvent" in event:
        print(
            event["responseEvent"].get("text", ""),
            end="",
            flush=True
        )

    elif "traceEvent" in event:
        print("\n[Retrieval trace event received]")

    elif "result" in event:
        result = event["result"]

        print("\n\n--- Retrieved Sources ---")

        for index, item in enumerate(
            result.get("results", []),
            start=1
        ):
            content = item.get("content", {})
            metadata = item.get("metadata", {})

            print(f"\nSource {index}")
            print("Retriever:", item.get("sourceRetriever"))
            print("Metadata:", metadata)
            print("Content:", content.get("text", ""))

        generated = result.get("generatedResponse", {})

        print("\n--- Citation Mappings ---")
        print(generated.get("citations", []))

The example uses managed retrieval planning and reranking. The returned documents, response text, trace events, and citation mappings depend on the service configuration and available evidence.

For production, send traces to structured logging rather than printing sensitive document contents.

API reference: AgenticRetrieveStream

Step 7: Run the demo

In PowerShell:

$env:AWS_REGION = "us-east-1"
$env:KNOWLEDGE_BASE_ID = "YOUR_KNOWLEDGE_BASE_ID"

python query_enterprise_kb.py

In Bash:

export AWS_REGION="us-east-1"
export KNOWLEDGE_BASE_ID="YOUR_KNOWLEDGE_BASE_ID"

python query_enterprise_kb.py

Use your actual knowledge base ID and the Region in which it was created. Ensure the identity running the script has the required permissions for the agentic retrieval API and configured managed functionality.

7. What should the answer contain?

Based on the sample documents, a successful response should identify:

• Team ownership: The ML Platform team owns the internal ML platform.
• Annual budget: The illustrative annual cloud infrastructure budget is $120,000.
• Prepayment policy: Annual prepayment may be considered, but is not automatically approved.
• Required conditions: The request needs a business justification, contract terms, expected savings, budget confirmation, and financial review before a commitment is made.

The response should include source references so that users can verify important claims.

This is the expected result based on the example documents, not output from a live AWS execution. Run the demo and validate the actual response and citations in your environment.

8. Production considerations

Before deploying an enterprise RAG application, consider these areas.

Access control: Ensure users retrieve only documents they are authorized to access. Test supported access-control mechanisms and identity behavior.

Data freshness: Monitor synchronization and ingestion failures.

Grounded answers: Require evidence for important claims and allow the assistant to say when information is missing.

Prompt injection: Treat retrieved content as untrusted input. A document must not be able to override your application's security rules.

Observability: Track failed queries, retrieval traces, latency, and source coverage while protecting sensitive information.

Cost and latency: Agentic retrieval may involve multiple retrieval steps. Measure whether the additional work improves results enough to justify the cost.

Evaluation: Test against representative questions with known answers and expected sources. Measure answer accuracy, citation quality, and retrieval completeness.

9. When should you use agentic retrieval?

Use agentic retrieval when questions need information from multiple sources or involve several related steps.

For straightforward lookups, conventional retrieval may be simpler and faster. Compare both approaches with a representative evaluation set before choosing the production design.

Next, you can extend this demo by adding a second supported data connector, testing documents containing tables and diagrams, comparing retrieval approaches, and creating a simple web interface for employees.

Conclusion

Enterprise RAG is about more than searching document chunks. It requires connecting knowledge, processing documents, retrieving the right evidence, and generating answers that users can verify.

Amazon Bedrock Managed Knowledge Base brings native data connectors, Smart Parsing, managed indexing, and agentic retrieval into an integrated experience.

The demo illustrates how Python can query a knowledge base, retrieve information across documents, and expose source references for validation.

My takeaway: Build AI assistants that do not just answer questions—they find evidence, connect information, and make their conclusions verifiable.

What would you build first: an internal policy assistant, a technical documentation assistant, or a customer support knowledge agent?

References

• Amazon Bedrock Managed Knowledge Base
• Managed Knowledge Base announcement
• Agentic retrieval guide
• AgenticRetrieveStream API
• Amazon Bedrock pricing