A. Sinai Rivera

Posted on Nov 22 • Edited on Nov 26

Breaking Bedrock: What Really Happens When Your Agent Doesn’t Work

#aws #cloud #ai #beginners

TL;DR

In this lab you’re going to build a fully functional Bedrock Agent that can:

Read your documents in S3
Extract information
Execute functions
Answer questions in a structured way
Reason step by step using Amazon Nova Micro
All 100% from the console, no Studio and no manual code.

You will learn to:

Create a bucket, upload PDFs, create an Agent, add Actions, connect S3, test with real questions, and validate responses using grounding.

Estimated time: 20–30 min
Services: Bedrock Agents, S3, IAM
Cost: Very low (< $0.25)

When I decided to build a Bedrock Agent, I wasn’t trying to be fancy, I just wanted to truly understand how this thing worked instead of relying on wizards and hoping for the best.

Spoiler: The agent took its job way too seriously.

I ran into random errors, confusing IAM messages, and moments where I literally asked myself, “Why does AWS behave differently depending on the day of the week?”
But instead of giving up, I treated every issue like part of the learning process.

That’s why I wrote this: not a perfect guide, but the real story of what actually happened, what confused me, what I learned, and how you can avoid the same pain.

🧭 Quick Metadata

Field	Value
CB Category	AI/ML
AWS Services	Amazon Bedrock Agents, Amazon S3
Prerequisites	AWS account, S3 + Bedrock enabled, region `us-east-1`
Estimated costs	Low (< $0.25)
Architecture	See diagram

💡 Why It Matters

Companies are moving from “just using LLMs” to building intelligent agents with structure, reasoning, and real data access.

A well-designed agent can:

Plan
Reason
Retrieve data
Execute actions
Respond in context…without you having to manually code the entire pipeline.

That’s why understanding this now isn’t just helpful — it’s a skill that’s becoming essential.

🧰 Architecture

📄 PDFs in S3
⬇️
⚙️ Bedrock Agent Action
⬇️
🤖 Amazon Nova Micro reasoning
⬇️
💬 Final answer based on your documents

🧱 Prerequisites

AWS account with billing enabled
Amazon Bedrock enabled in us-east-1
Access to create:

S3 buckets
IAM roles / policies
Bedrock Agents

🪜 Step by Step

🛠️ STEP 1 — Create the S3 Bucket (for your documents)

In the AWS Console, search for S3
Click Create bucket
Configure:
- Bucket name: kb-agent-sina-lab
- Region: us-east-1
- Block Public Access: ON (all options)
- Versioning: OFF
- Encryption: SSE-S3 (default)
Click Create bucket

🛠️ STEP 2 — Upload the Documents to Your S3 Bucket

We’re going to upload the three PDFs you just created.

2.1 Go to the bucket

In the AWS Console, go to S3
Click your bucket: kb-agent-sina-lab (or the name you used)

2.2 Upload the files

Click Upload
Then click Add files
I selected:
- company-overview.pdf
- internal-policies.pdf
- service-pricing.pdf
Click Upload

2.3 Confirm they’re there

In your S3 bucket you should now see the documents:

🛠️ STEP 3 — Create Your Bedrock Agent

3.1 Open Amazon Bedrock

In the AWS Console search bar, type: Bedrock
In the left menu, select Agents

3.2 Create a new Agent

Click Create agent
Fill in the fields:
- Agent name: sina-agent-rag-lab
- Description: Agent that answers questions using internal documents stored in Amazon S3 via automatic actions.
Click Create
Select the Agent model. For this lab, we’ll use: Amazon Nova Micro (the free + secure model recommended by AWS for agents).
Agent instructions: "You are a corporate assistant designed to answer questions using exclusively the information stored in the company’s internal documents in Amazon S3."
Click Save

🛠️ STEP 4 — Create the Action Group That Reads from S3

4.1 Find the “Action groups” section

Click Add

Action group name: read-company-docs
Description (optional): "read documents from S3 and retrieve relevant content"
Action group type: select Define with function details
- Here you describe what the function does (name, description, S3 access type) and Bedrock auto-generates the Lambda for you.
Action group invocation: select create a new Lambda

4.2 Action group function

Name: get_company_policy
Description: "Function that searches for information within the corporate documents stored in S3. It simulates retrieving internal policies, processes, or relevant details to answer the user’s questions."

4.3 Parameters

Your agent needs to know what information to send to the Lambda function when it wants to use it.

Click Add parameter
Name: query
Description: user question looking for information
Type: String
Required: True
Click Create

🛠️ STEP 5 — Save and Prepare the Agent

Once the Action Group is created:

Click Save at the top
Then click Prepare (this validates instructions + model + actions)

🛠️ STEP 6 — Test the Agent

In this step we’re going to test the Agent by asking questions related to the company document stored in the S3 bucket.

6.1 Use Test agent on the right side

Inside your Agent view, on the right panel, in the text box, type a real question based on the PDFs you created, for example:

“What does the company say about its mission or vision?”

ERROR

We got an error – and thanks to AWS services, we can investigate that error and fix it.

🛠️ STEP 7 Fix the Response Error

We can solve this problem in different ways.

One option is to look at the detailed error and read exactly where it failed.

However, if you’re new and don’t have much experience reading code, AWS has a resource called Amazon Q.

Amazon Q is a chatbot that lets you ask questions about what you’re building and helps you debug.

Let’s use that option first.

7.1 In the error message, click Show trace

Click Trace step 1
In the upper-right corner, you’ll see the Amazon Q icon

7.2 Ask Amazon Q

In the chat box, type:

“Why is my agent saying it is unable to help me with my request?”

According to Amazon Q, the possible causes are:

Common Causes and Fixes

1. Permissions and roles

IAM permissions:We need to verify that the service role of the agent has the necessary permissions to invoke the base model.

2. Agent configuration issues

Instructions: The agent instructions may be too restrictive or unclear. We need try to simplifying them and specifying more clearly what the agent should do.
Action group names: if we’re using action groups, we need to avoid using hyphens (-) in the names; use underscores (_) instead.

🛠️ STEP 8 Let’s Fix It

We’re going to apply each recommendation from Amazon Q.

8.1 Identify the service role of your Agent

Go to Amazon Bedrock in the console
In the left menu, click Agents
Click your Agent
On the Agent screen, look for the blue Permissions link

8.2 Check the role permissions

Inside the role, go to the Permissions tab
In Permissions policies, review the attached policies
Click the policy to open it

From the screenshot, we can see that:

AmazonBedrockAgentBedrockFoundationModelPolicy_S1GQAOZ17SK
has a limited access level – read-only. That means:

❌ It cannot invoke models
❌ It cannot call Nova Micro
❌ It cannot generate responses
✔️ It can only read metadata

8.3 Let’s fix it

Click the JSON tab on the top right
Click Edit, remove the existing policy content
Paste the following policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AmazonBedrockAgentBedrockFoundationModelPolicyProd",
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream"
      ],
      "Resource": "*"
    }
  ]
}

Click Next, then Save

8.4 Test the Agent again

In your Agent, go to Action details
Select Edit in Agent builder
Click Save and exit
Click Prepare
Ask the same question again

❌ Still Error 😆😆😆😆

If this didn’t fix the problem, we move to the next likely cause.

🛠️ STEP 9 Fix Cause #2: Agent Configuration

9.1 Review and simplify Agent instructions

Go to Amazon Bedrock then Agents
Click your Agent
In the Agent panel, look for the Instructions section

Solution 1: The word “EXCLUSIVELY” is killing the Agent

What happens?
If the Agent does not find information in S3 that exactly answers the question… by design it must block the response.

And the default answer in that case is:“ Sorry, I am unable to assist you with this request.”

Update the Agent instructions

Let's replace the entire prompt with:

"You are a helpful corporate assistant. Use information from the company's internal documents stored in Amazon S3 when it is relevant. If the information is not available, provide the best possible answer based on your general reasoning and politely indicate when the exact information is not found."

Solution 2: Invalid Action Group name

The Action Group is called: read-company-docs
The hyphens - can cause failures in Bedrock Agents (2024–2025).

Change read-company-docs to read_company_docs
Click Save
Click Prepare
Ask the question again

Possible error 😆😆😆😆 — Incomplete response

Even though the Agent responded, the answer was incomplete.

What do we do now? Let’s check the trace to understand the possible causes.

In the error message, click Show trace
Click Trace step 1
Copy the code and paste it again into Amazon Q

Based on Amazon Q, these are the possible causes of the incomplete response:

1. The Agent is showing [retrieved information] because:

The instructions or internal prompts are written as if a Knowledge Base existed, but we do not have a Knowledge Base connected and we are using Action Groups, which do not need a Knowledge Base.

So any phrase inside the Agent, like:

“according to the retrieved information…”
“based on the retrieved information…”
“from the knowledge base…”
or the literal placeholder [retrieved information]

…cannot be replaced with anything, because there is no data source to fill it.

2. The Agent then returns the placeholder as-is, causing confusion.

So there is some text either in Agent instructions, Prompt templates, or Action group outputs that contains placeholders or assumptions about a Knowledge Base.

🛠️ STEP 10 — Let’s Fix That

10.1 Review the instructions again and remove any mention of:

Knowledge Base
Internal documents (as a hard requirement)
“Retrieved information”
Placeholders like [retrieved information]

For this part of the lab, the Agent should work only with Action Groups, not with a Knowledge Base.

What’s wrong here?

It mentions internal documents
It mentions S3 explicitly
It tells the Agent it can ONLY respond using “documents”
That forces the Agent to look for retrieved information → but there is no Knowledge Base

Let’s change the instruction to:

"You are a corporate assistant designed to answer questions using the information provided by the system and the available action groups. Do not assume information; always rely on the results returned by the actions."

Click Save
Click Prepare
Ask the question again

The Agent should now respond, but it's asking for the company name.
Let’s answer and see the response

I asked another question:

“What does the company say about confidentiality?”

Then:

“What does it say about the confidentiality policy?”

At this point, even though the Agent answered the question, it should have been simpler, so there’s probably still something off.

And that’s okay.For me, it’s valuable to open the process, show real learnings, and create spaces where others can also learn, contribute, and improve this path together.

🧯 Errors

Symptom	Probable cause	Fix / Lesson learned
“Sorry, I am unable to assist with this request.”	Agent role didn’t have permission for `bedrock:InvokeModel`	Update IAM policy JSON to allow model invocation.
Agent asks for clarification (e.g., “What is the company name?”)	Automatic `user__askuser` triggered by internal orchestration	Add rules in Agent instructions to avoid unnecessary clarifications.
Placeholder `[retrieved information]` appears in the response	Instructions assume a non-existing Knowledge Base	Update instructions to use only Action Groups.
Action Group doesn’t execute	Required parameter / invalid name with hyphens	Set `required = false` and rename using `_` instead of `-`.
Incomplete responses	Double `<thinking>` or invalid structure	Enforce a single `<thinking>` followed by a single `<answer>`.
Lambda not returning useful content	Auto-generated Lambda has no actual logic	Replace it with a function that reads files from the S3 bucket.

🚧 What I Would Do Differently Next Time

Even though this project didn’t end in a perfect “everything works on the first try” moment (nothing in AWS ever does), it showed me exactly where I want to improve on the next iteration.

Here’s what I would approach differently moving forward:

1. Start with the IAM permissions first

Most of my errors came from missing or half-complete IAM policies.
Next time, I’ll begin by writing a clean, minimal policy before touching the agent.

It would have saved me more than a hour of detective work.

2. Keep the instructions extremely explicit

I learned that Agents love to ask unnecessary clarifying questions.
Next time, I’ll write more direct rules to force the agent to act instead of overthinking.

Basically: Fewer “what do you mean?” and more “this is what I’m doing.”

3. Build and test the Lambda function manually first

The auto-generated Lambda is a nice idea… but useless in real scenarios.
Next time, I’ll be careful when buildings the function, test it with sample inputs, and then plug it into the Action Group.

4. Don’t rely on assumptions about the Knowledge Base

If the Agent expects a KB but you’re not using one, it will hallucinate placeholders like [retrieved information].

Next time: I willl remove all KB references from the instructions if the project doesn’t include one.

5. Document the failures as I go

This write-up felt much easier because I documented the errors while I hit them, so next time, I’ll keep a dedicated “error journal” open from the beginning.